MikroTik

Dear forum members,

I would like to mark traffic that goes to Amazon AWS, e.g. github-production-release-asset-2e65be.s3.amazonaws.com, so that routing goes throw my second WAN.
Usually I use an address list and then mark the traffic via a mangle rule.

The problem with Amazon AWS is that the IP-address changes very quickly, so that the address list is not updated quick enough.

Any ideas, how to mangle that traffic?

Thanks and best regards
dksoft

How exactly you do it? Do you mark routing directly based on address list? That wouldn't work well if it changes very often. But if you mark connections based on address list and then mark routing based on connection marks, it should work.

The IP ranges are published at https://ip-ranges.amazonaws.com/ip-ranges.json, just script something to update the address list.

How exactly you do it?

This works, I hope it's the Mikrotik way:

The IP ranges are published at https://ip-ranges.amazonaws.com/ip-ranges.json, just script something to update the address list.

That's the solution, thank you. I removed duplicates and added 2.766 IPv4 addresses to my address list.
Deutsche Telekom has a very poor peering to github hosted on Amazon AWS. Without routing the traffic over my second WAN, I download at 24 KB/s over FTTH.

There is one drawback: As ROS 6 does no IPv6 source routing, I can no longer use IPv6 in my network till ROS 7 is released.

As it's not clear from your post, do you know that you can use address-list items with an fqdn as an address, which is then kept up-to-date automatically?

The TCP session cannot survive a change of the remote address, so even if the fqdns are migrating between IPs fast, tracking the current addresses using such address list, marking the TCP connection just once when it is initiated, and using the connection-mark to assign the routing-mark as suggested by @Sob should be a maintenance-free solution in terms that you wouldn't need to watch for changes on the list published by Amazon.

The only question is how many individual fqdns we talk about here. If units, I'd say the above is better; if tens or more and the list of these fqdns is constantly changing, you'd have to use a periodically spawned script to learn the individual fqdns from the dns cache and add them to the address-list, so automated synchronisation of the list from AWS web may be equally complex.

The IP ranges are published at https://ip-ranges.amazonaws.com/ip-ranges.json, just script something to update the address list.

I knocked this together in Python to generate address-lists for specific AWS regions. I have some stuff running in eu-west-1 that needs to connect to a VM at home. I ran this, redirected the output to a file and then pasted it in to a Winbox terminal window. Worked a treat. Hope it might be of use to someone else.

https://gist.github.com/marria05/da183d ... 75f8eab11c

Hi all!

Who can help and post a script that will import the Amazon IP address list to the MirkoTik?

Hi EIKA,

here is s Powershell script which fetches the latest file from Amazon and converts it into a importable file for Mikrotik

right now its filtered on service "AMAZON" and region "eu-west-1" but you can remove the "| Where-Object { $_.service -eq "AMAZON" -and $_.region -match "eu-west-1" };" to get the full list.

After execution you will get a file in C:\temp\test.txt which can be uploaded to the router and import with the following command "/import file-name=amazon_aws.txt"

$Mikrotik_ListName = "AmazonAWS_IPS";
$ExportFilePath = "C:\temp\amazon_aws.txt";
$Response = Invoke-WebRequest -Uri "https://ip-ranges.amazonaws.com/ip-ranges.json" | ConvertFrom-Json;
$Filtered = $Response.prefixes | Where-Object { $_.service -eq "AMAZON" -and $_.region -match "eu-west-1" };
foreach ($ip in $Filtered)
{
write-host "Service: $($ip.service) CIDR: $($ip.ip_prefix)";
Add-Content $ExportFilePath "/ip firewall address-list add list=$($Mikrotik_ListName) address=$($ip.ip_prefix)";
}

Hope this will help

Best regards

Alex

Hi EIKA,

here is s Powershell script which fetches the latest file from Amazon and converts it into a importable file for Mikrotik

Yes, I know how to do that one time. Text editor and a few regexps, then copy and paste to the MikroTik console.
But I am looking for an automated scripted ROS solution like this (CloudFlare in this example).
I can write it by myself, but because I am not an experienced coder and have a full-time job for 80 hours a week, I don't have the time to do that. And because Mikrotik is not a business for me but a hobby, I don't have enough motivation for it.
As a result, I will wait until someone does that for the whole world and the MikroTik community. LOL.

Parsing such large JSON (1.6MB) in ROS script won't work anyway due to variables size limitation. If you want all on single device solution you will need to use container for that if your device supports it.
You can build container image with minimal Alpine Linux image + depended packages for script language which you want to use for fetching, parsing and generating rsc file for import and with actual script in it that performs operation. It can be also done with shell script in container (eg. bash) and jq command line tool for JSON parsing, then you need curl and jq along with script in it.
When container is created with such image you can create scheduled ROS script that starts container and watches until is stopped (if container CMD in not running in background (as daemon) it stops after is finished executing), when is stopped, it can import generated rsc file that updates such address list. That rsc file for import needs to be generated in mounted dir for container so that can be possible to access it from ROS scheduled script.
Since you mentioned you don't have time just to write ROS script that will perform requested operation (if even would be possible) I doubt you will have for all mentioned above, but I write proposal anyway to have it if you decide to do it after all.

Thanks for pointing out the out-of-resources possible issue. But I run CHR with 2 EPYC cores, 4-6GB of RAM and 40–80GB of HDD. So, this job would be like a joke for them.

See viewtopic.php?t=197754

Hi EIKA,

I have now set up a daily Python script that analyzes and uploads the AWS IP ranges daily at midnight.

https://github.com/amader/mikrotik_aws_iplist

there are complete files for ipv4 and ipv6 and in a subfolder "regions" they are divided into regions.

But please be careful with third party configuration files.

Kind regards
Alex