Transparent proxy not caching that well

glemme · Sun Dec 04, 2011 9:15 am

Checking my proxy stats and it gives me this:

[admin@MikroTik] /ip proxy> monitor 
                 status: running
                 uptime: 3d21h41m44s
               requests: 88534
                   hits: 2340
             cache-used: 8186KiB
         total-ram-used: 4830KiB
  received-from-servers: 3489498KiB
        sent-to-clients: 3545171KiB
   hits-sent-to-clients: 44504KiB

my settings are:

[admin@MikroTik] /ip proxy> print 
                 enabled: yes
             src-address: 0.0.0.0
                    port: 8080
            parent-proxy: 0.0.0.0
       parent-proxy-port: 0
     cache-administrator: me
          max-cache-size: 25000KiB
           cache-on-disk: yes
  max-client-connections: 600
  max-server-connections: 600
          max-fresh-time: 3d
   serialize-connections: no
       always-from-cache: no
          cache-hit-dscp: 4
             cache-drive: system

Is there something I should change? Only 45 mb of 3.5gb has been sent from the proxy. Feel like it should be better?
I'm on a rb450g. No extra drive (might buy a micro sd just to add some storage for the proxy)

Thanks for the help.

ojsa · Sun Dec 04, 2011 11:43 am

Could you provide the firewall rules that fetches the traffic and redirects to your transparent proxy?

glemme · Sun Dec 04, 2011 1:05 pm

[admin@MikroTik] /ip firewall filter> print
11   chain=input action=accept protocol=tcp in-interface=en2-lo dst-port=80
18   chain=input action=accept protocol=tcp in-interface=en2-lo dst-port=8080

and

[admin@MikroTik] /ip firewall nat> print 
12   ;;; transparent proxy config
     chain=dstnat action=dst-nat to-addresses=10.0.10.1 to-ports=8080 protocol=tcp dst-port=80

ojsa · Sun Dec 04, 2011 2:25 pm

And do you see the packet count increase on those rules when you try too surfe the internet?

glemme · Sun Dec 04, 2011 3:04 pm

Yes, I see things hitting the rules. But not that much traffic trough them.

On the FW rule:
Bytes: 184.9 MiB
Packets: 2 112 331

On the NAT rule:
Bytes: 2625.7 KiB
Packets: 41 901

Looking at the Web Proxy under Connections I see 21 rows. Under Cache Contents there are 663 items cached.

fewi · Sun Dec 04, 2011 3:20 pm

Why should it be better, though? That seems about right.

Caching proxies achieve huge cache rates when they're used in front of web servers, where there are 10,000 resources to request and 8,000 of them are static and can be served from cache.

For an ISP there just isn't much to cache, realistically, particularly if you only give it 25MB to store on the disk. It keeps caching images and javascript resources and css files only to throw them back out FIFO style as it serves more content. So you get to satisfy a couple of hits while two customers happen to hit the same site, and then everything gets thrown into the bit bucket. If you want to see better hit rates you need to give the proxy WAY more storage than 25MB. Something to the tune of a couple of GB worth. And you still won't see huge hit rates because your customers have the entire web to choose from, whereas a proxy in front of a server cluster has a very limited scope of work. Whenever you read huge proxy hit rates it's for those server clusters, not general web traffic.

aacable · Sun Dec 04, 2011 4:01 pm

Is there something I should change? Only 45 mb of 3.5gb has been sent from the proxy. Feel like it should be better?
I'm on a rb450g. No extra drive (might buy a micro sd just to add some storage for the proxy)
Thanks for the help.

To get maximum cache hit, I suggest you to use some external proxy, like SQUID proxy server, If you well configure SQUID , you can get upto 30-40% Cache Hit Ratio. Mikrotik have a very basic proxy server. There is not much that you can modify in it. Using squid, you can very well customize it as per your requirements.

Muhammad · Sun Dec 04, 2011 6:49 pm

Is there something I should change? Only 45 mb of 3.5gb has been sent from the proxy. Feel like it should be better?
I'm on a rb450g. No extra drive (might buy a micro sd just to add some storage for the proxy)
Thanks for the help.
To get maximum cache hit, I suggest you to use some external proxy, like SQUID proxy server, If you well configure SQUID , you can get upto 30-40% Cache Hit Ratio. Mikrotik have a very basic proxy server. There is not much that you can modify in it. Using squid, you can very well customize it as per your requirements.

You can get 30-40% cache hit ratio in Mikrotik also, configure your Mikrotik Web-proxy with some advance configuration,

Right now that guy just need to assign some more Space for cache
max-cache-size: 25000KiB (is equal to 25MB only or less)

glemme · Mon Dec 05, 2011 1:30 am

You can get 30-40% cache hit ratio in Mikrotik also, configure your Mikrotik Web-proxy with some advance configuration,

Right now that guy just need to assign some more Space for cache
max-cache-size: 25000KiB (is equal to 25MB only or less)

Hmm, yeah, that will be one thing I'm changing, going to get a 4gb micro sd card today to use it instead. That should be better.

What other advanced configurations are you talking about?

This is not an ISP setup, it is my apartment. Just because it can be done.

Transparent proxy not caching that well

Transparent proxy not caching that well

Re: Transparent proxy not caching that well

Re: Transparent proxy not caching that well

Re: Transparent proxy not caching that well

Re: Transparent proxy not caching that well

Re: Transparent proxy not caching that well

Re: Transparent proxy not caching that well

Re: Transparent proxy not caching that well

Re: Transparent proxy not caching that well