Greetings,

I really need help with bandwidth control.
My office topology is :

ADSL----Modem------mikrotik ROS 2.9.27-----Switch ---- clients (10 ws)
|
|
Ubuntu 7.10 server
w/ squid 2.6

The condition:
Modem : Ip address = 192.168.1.1
MIkrotik : Dell 600 MHZ, 256 + 128 MB (ram), 6 GB hdd
interfaces : ip address to modem : 192.168.1.2/30 (public)
ip address to switch : 192.168.10.254/24 (local)
ip address to squid : 192.168.2.1/24 (proxy)
Nat: masquerading applied to public interface
dst-nat to redirect the traffic to proxy interface
Squid : intel dual core 3,0 GHz, 256 + 512 MB ram, 40 gb hdd (30 gb allocated to /cache)
OS : Ubuntu server 7.10 (squid installed using command: apt-get install squid)

The Problem:
The only purpose i'm using the above topology is to efficiently use my 384 kbps connection. As from sources i read, caching can save my bandwidth. FIrst i tried using the built-in squid in mikrotik. It worked like magic.
After reading here and there, i found out that my dell box containing mikrotik isn't recomended to use squid due to less memory it has. Thats when i decided to add one more box only for squid and caching.

After some time using the squid box, i noticed that my bandwidth control using queue tree was not working as i wanted to. What i really wanted to is that what is cached in the squid box don't have to be limited, but each workstation that request not from squid is restricted to certain ammount of bandwidth. http connection should have the same restriction also.

what really happened is that all http request bound to port 80 (which is already dst-nated in mikrotik to squid) count for the queue tree for each workstation. even if its already cached in squid box.
here is my mangle for each workstation in my office : Each workstation applied that same mangle, and i have made an adresslist for each workstation ip.
Note that Bozz-Up is for upload traffic from bozz's pc, Bozz-MCD is for connection mark for Bozz's PC, Bozz-DDL is for bozz's direct download traffic, Bozz-PDL is for proxy download, Bozz-LocSquid is for local squid.

this mangles is made on trial and error. so there are still packet mark for local squid. that was used when i use the mikrotik's squid. then there is this proxy download that mark the packet from proxy, those DDL was suppose to mark for direct download that don't pass the squid/proxy.

i have about 60 mangle to address each pcs.

then there is the queue tree: there.

when using built-in squid in mikrotik, all that has already cached can reach client's pc in MBps bandwidth, while those that aren't cached, arrived in 192 kbps. this was achieved using the output chain. but NOw, all traffic arrive in 192 kbps to clients. Cached or not.

how can i make that kind of traffic using the external cache? so that when http traffic comes from inside squid don't have to be limited. and those that passing through the squid are limited.

Thanks before.

Thats a situation very common in some setups i have done. At MUM in Sharm El Sheikh a guy named Sunday has done a very nice presentation for the solution of this problem.

I think you can download the presentation from http://mum.mikrotik.com/presentations/EG07/sunday.pdf
If you can't i can send it to you.

hi GuJack20

This rule not working i am using 2.9.27

chain=output out-interface=lan tos=48 action=mark-packet new-packet-mark=proxy-hit passthrough=no

i am not expert of mikrotik but as per your scenario i have little bit confused .

thats right that if you are using squid box after mikrotik then it can not control your bandwidth .

so thats a great idea to deal .

terminate your whole bandwidth on Ubuntu squid --> then after NAT --> give ip to your mikrotik of ubuntu's second interface .


your no need to use mikrotik proxy.

so when mikrotik send 80 port request its automatically send hits from squid-box .

adsl--> squid-box-->mikrotik--> switch

may it help you

well let me clear my senario

3 interface card

Mkrotik
1 Lan connection xxx.xxx.xxx.1/23
2 wan connection a.b.c.112/29
3 Cache connection 10.10.10.1/24 squid box connected with this interface

squid box
1 cache connection 10.10.10.2/24 squid running 8080
2 wan connection a.b.c.113/29

/ip filter nat print
0 chain=srcnat src-address=xxx.xxx.xxx.0/23 action=masquerade

1 chain=dstnat in-interface=ether1 src-address=xxx.xxx.xxx.0/23 protocol=tcp dst-port=80 action=dst-nat to-addresses=10.10.10.2 to-ports=8080


/ip route print

0 ADC xxx.xxx.xxx.0/23 xxx.xxx.xxx.1 ether1
1 ADC 10.10.10.0/24 10.10.10.1 ether3
2 ADC xxx.xxx.xxx.xxx/29 xxx.xxx.xxx.112 ether2
3 A S 0.0.0.0/0 r xxx.xxx.xxx.111 ether2


/ip firewall mangle
0 chain=output out-interface=ether1 tos=48 action=mark-packet new-packet-mark=proxy-hit passthrough=no

mangle packets are not mark the 80 packets in this senario my all clients are running fine 80 traffic redirect to my squid box and except traffic masqurade to my wan interface.
i am refer this guide
http://wiki.mikrotik.com/wiki/Queue_wit ... _Web-Proxy
but i read another topic regarding with this
http://mum.mikrotik.com/presentations/EG07/sunday.pdf
in this intro they example given in 2.9.x

any idea mikrotik peoples ?

Reference the attached image.

Depending on your network, you can have varying configurations. All you need do is adjust the diagram to suite your configuration. In my case, RT1 is a cisco router connecting my network via satellite. MT1 and MT2 are present and SRV1 is Debian with Squid. MT1 is a bridge and MT2 is a Router. Transparent redirect is handled by the Cisco Router via WCCP V2. Both MT1 and MT2 have mangle rules marking cache hits based on TOS/DSCP modifications by the ZPH patch in SRV1's squid.

In your case, RT1 is simply your ADSL device. SRV1 is Ubuntu and you have MT1 managing bandwidth using Queue Trees. Your redirection to squid is by dst-nat from MT1. You can remove MT2 and the rest and the logic still remain same. MT1 can be bridge or router with NAT. Doesn't matter. Mangle rules based on TOS/DSCP need only match in postrouting chain.

I will throw more light on the presentation here.

The solution makes it possible for those using an external squid cache server together with a Mikrotik bandwidth manager on their network to improve client experience by allowing full access to the cache (hits only). It makes it possible to bypass limitations that would otherwise have been imposed by the bandwidth manager. When deployed correctly, the bandwidth manager only limits traffic when objects are refreshed or not residing on the hard drive. This solution is what you need to solve your problem, but for things to work, the following must be met in order listed:

(1) Run a ZPH patched Squid - Download Squid Source and patch using Marin Stavrev's ZPH at the URL in the presentation. Instructions are on Marin's site. Configure, make and install. This stage is a MUST. The default squid you installed using apt-get does not have this patch and cannot mark cache hits. Let me know if you need help getting past this stage.
(2) Create your usual squid.conf file and set the extra ZPH options that are added. See the presentation. Cache hits will be marked using the value specified in ZPH_TOS_local (value in hexadecimal)
(3) if MT1 or MT2 is version 2.9.x, your mangle will be based on TOS and in Decimal. You can follow the instructions in the presentation. Just convert the hex value in ZPH_TOS_local to DEC and you start seeing the marks.
(4) if MT1 or MT2 is version 3.x and above, your mangle is based on DSCP and in Decimal. I explained how to convert TOS to DSCP in the presentation.
(5) Finally, create queue using the new packet mark and make bandwidth unlimited or limited as you want! Unlike mangle, TOS/DSCP marks are permanent and passes from network to network unless changed. So you can mark on as many Mikrotik Routers/Bridges in the chain.

Hope this helps.

hello sunday idajili

thanks for your guidness the following steps are complete !

1 . patch my external squid cache with ZPH
2. add ZPH setting in squid.conf

step 3 are not working i can see any packets in my mangle here is mangle rule
/ip firewall mangle print
chain=output out-interface=lan tos=48 action=mark-packet new-packet-mark=proxy-hit passthrough=no

i read the article on wiki says ! ZPH only works with routeros 3.0

waiting your positive response

step 3 are not working i can see any packets in my mangle here is mangle rule
/ip firewall mangle print
chain=output out-interface=lan tos=48 action=mark-packet new-packet-mark=proxy-hit passthrough=no

i read the article on wiki says ! ZPH only works with routeros 3.0

waiting your positive response

"Output" will match packets leaving processes within the router. Since squid is external, you must look for the hits in another chain. In 2.9.x, use

goldclick , i tried to call you so many times , i just need to know how can i apply the patch to squid ? i have fedora c8 ..

goldclick , i tried to call you so many times , i just need to know how can i apply the patch to squid ? i have fedora c8 ..

Unpack the version of SQUID supported by the patch (I have tested up to 2.6 STABLE16). Change directory into its source tree directory and issue:
Am assuming, the patch is one step above the squid source directory
Configure, Make and install Squid.

SQUID configuration:
The extension of ZPH TOS marking patch has these knobs you can tweak:

zph_tos_local (integer)
Allows you to select a TOS/Diffserv value to mark local hits.
Default: 0 (disabled)

zph_tos_peer (integer)
Allows you to select a TOS/Diffserv value to mark peer hits (siblings and parent).
Default: 0 (disabled)

zph_tos_parent (on|off)
Set this to off if you want only sibling hits to be marked.
If set to on (default), parent hits are being marked too.
Default: on

zph_preserve_miss_tos on (on|off)
If set to on (default), any HTTP response towards clients will have the TOS value of the response comming from the remote server, masked with the value of zph_preserve_miss_tos_mask. For this to work correctly, you will need to patch your linux kernel with the TOS preserving ZPH patch.
Default: on

zph_preserve_miss_tos_mask (0-255)
Allows you to mask certain bits in the TOS received from the remote server, before copying the value to the TOS send towards clients.
Default: 255 (TOS from server is not changed)

ok , i think i have some conflict , can I remove the existing squid then i'll install the new one ?
cause i think there is 2 versions now are installed !!!

goldclick , as in your presentation in mum , it said to put zph_tos_loca 0x30 , then marking dscp=12 ..
now i have the patched squid running , using the following rule in the mangle : i can see its counter is counting but its not counting all the hits exactly , bay making a test on a clip of 1Mbyte , the counter is not increasing by the same amount ..
even postrouting chain doesnt work !!

Hello goldclick,
I am using the squid-2.6.STABLE16 and zph squid-2.6.16-ToS_Hit_ToS_Preserve.patch with mikrotik 2.6.51, but I apply the rule /ip firewall mangle add chain=postrouting action=mark-packet new-packet-mark=proxy-hit passthrough=no tos=48 comment="Proxy Cache Hits Mark" disabled=no

The mangle rule no count. what I can do? Need I use zph squid-2.6.STABLE2-ToS_Hit_ToS_Preserve.patch?

Thanks man.

did you tried prerouting chain instead of postrouting?

Hello Chupaka, thanks.
Yes I tried this, but not work.
What happend here?

Thanks man.

hmmm... maybe 'input'? if it works on squld at all...

No work yet.
Thanks

User of MTK 2.9.27, I got only one thing to say: If you like it, buy it. Programmers deserve it.
Why don't you use mtk internal webproxy?
por que no usas el webproxy interno de MTK
eso no +

saludos

Oscar

Because no work very good, I want to use DNAT. I used the squid version 2.6.STABLE13 with patch for this and the hits is counted now. Its very works.

Thanks all.

I have ubuntu 8.04 and installed is squid-2.6.STABLE18 Can anyone help me hot to patch.

Hi, I was working on your guide I made all steps except one, I've downloaded de zph patch to squid3stable8 It comes with two patch one for squid source and one to the linux kernel but I see the linux kernel version do not match with my linux kernel version even browsing in the reposotories I dind't find a match kernel my server is centos 5.2 x64 it's running squid 3 stable 8 with zph patch.

What linux distro I must use?
I really need patch the kernel it's mandatory?

If I use PPPoE or hotspot how to configure it, tried to patch squid and did it succesfully but after installing it I can not access internet something goes wrong, I says proxy refuzed connections. I am trying to set up squid on ubuntu 8.04. Do I need to patch kernel too in order to work? Does any one has squid patched and ripacked in .tar.gz if yes can any send the link?

Thank in advance!!!

I need help to set up my Mikrotik with squid. My configuration is:
Mikrotik Interfaces:
WAN 192.168.0.1
LAN 10.0.0.0/24 for PPPoE
SQUID 192.168.1.2

Squid: Lan 192.168.1.1

I need help how to use squid as transparent proxy what queues to create, what mangle rule te create.

I need help about this. From Mikrotik I can ping 192.168.1.1 but from my PPPoE clients I can not 192.168.1.1 but I can ping 192.168.1.2.

Did you use delay pools?