[BUG 3.x] All Hotspot features based on WebProxy fail

ebruni · Tue Jun 24, 2008 12:19 pm

Starting from 3.x we cannot use Walled Garden and trasparent proxy because they are based on Web Proxy and Web Proxy seems bugged.

We have a walled garden to our remote login page

/ip hotspot walled-garden 
add action=allow comment="" disabled=no dst-host=*.login.wifi.micso.it

and in 2.x all works fine. In 3.x I see this strange beaviuor in web proxy connection during an attempt to access login page

[admin@micso] /ip proxy connections> print
Flags: S - server, C - client 
 #   SRC-ADDRESS     DST-ADDRESS     PROTOCOL  STATE      TX-BYTES   RX-BYTES  
 0 C 172.21.7.253    195.32.69.5     HTTP/1.1  waiting    0          561       
 1 S 195.32.69.5     172.21.7.253    HTTP/1.1  tx-header  0          0

but no connection is made as you can see by the counter and by a tcpdump on remote host.
Adding a Walled Garder Ip entry is a workaround but clearly it's not an acceptable solution.
Same problem with trasparent proxy on user profiles. When enabled presents same behaviour.
Hoping someone solves this annoying bug.

BuccaNET · Tue Jun 24, 2008 12:47 pm

I had the same issue. I had to disable web caching to get the hotspot walled garden working. What port are you using for web cache? I think the hotspot does something on 8080. Not sure how it all works yet. Still trying to figure it out.

Cory

ebruni · Tue Jun 24, 2008 5:52 pm

This is not related to problem however...

When walled garden by hostname or transparent proxy are enabled on user profile a (hidden) web proxy is automatically used also if web proxy is disable.
In fact, my (disable) web proxy configuration is

/ip proxy 
set always-from-cache=no cache-administrator="webmaster" cache-drive=system \
    cache-hit-dscp=4 cache-on-disk=no enabled=no max-cache-size=unlimited \
    max-client-connections=600 max-fresh-time=3d max-server-connections=600 \
    parent-proxy=0.0.0.0 parent-proxy-port=0 port=8080 \
    serialize-connections=no src-address=0.0.0.0

but when walled garder or trasparent proxy are enabled, I see web proxy connection as you can see in my first post. Standard web proxy listen on port 8080 while, I suppose, a hidden web proxy runs on port 64874
owing to this dynamic nat configuration

chain=hs-unauth action=redirect to-ports=64874 dst-port=80 protocol=tcp

gmsmstr · Tue Jun 24, 2008 6:08 pm

I have heard lots of 3.x proxy bugs, but have not experienced this yet. We have over 30 systems from small T1 links, to 20-30 meg connections with 1000+ customers behind it. All have not had any issues with web proxy.

Has someone identified what the symptoms are? What is done to correct the issue? Have we sent Mikrotik supouts to support@mikrotik.com? Have they confirmed this issue? Is it on RouterBoard systems only, or all RouterOS devices?

ebruni · Tue Jun 24, 2008 6:28 pm

Has someone identified what the symptoms are? What is done to correct the issue? Have we sent Mikrotik supouts to support@mikrotik.com? Have they confirmed this issue? Is it on RouterBoard systems only, or all RouterOS devices?

I opened a ticket today and attached a supout.rif.
My hardware is a RB433 and I found problem on v3.8 and v3.10. On this hardware I have currently only a customer, me

but tested with a WinXp, MacOs and WM6 clients.

Of course, I will not upgrade my 150 hotspots untill walled garden will work again

changeip · Tue Jun 24, 2008 7:52 pm

i ran into this problem when running the hotspot on a bridge, or from clients that couldn't do 1500 bytes. Not sure if its related. In your tcpdump did you see packets too large for the connection being sent but not received on the other end ? We would see the login redirect, the beginning of the http load, and then the packet was too large and the bridge failed to pass it.

Sam

gmsmstr · Tue Jun 24, 2008 10:51 pm

ebrun:

So on a 433, what are you doing as far as web-proxy? is this just he hotspot, or actually doing caching of data? In most of my systems its a combo, some doing caching and some are just doing hot spots..

ebruni · Wed Jun 25, 2008 11:28 am

So on a 433, what are you doing as far as web-proxy? is this just he hotspot, or actually doing caching of data? In most of my systems its a combo, some doing caching and some are just doing hot spots..

As you can see in my previous post I have web-proxy disabled. So I use only the hidden web-proxy on 64874, used by MT for walled garder URL based and trasparent proxy.

ebruni · Wed Jun 25, 2008 11:32 am

In your tcpdump did you see packets too large for the connection being sent but not received on the other end ?

I don't believe this is the problem because proxy connection shows no RX/TX data trasmission on reply socket #1

[admin@micso] /ip proxy connections> print
Flags: S - server, C - client
#   SRC-ADDRESS     DST-ADDRESS     PROTOCOL  STATE      TX-BYTES   RX-BYTES 
0 C 172.21.7.253    195.32.69.5     HTTP/1.1  waiting    0          561       
1 S 195.32.69.5     172.21.7.253    HTTP/1.1  tx-header  0          0

ebruni · Mon Jul 07, 2008 5:21 pm

I solved this problem.

To masquerade hotspot connection I used this rule

/ip firewall mangle
add chain=srcnat out-interface=ether1 src-address=!172.21.7.1 \
    action=masquerade comment="" disabled=no

where 172.21.7.1/24 is the mikrotik ip address on hotspot side. However in routeros < 3.0 this works maybe because webproxy uses the external ip address to process hotspot users requests.

In 3.0 it seems that this is changed and now webproxy uses its hotspot ip address and then just 172.21.7.1 that, because of the rules above, it's not masquerade and so unable to be routed in Internet.

Simply changing rule in this way

/ip firewall mangle
add chain=srcnat out-interface=ether1 src-address=172.21.7.0/24 \
    action=masquerade comment="" disabled=no

all now works magically.

Be aware...in routeros < 3.0 webproxy packets have external interface source ip. In routeros >=3.0 it's have internal interface source ip.

magnavox · Mon Jul 07, 2008 5:32 pm

info: why src-address=!172.21.7.1 in your NAT rules?

ebruni · Tue Jul 08, 2008 1:28 pm

info: why src-address=!172.21.7.1 in your NAT rules?

We don't remember

. Maybe the idea was to masquerade whatever goes out from external interface which isn't the MK itself without thinking that if Mk goes out with its internal IP, it must be masquerade too.

Our old rule is obviously wrong, however changes in webproxy behaviour could have other side effects and it could be the responsible of other hotspot and webproxy problems I saw in other threads.

cramerit · Thu Jul 17, 2008 5:58 pm

We are having a similar problem. We just upgraded our 2.9.49 hotspot to 3.11 and are receiving some strange behavior. This system had been working for a couple years prior, then after upgrading it to 3.11, attempting to access our hotspot homepage through the walled garden results in half loaded web pages. We tried downgrading to 3.10 and had the same results.

A little more detail on what is happening:

1. When a user connects to the hotspot, we immediately redirect them to a page, register.cramerit.net, through the walled garden. (We have walled garden rules added for this - see the export below).

2. We frame in the login page to the router with an iframe.

The redirect from rlogin.html is working fine, but the destination page will not fully load. The html gets truncated. If you repeatedly try to reload the page, it will get varying degrees of truncation and sometimes will mostly load, but it is very inconsistent.

If I disable hotspot, the page loads right up with no problem. Also, if you can mange to authenticate with hotspot running, all other web pages load fine.

The only page having trouble is the hotspot home page that is accessed through the walled garden.

There is clearly a problem here - can anyone tell me what is wrong?

# jul/16/2008 20:59:46 by RouterOS 3.10
/ip hotspot profile
set default dns-name="" hotspot-address=0.0.0.0 html-directory="" \
    http-cookie-lifetime=3d http-proxy=0.0.0.0:0 login-by=cookie,http-chap \
    name=default rate-limit="" smtp-server=0.0.0.0 split-user-domain=no \
    use-radius=no
add dns-name="" hotspot-address=12.201.133.254 \
    html-directory=hotspot http-cookie-lifetime=8w4d http-proxy=0.0.0.0:0 \
    login-by=cookie,http-chap,http-pap name=hsprof1 nas-port-type=\
    wireless-802.11 radius-accounting=yes radius-default-domain="" \
    radius-interim-update=1m radius-location-id="" radius-location-name="" \
    rate-limit="" smtp-server=0.0.0.0 split-user-domain=no use-radius=yes
/ip hotspot
add disabled=yes idle-timeout=none interface=Hotspot keepalive-timeout=none \
    name=server1 profile=hsprof1
add disabled=yes idle-timeout=none interface="ether 2 - Students" \
    keepalive-timeout=none name=newserver profile=hsprof1
/ip hotspot user profile
set default advertise=no idle-timeout=8w4d keepalive-timeout=5m name=default \
    open-status-page=always shared-users=1 status-autorefresh=1m \
    transparent-proxy=yes
/ip hotspot ip-binding
add address=10.2.7.0/24 comment="" disabled=no type=bypassed
add address=10.0.0.0/8 comment="" disabled=yes server=server1 type=bypassed
add address=12.201.133.10 comment="" disabled=yes server=server1 type=\
    bypassed
add address=12.201.132.125 comment="" disabled=yes type=bypassed
add address=12.201.132.226 comment="" disabled=no type=bypassed
/ip hotspot service-port
set ftp disabled=no ports=21
/ip hotspot user
add comment="" disabled=no name=admin password=jazkot&76 profile=default
/ip hotspot walled-garden
add action=allow comment="" disabled=no dst-host=register.cramerit.net \
    dst-port=80 server=newserver
add action=allow comment="" disabled=no dst-host=register.cramerit.net \
    dst-port=443 server=newserver
add action=allow comment="" disabled=no dst-host=12.201.197.1 dst-port=80 \
    server=newserver
add action=allow comment="" disabled=no dst-host=12.201.197.1 dst-port=443 \
    server=newserver

SurferTim · Thu Jul 17, 2008 10:59 pm

I found any attempt to use the standard walled garden list with SSL pages will cause problems. I am using 3.11 on a RB333 and had a serious problem trying to get authorize.net working. It seems to be a problem with the dns, since my cure was to use the
/ip hotspot walled-garden ip

All secure sites used in the walled garden I put in here after doing an nslookup. For authorize.net, I used:
/ip hotspot walled-garden ip add dst-host=secure.authorize.net dst-address=64.94.118.32/31 action=accept
/ip hotspot walled-garden ip add dst-host=verify.authorize.net dst-address=64.94.118.132 action=accept

Now all seems to be back to normal. Makes me uneasy tho...

dsobin · Thu Jul 17, 2008 11:35 pm

I also suspected that the walled garden was having trouble using host names, so I substituted ip's using walled garden ip. It didn't change anything. My failed pages were also https, so I think the problem is related to transparent proxy of redirected pages that are in the walled garden list.

However, my haunted node in Florida is now back up. It suffered from the problem of big pages from the walled garden refused to load, but small test pages (all via https) did load.

I rebooted a Linksys router that is in front of my hotspot, and all is now working. No one one else behind the Linksys router complained about anything, but the MT hotspot was very confused, but only on https walled garden pages.

I still don't know what's causing this. In this particular case, there is some evidence that MT was not at fault, but I've had other MT hotspots that are not behind another router that did the same thing.

I should also point out that I have over 10 hotspots that have https walled garden entries entered with host name (vs IP address) and all of them work fine.

If one of my hotspots that are not behind another router gets into this mode, I'll try the trick of using walled-garden IP vs walled-garden host-name to see if it makes a difference. For now, my hotspots are working ok with https host-name walled garden entries.

omega-00 · Fri Jul 18, 2008 1:28 pm

Just an FYI: As far as I was aware, walled garden was originally supposed to be for all web-based pages (eg http and https) while walled garden IP list was supposed to be for *any* connections.

pjulian · Fri Aug 08, 2008 6:23 am

I have just tried the adding of our login servers to the IP list, we will see what happens.

So what I want to know though, is how come when you have transparent proxy disabled in all user profiles and everywhere you can select it, that the user still gets a proxy message when the redirect is happening, any ideas ?

Regards
Paul

omega-00 · Tue Aug 12, 2008 2:54 am

because all the traffic thats captured by the hotspot has to initially pass via the proxy/webserver built into the mikrotik to get the login page.

Also, all walled garden traffic (while not logged in) passes through this inbuilt proxy also.
There is no way to completely turn it off, only reduce the load on it so it doesn't crash.

cramerit · Tue Aug 12, 2008 3:31 am

What about the "HTTP Proxy" setting in the Server Profiles where you can specify an external proxy server to use?

If we did set up an external proxy server and configured hotspot to use it, would this completely bypass the internal proxy server?

[BUG 3.x] All Hotspot features based on WebProxy fail

[BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

[SOLVED] [BUG 3.x] All Hotspot features based on WebProxy fa

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Re: [BUG 3.x] All Hotspot features based on WebProxy fail

Who is online