Community discussions

MikroTik App
  4. RouterOS
  5. General

Traffic flow bug

Post Reply
 
rucevzhuru
newbie
Topic Author
Posts: 43
Joined: Fri May 25, 2007 6:57 pm

Traffic flow bug

Sat May 02, 2009 6:57 pm

I tried several linux flows capturers and none of them worked with Mikrotik Traffic flows.

Router: ROS 3.10 (IP 192.168.6.1)
Server: Debian squeeze (connected to router via ethernet cable)

Both PC's have almost unused cpu power.

Software: flowd, pmacct, flow-tools.

For example log from flow-tools:
Code: Select all
May  2 17:47:29 ali flow-capture[4393]: ftpdu_seq_check(): src_ip=192.168.6.1 dst_ip=192.168.6.2 d_version=5 expecting=13785 received=13873 lost=88
May  2 17:47:29 ali flow-capture[4393]: ftpdu_seq_check(): src_ip=192.168.6.1 dst_ip=192.168.6.2 d_version=5 expecting=13903 received=13873 lost=4294967265
May  2 17:47:29 ali flow-capture[4393]: ftpdu_seq_check(): src_ip=192.168.6.1 dst_ip=192.168.6.2 d_version=5 expecting=13903 received=13873 lost=4294967265
May  2 17:47:29 ali flow-capture[4393]: ftpdu_seq_check(): src_ip=192.168.6.1 dst_ip=192.168.6.2 d_version=5 expecting=13903 received=13873 lost=4294967265
May  2 17:47:29 ali flow-capture[4393]: ftpdu_seq_check(): src_ip=192.168.6.1 dst_ip=192.168.6.2 d_version=5 expecting=13903 received=13873 lost=4294967265
May  2 17:47:29 ali flow-capture[4393]: ftpdu_verify(): src_ip=192.168.6.1 failed.
May  2 17:47:29 ali flow-capture[4393]: ftpdu_verify(): src_ip=192.168.6.1 failed.
May  2 17:47:29 ali flow-capture[4393]: ftpdu_verify(): src_ip=192.168.6.1 failed.
May  2 17:47:29 ali flow-capture[4393]: ftpdu_verify(): src_ip=192.168.6.1 failed.
May  2 17:47:29 ali flow-capture[4393]: ftpdu_verify(): src_ip=192.168.6.1 failed.
May  2 17:47:29 ali flow-capture[4393]: ftpdu_verify(): src_ip=192.168.6.1 failed.
I saw several similar bugs in the forum and there aren't answers. Have somebody seen similar behaviour?
Thanks
Top
 
JJCinAZ
Member
Member
Posts: 475
Joined: Fri Oct 22, 2004 8:03 am
Location: Tucson, AZ

Re: Traffic flow bug

Mon May 04, 2009 7:10 pm

I've never had those issues with NTOP (http://www.ntop.org/overview.html) or commercial tools from IPSwitch and others. I can't vouch for flowd, pmacct, flow-tools, etc.
Top
 
changeip
Forum Guru
Forum Guru
Posts: 3833
Joined: Fri May 28, 2004 5:22 pm

Re: Traffic flow bug

Mon May 04, 2009 7:48 pm

i used 2.9.x versions with flowd, never tried 3.x...
Top
 
rucevzhuru
newbie
Topic Author
Posts: 43
Joined: Fri May 25, 2007 6:57 pm

Re: Traffic flow bug

Fri May 08, 2009 10:24 pm

I've never had those issues with NTOP (http://www.ntop.org/overview.html) or commercial tools from IPSwitch and others. I can't vouch for flowd, pmacct, flow-tools, etc.
Ntop seems to work with flows version 5 but there are some errors with version 9
Code: Select all
Pá 8. květen 2009, 21:20:45 CEST  **WARNING** Template len mismatch [tot_len=1230][flow_len=1234]
Pá 8. květen 2009, 21:20:45 CEST  **WARNING** Template len mismatch [tot_len=1148][flow_len=1152]
Pá 8. květen 2009, 21:20:47 CEST  **WARNING** Template len mismatch [tot_len=1230][flow_len=1234]
Pá 8. květen 2009, 21:20:47 CEST  **WARNING** Template len mismatch [tot_len=1230][flow_len=1234]
Pá 8. květen 2009, 21:20:47 CEST  **WARNING** Template len mismatch [tot_len=1230][flow_len=1234]
Pá 8. květen 2009, 21:20:47 CEST  **WARNING** Template len mismatch [tot_len=820][flow_len=824]
Pá 8. květen 2009, 21:20:49 CEST  **WARNING** Template len mismatch [tot_len=1230][flow_len=1234]
Pá 8. květen 2009, 21:20:49 CEST  **WARNING** Template len mismatch [tot_len=1230][flow_len=1234]
Pá 8. květen 2009, 21:20:49 CEST  **WARNING** Template len mismatch [tot_len=1230][flow_len=1234]
Pá 8. květen 2009, 21:20:49 CEST  **WARNING** Template len mismatch [tot_len=410][flow_len=414]
Pá 8. květen 2009, 21:20:51 CEST  **WARNING** Template len mismatch [tot_len=1230][flow_len=1234]
Pá 8. květen 2009, 21:20:51 CEST  **WARNING** Template len mismatch [tot_len=1230][flow_len=1234]
Pá 8. květen 2009, 21:20:51 CEST  **WARNING** Template len mismatch [tot_len=1230][flow_len=1234]
Pá 8. květen 2009, 21:20:51 CEST  **WARNING** Template len mismatch [tot_len=1230][flow_len=1234]
Pá 8. květen 2009, 21:20:51 CEST  **WARNING** Template len mismatch [tot_len=369][flow_len=373]
i used 2.9.x versions with flowd, never tried 3.x...
We have some routers with 2.9.x and i can confirm that 2.9.x works perfectly but 3.x doesnt.
Top
 
changeip
Forum Guru
Forum Guru
Posts: 3833
Joined: Fri May 28, 2004 5:22 pm

Re: Traffic flow bug

Fri May 08, 2009 11:29 pm

please email support and work with them to fix it. It looks like all those packets are 4 bytes too big.
Top
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:
Contact Chupaka

Re: Traffic flow bug

Sat May 09, 2009 5:35 pm

why do you need v9?
Top
 
rucevzhuru
newbie
Topic Author
Posts: 43
Joined: Fri May 25, 2007 6:57 pm

Re: Traffic flow bug

Sat May 16, 2009 1:59 am

I thought there are more informations within ipflows v9 but there aren't so you are right, v5 is enough for me.

why do you need v9?
Top
 
savage
Forum Guru
Forum Guru
Posts: 1264
Joined: Mon Oct 18, 2004 12:07 am
Location: Cape Town, South Africa
Contact:
Contact savage

Re: Traffic flow bug

Wed Oct 07, 2009 4:16 pm

various 3.30 systems on x86, mibsbe as well as mibsbe... Problem is persisting...
Code: Select all
flow-capture: ftpdu_seq_check(): src_ip=192.168.1.1 dst_ip=192.168.1.253 d_version=5 expecting=556786 received=556756 lost=4294967265
flow-capture: ftpdu_seq_check(): src_ip=192.168.1.1 dst_ip=192.168.1.253 d_version=5 expecting=556764 received=556806 lost=42
flow-capture: ftpdu_seq_check(): src_ip=192.168.1.1 dst_ip=192.168.1.253 d_version=5 expecting=556836 received=556806 lost=4294967265
flow-capture: ftpdu_seq_check(): src_ip=192.168.1.1 dst_ip=192.168.1.253 d_version=5 expecting=556807 received=556851 lost=44
flow-capture: ftpdu_seq_check(): src_ip=192.168.1.1 dst_ip=192.168.1.253 d_version=5 expecting=556881 received=556851 lost=4294967265
flow-capture: ftpdu_seq_check(): src_ip=192.168.1.1 dst_ip=192.168.1.253 d_version=5 expecting=556860 received=556892 lost=32
Top
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:
Contact Chupaka

Re: Traffic flow bug

Wed Oct 07, 2009 4:40 pm

btw, '[Ticket#2009070666000202]'

still no answer...
Top
 
savage
Forum Guru
Forum Guru
Posts: 1264
Joined: Mon Oct 18, 2004 12:07 am
Location: Cape Town, South Africa
Contact:
Contact savage

Re: Traffic flow bug

Wed Dec 09, 2009 9:46 am

At v4.3 now - STILL not fixed....
Top
 
savage
Forum Guru
Forum Guru
Posts: 1264
Joined: Mon Oct 18, 2004 12:07 am
Location: Cape Town, South Africa
Contact:
Contact savage

Re: Traffic flow bug

Wed Dec 09, 2009 10:19 am

The below patch to flow-tools will at least stop it filling my logs with useless errors due to MT netflow exports. It's hardly a fix to the problem and I'm pretty sure the issue is on the MT, but this stops the errors in the logs being generated by flow-tools. Use it, abuse it, do with it as you please.
Code: Select all
--- src/flow-capture.c.orig 2009-12-09 10:10:29.000000000 +0200
+++ src/flow-capture.c      2009-12-09 10:10:58.000000000 +0200
@@ -960,12 +960,14 @@
         fmt_ipv4(fmt_src_ip, ftch_recexp.src_ip, FMT_JUST_LEFT);
         fmt_ipv4(fmt_dst_ip, ftch_recexp.dst_ip, FMT_JUST_LEFT);
         fmt_uint16(fmt_dst_port, ftch_recexp.dst_port, FMT_JUST_LEFT);
+/*
         fterr_warnx(
           "ftpdu_seq_check(): src_ip=%s dst_ip=%s d_version=%d expecting=%lu received=%lu lost=%lu",
           fmt_src_ip, fmt_dst_ip, (int)ftpdu.ftv.d_version,
           (u_long)ftch_recexpp->ftseq.seq_exp,
           (u_long)ftch_recexpp->ftseq.seq_rcv,
           (u_long)ftch_recexpp->ftseq.seq_lost);
+ */

         /* only count these lost if "lost" is a reasonable number */
         if (ftch_recexpp->ftseq.seq_lost < FT_SEQ_RESET) {
EDIT: And as per the code, the error is being generated due to flow-tools receiving the netflows 'out of sequence'. It will only receive them out of sequence because MT is sending them out of sequence... :?
Top
 
changeip
Forum Guru
Forum Guru
Posts: 3833
Joined: Fri May 28, 2004 5:22 pm

Re: Traffic flow bug

Wed Dec 09, 2009 7:36 pm

regarding out of sequence - do you have ANY QoS rules in place anywhere between the sending router and the flow receiver?
Top
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:
Contact Chupaka

Re: Traffic flow bug

Wed Dec 09, 2009 8:18 pm

changeip, it's not QoS, it's just a bug: about every two seconds 'flush' of NF info happens, and for every such flush all NF packets have the same sequence_id - sequence_id that firsh packet should have. you can see it even in posts above
Top
 
rucevzhuru
newbie
Topic Author
Posts: 43
Joined: Fri May 25, 2007 6:57 pm

Re: Traffic flow bug

Sat Jan 23, 2010 6:00 am

btw, '[Ticket#2009070666000202]'

still no answer...
Where can I see these tickets? i can't find anything on the mikrotik web.
Top
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:
Contact Chupaka

Re: Traffic flow bug

Sat Jan 23, 2010 1:18 pm

Where can I see these tickets? i can't find anything on the mikrotik web.
it's internal MT system, you cannot see it
Top
 
rucevzhuru
newbie
Topic Author
Posts: 43
Joined: Fri May 25, 2007 6:57 pm

Re: Traffic flow bug

Sat Jan 23, 2010 6:28 pm

Where can I see these tickets? i can't find anything on the mikrotik web.
it's internal MT system, you cannot see it
Thanks for the reply. Any possibility to see it? Why is it not public? Or at least possible to login for registered owners of license.
Top
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:
Contact Chupaka

Re: Traffic flow bug

Sat Jan 23, 2010 8:31 pm

it's available only for Support team of MikroTik =) any ticket number mentioning on the forum is only for MT staff :)
Top
 
gjallen
just joined
Posts: 3
Joined: Fri Mar 26, 2010 5:28 am

Re: Traffic flow bug

Fri Mar 26, 2010 5:32 am

I am trying to implement netflows. V1 has no erors V5 has the exact erros in this post.

Is this still a bug in V3.30? Has it been fixed in 4.4?

Jerry
Top
 
savage
Forum Guru
Forum Guru
Posts: 1264
Joined: Mon Oct 18, 2004 12:07 am
Location: Cape Town, South Africa
Contact:
Contact savage

Re: Traffic flow bug

Fri Mar 26, 2010 8:21 am

Not fixed in latest 4.6, nevermind 4.5...
Top
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:
Contact Chupaka

Re: Traffic flow bug

Fri Mar 26, 2010 11:26 am

still not 'fixed'... developer is pretty sure that all is working correctly :D

there's no RFC about NF v5, so I have nothing to say against that :(

if anybody can collect exports from MT and correct ones from Cisco - please send them to support =)
Top
 
User avatar
martini
Member Candidate
Member Candidate
Posts: 296
Joined: Tue Dec 21, 2004 12:13 am

Re: Traffic flow bug

Fri Mar 26, 2010 12:21 pm

Same bug with netflow V5, problem with sequence, any version 3.xx or 4.xx or 5beta.
When it will be fixed ????
Top
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:
Contact Chupaka

Re: Traffic flow bug

Fri Mar 26, 2010 12:28 pm

please read my prevoius post!
Top
 
User avatar
martini
Member Candidate
Member Candidate
Posts: 296
Joined: Tue Dec 21, 2004 12:13 am

Re: Traffic flow bug

Fri Mar 26, 2010 2:06 pm

canot collect netflow export from cisco, but MK may install IPCAD or something else to generate normal netflow...
Top
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:
Contact Chupaka

Re: Traffic flow bug

Fri Mar 26, 2010 2:20 pm

the problem is the only 'normal' is Cisco's :D

for now, all is exported as expected by that developer...
Top
 
gjallen
just joined
Posts: 3
Joined: Fri Mar 26, 2010 5:28 am

Re: Traffic flow bug

Fri Mar 26, 2010 3:02 pm

If we can collect a netflow from a cisco that works and the same exact netflow from a MT does not, that would be helpful? Wouldn't it be as simple for MT to duplicate this in a lab rather than on my production network?


Jerry
Top
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 26821
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia
Contact:
Contact normis

Re: Traffic flow bug

Fri Mar 26, 2010 3:03 pm

because we don't see the problem :) that's why we need the file from you
Top
 
gjallen
just joined
Posts: 3
Joined: Fri Mar 26, 2010 5:28 am

Re: Traffic flow bug

Fri Mar 26, 2010 3:24 pm

I am curious now I will give it a shot. I have access to both and the network change can be made with about a 30 second outage.



Jerry
Top
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:
Contact Chupaka

Re: Traffic flow bug

Fri Mar 26, 2010 5:39 pm

I think, it will be better if it will be done in controlled lab environment, for predictive results =)
Top
 
geebs
Frequent Visitor
Frequent Visitor
Posts: 89
Joined: Tue Jan 04, 2005 3:22 am
Location: Melbourne, Australia.

Re: Traffic flow bug

Tue Mar 30, 2010 5:46 am

I am also seeing this at all sites, with a variety of MT's, from Routerboards to x86 servers.
From 3.11 all the way to the latest 4.x.

We use Netflow exclusively for all traffic data collection.
All our Cisco's are fine, no issues, however the MT's are always reporting errors in flow-capture.

Mar 30 13:14:13 localhost flow-capture[3600]: ftpdu_seq_check(): src_ip=xxx.xxx.xxx.30 dst_ip=xxx.xxx.xxx.28 d_version=5 expecting=173489248 received=173489270 lost=22
Mar 30 13:14:13 localhost flow-capture[3600]: ftpdu_seq_check(): src_ip=xxx.xxx.xxx.30 dst_ip=xxx.xxx.xxx.28 d_version=5 expecting=173489300 received=173489270 lost=4294967265
Mar 30 13:14:15 localhost flow-capture[3600]: ftpdu_seq_check(): src_ip=xxx.xxx.xxx.30 dst_ip=xxx.xxx.xxx.28 d_version=5 expecting=173489287 received=173489322 lost=35
Mar 30 13:14:15 localhost flow-capture[3600]: ftpdu_seq_check(): src_ip=xxx.xxx.xxx.30 dst_ip=xxx.xxx.xxx.28 d_version=5 expecting=173489352 received=173489322 lost=4294967265
Mar 30 13:14:17 localhost flow-capture[3600]: ftpdu_seq_check(): src_ip=xxx.xxx.xxx.29 dst_ip=xxx.xxx.xxx.28 d_version=5 expecting=1342772 received=1342773 lost=1
Mar 30 13:14:17 localhost flow-capture[3600]: ftpdu_seq_check(): src_ip=xxx.xxx.xxx.30 dst_ip=xxx.xxx.xxx.28 d_version=5 expecting=173489329 received=173489366 lost=37
Mar 30 13:14:19 localhost flow-capture[3600]: ftpdu_seq_check(): src_ip=xxx.xxx.xxx.30 dst_ip=xxx.xxx.xxx.28 d_version=5 expecting=173489391 received=173489412 lost=21
Mar 30 13:14:19 localhost flow-capture[3600]: ftpdu_seq_check(): src_ip=xxx.xxx.xxx.30 dst_ip=xxx.xxx.xxx.28 d_version=5 expecting=173489442 received=173489412 lost=4294967265
Mar 30 13:14:21 localhost flow-capture[3600]: ftpdu_seq_check(): src_ip=xxx.xxx.xxx.30 dst_ip=xxx.xxx.xxx.28 d_version=5 expecting=173489430 received=173489466 lost=36
Mar 30 13:14:21 localhost flow-capture[3600]: ftpdu_seq_check(): src_ip=xxx.xxx.xxx.30 dst_ip=xxx.xxx.xxx.28 d_version=5 expecting=173489496 received=173489466 lost=4294967265
Mar 30 13:14:23 localhost flow-capture[3600]: ftpdu_seq_check(): src_ip=xxx.xxx.xxx.30 dst_ip=xxx.xxx.xxx.28 d_version=5 expecting=173489482 received=173489503 lost=21
Mar 30 13:14:23 localhost flow-capture[3600]: ftpdu_seq_check(): src_ip=xxx.xxx.xxx.30 dst_ip=xxx.xxx.xxx.28 d_version=5 expecting=173489533 received=173489503 lost=4294967265

We use flow-capture.
http://www.splintered.net/sw/flow-tools ... pture.html
Top
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8712
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:
Contact Chupaka

Re: Traffic flow bug

Tue Mar 30, 2010 10:16 am

please, write to support@mikrotik.com
Top
 
User avatar
sergejs
MikroTik Support
MikroTik Support
Posts: 6697
Joined: Thu Mar 31, 2005 3:33 pm
Location: Riga, Latvia
Contact:
Contact sergejs

Re: Traffic flow bug

Tue May 18, 2010 2:15 pm

I was able to get the same Template Len mismatch errors with Ntop and router running MikroTik RouterOS 4.9.
We will see what we can do.
Top
 
User avatar
martini
Member Candidate
Member Candidate
Posts: 296
Joined: Tue Dec 21, 2004 12:13 am

Re: Traffic flow bug

Tue May 18, 2010 4:27 pm

finally !! )) waiting new version with fixed traffic-flow
Top
 
User avatar
sergejs
MikroTik Support
MikroTik Support
Posts: 6697
Joined: Thu Mar 31, 2005 3:33 pm
Location: Riga, Latvia
Contact:
Contact sergejs

Re: Traffic flow bug

Wed May 19, 2010 9:49 am

So to clarify my previous post. There will be improvements in the next MikroTik RouterOS version.

Traffic-flow v5,
- fixed and improved flow sequence algorithm, which should help for v5 described problem;
Traffic-flow v9,
- flowset is aligned and tuned in the same way as Cisco does.
By the way Ntop still reports Template Len Mismatch, however the same errors are present for Cisco netflow v9.
Flowset alignment should help for the v9 problems described above.
Top
 
User avatar
martini
Member Candidate
Member Candidate
Posts: 296
Joined: Tue Dec 21, 2004 12:13 am

Re: Traffic flow bug

Wed May 19, 2010 12:47 pm

ok, whaiting new version
Top
Post Reply

Who is online

Users browsing this forum: anav, Google [Bot], junbr0 and 41 guests
  4. RouterOS
  5. General