I am struggling to setup Mikrotik CHR on ESXI 6.5 with Intel 82599ES 10Gb SFP+ Adapter but its showing weird issues which i have tried to resolve since a week but i am failing in all aspects.

Issue :
When Mikrotik CHR is used with 1G Virtual NIC , everything works perfectly.
When Mikrotik CHR is used with 10G Intel 82599ES , I see vmware throwing logs of adapter reset. Error log below

[root@localhost:~] tail -f /var/log/vmkernel.log
2017-04-27T10:55:09.197Z cpu22:70056)NetSched: 628: vmnic2-5-tx: worldID = 70056 exits
2017-04-27T10:55:09.197Z cpu27:70054)NetSched: 628: vmnic2-3-tx: worldID = 70054 exits
2017-04-27T10:55:09.197Z cpu14:70052)NetSched: 628: vmnic2-1-tx: worldID = 70052 exits
2017-04-27T10:55:09.197Z cpu0:70058)NetSched: 628: vmnic2-7-tx: worldID = 70058 exits
2017-04-27T10:55:09.197Z cpu30:70053)NetSched: 628: vmnic2-2-tx: worldID = 70053 exits
2017-04-27T10:55:09.197Z cpu23:65645)NetLB: 2246: Driver claims supporting 7 RX queues, and 7 queues are accepted.
2017-04-27T10:55:09.197Z cpu19:70057)NetSched: 628: vmnic2-6-tx: worldID = 70057 exits
2017-04-27T10:55:09.197Z cpu23:65645)NetLB: 2242: Driver claims supporting 7 TX queues, and 7 queues are accepted.
2017-04-27T10:55:09.197Z cpu23:65645)Uplink: 9893: enabled port 0x3000002 with mac 00:1b:21:bc:7f:0c
2017-04-27T10:55:09.197Z cpu23:70055)NetSched: 628: vmnic2-4-tx: worldID = 70055 exits
2017-04-27T10:56:29.460Z cpu14:65550)<6>ixgbe 0000:01:00.0: vmnic2: Received unrecoverable ECC Err, please reboot
2017-04-27T10:56:34.033Z cpu19:66175)<3>ixgbe 0000:01:00.0: vmnic2: Detected Tx Unit Hang
Tx Queue <0>
TDH, TDT <14f>, <15c>
next_to_use <15c>
next_to_clean <14f>
2017-04-27T10:56:34.033Z cpu19:66175)<3>ixgbe 0000:01:00.0: vmnic2: tx_buffer_info[next_to_clean]
time_stamp <28d9e>
jiffies <28f67>
2017-04-27T10:56:34.033Z cpu19:66175)<6>ixgbe 0000:01:00.0: vmnic2: tx hang 29 detected on queue 0, resetting adapter
2017-04-27T10:56:34.033Z cpu11:66097)<3>ixgbe 0000:01:00.0: vmnic2: Reset adapter
2017-04-27T10:56:34.265Z cpu11:66097)<6>ixgbe 0000:01:00.0: vmnic2: detected SFP+: 5
2017-04-27T10:56:34.505Z cpu11:66097)<6>ixgbe 0000:01:00.0: vmnic2: NIC Link is Up 10 Gbps, Flow Control: None
2017-04-27T10:56:35.198Z cpu0:65645)NetPort: 1879: disabled port 0x3000002
2017-04-27T10:56:35.198Z cpu6:70059)NetSched: 628: vmnic2-0-tx: worldID = 70059 exits
2017-04-27T10:56:35.198Z cpu1:70060)NetSched: 628: vmnic2-1-tx: worldID = 70060 exits
2017-04-27T10:56:35.198Z cpu13:70061)NetSched: 628: vmnic2-2-tx: worldID = 70061 exits
2017-04-27T10:56:35.198Z cpu14:70063)NetSched: 628: vmnic2-4-tx: worldID = 70063 exits
2017-04-27T10:56:35.198Z cpu18:70064)NetSched: 628: vmnic2-5-tx: worldID = 70064 exits
2017-04-27T10:56:35.198Z cpu20:70065)NetSched: 628: vmnic2-6-tx: worldID = 70065 exits
2017-04-27T10:56:35.198Z cpu6:70062)NetSched: 628: vmnic2-3-tx: worldID = 70062 exits
2017-04-27T10:56:35.198Z cpu22:70066)NetSched: 628: vmnic2-7-tx: worldID = 70066 exits
2017-04-27T10:56:35.198Z cpu0:65645)NetLB: 2246: Driver claims supporting 7 RX queues, and 7 queues are accepted.
2017-04-27T10:56:35.198Z cpu0:65645)NetLB: 2242: Driver claims supporting 7 TX queues, and 7 queues are accepted.
2017-04-27T10:56:35.198Z cpu0:65645)Uplink: 9893: enabled port 0x3000002 with mac 00:1b:21:bc:7f:0c
2017-04-27T10:56:36.197Z cpu0:65645)NetPort: 1879: disabled port 0x3000002
2017-04-27T10:56:36.197Z cpu1:70069)NetSched: 628: vmnic2-0-tx: worldID = 70069 exits
2017-04-27T10:56:36.197Z cpu21:70070)NetSched: 628: vmnic2-1-tx: worldID = 70070 exits
2017-04-27T10:56:36.197Z cpu6:70071)NetSched: 628: vmnic2-2-tx: worldID = 70071 exits
2017-04-27T10:56:36.197Z cpu16:70072)NetSched: 628: vmnic2-3-tx: worldID = 70072 exits
2017-04-27T10:56:36.197Z cpu19:70073)NetSched: 628: vmnic2-4-tx: worldID = 70073 exits
2017-04-27T10:56:36.197Z cpu22:70074)NetSched: 628: vmnic2-5-tx: worldID = 70074 exits
2017-04-27T10:56:36.197Z cpu26:70075)NetSched: 628: vmnic2-6-tx: worldID = 70075 exits
2017-04-27T10:56:36.197Z cpu12:70076)NetSched: 628: vmnic2-7-tx: worldID = 70076 exits
2017-04-27T10:56:36.197Z cpu0:65645)NetLB: 2246: Driver claims supporting 7 RX queues, and 7 queues are accepted.
2017-04-27T10:56:36.197Z cpu0:65645)NetLB: 2242: Driver claims supporting 7 TX queues, and 7 queues are accepted.
2017-04-27T10:56:36.198Z cpu0:65645)Uplink: 9893: enabled port 0x3000002 with mac 00:1b:21:bc:7f:0c
2017-04-27T10:56:52.520Z cpu14:66363)<6>ixgbe 0000:01:00.0: vmnic2: Received unrecoverable ECC Err, please reboot
2017-04-27T10:56:56.630Z cpu19:66175)<3>ixgbe 0000:01:00.0: vmnic2: Detected Tx Unit Hang
Tx Queue <0>
TDH, TDT <87>, <9f>
next_to_use <9f>
next_to_clean <87>
2017-04-27T10:56:56.630Z cpu19:66175)<3>ixgbe 0000:01:00.0: vmnic2: tx_buffer_info[next_to_clean]
time_stamp <296a0>
jiffies <2983b>
2017-04-27T10:56:56.630Z cpu19:66175)<6>ixgbe 0000:01:00.0: vmnic2: tx hang 30 detected on queue 0, resetting adapter
2017-04-27T10:56:56.630Z cpu9:66095)<3>ixgbe 0000:01:00.0: vmnic2: Reset adapter
2017-04-27T10:56:56.867Z cpu9:66095)<6>ixgbe 0000:01:00.0: vmnic2: detected SFP+: 5
2017-04-27T10:56:57.107Z cpu9:66095)<6>ixgbe 0000:01:00.0: vmnic2: NIC Link is Up 10 Gbps, Flow Control: None
2017-04-27T10:56:57.198Z cpu0:65645)NetPort: 1879: disabled port 0x3000002
2017-04-27T10:56:57.198Z cpu7:70077)NetSched: 628: vmnic2-0-tx: worldID = 70077 exits
2017-04-27T10:56:57.198Z cpu10:70078)NetSched: 628: vmnic2-1-tx: worldID = 70078 exits
2017-04-27T10:56:57.198Z cpu13:70079)NetSched: 628: vmnic2-2-tx: worldID = 70079 exits
2017-04-27T10:56:57.198Z cpu14:70080)NetSched: 628: vmnic2-3-tx: worldID = 70080 exits
2017-04-27T10:56:57.198Z cpu2:70081)NetSched: 628: vmnic2-4-tx: worldID = 70081 exits
2017-04-27T10:56:57.198Z cpu26:70082)NetSched: 628: vmnic2-5-tx: worldID = 70082 exits
2017-04-27T10:56:57.198Z cpu17:70083)NetSched: 628: vmnic2-6-tx: worldID = 70083 exits
2017-04-27T10:56:57.198Z cpu18:70084)NetSched: 628: vmnic2-7-tx: worldID = 70084 exits
2017-04-27T10:56:57.198Z cpu0:65645)NetLB: 2246: Driver claims supporting 7 RX queues, and 7 queues are accepted.
2017-04-27T10:56:57.198Z cpu0:65645)NetLB: 2242: Driver claims supporting 7 TX queues, and 7 queues are accepted.
2017-04-27T10:56:57.198Z cpu0:65645)Uplink: 9893: enabled port 0x3000002 with mac 00:1b:21:bc:7f:0c
2017-04-27T10:57:01.075Z cpu14:66363)<6>ixgbe 0000:01:00.0: vmnic2: Received unrecoverable ECC Err, please reboot
2017-04-27T10:57:05.213Z cpu19:66175)<3>ixgbe 0000:01:00.0: vmnic2: Detected Tx Unit Hang
Tx Queue <0>
TDH, TDT <f>, <1c>
next_to_use <1c>
next_to_clean <f>
2017-04-27T10:57:05.213Z cpu19:66175)<3>ixgbe 0000:01:00.0: vmnic2: tx_buffer_info[next_to_clean]
time_stamp <299f7>
jiffies <29b95>
2017-04-27T10:57:05.213Z cpu19:66175)<6>ixgbe 0000:01:00.0: vmnic2: tx hang 31 detected on queue 0, resetting adapter
2017-04-27T10:57:05.213Z cpu1:66091)<3>ixgbe 0000:01:00.0: vmnic2: Reset adapter
2017-04-27T10:57:05.445Z cpu1:66091)<6>ixgbe 0000:01:00.0: vmnic2: detected SFP+: 5
2017-04-27T10:57:05.685Z cpu1:66091)<6>ixgbe 0000:01:00.0: vmnic2: NIC Link is Up 10 Gbps, Flow Control: None
2017-04-27T10:57:11.198Z cpu0:65645)NetPort: 1879: disabled port 0x3000002
2017-04-27T10:57:11.198Z cpu10:70086)NetSched: 628: vmnic2-0-tx: worldID = 70086 exits
2017-04-27T10:57:11.198Z cpu3:70087)NetSched: 628: vmnic2-1-tx: worldID = 70087 exits
2017-04-27T10:57:11.198Z cpu6:70088)NetSched: 628: vmnic2-2-tx: worldID = 70088 exits
2017-04-27T10:57:11.198Z cpu8:70089)NetSched: 628: vmnic2-3-tx: worldID = 70089 exits
2017-04-27T10:57:11.198Z cpu13:70091)NetSched: 628: vmnic2-5-tx: worldID = 70091 exits
2017-04-27T10:57:11.198Z cpu14:70092)NetSched: 628: vmnic2-6-tx: worldID = 70092 exits
2017-04-27T10:57:11.198Z cpu22:70093)NetSched: 628: vmnic2-7-tx: worldID = 70093 exits
2017-04-27T10:57:11.198Z cpu0:65645)NetLB: 2246: Driver claims supporting 7 RX queues, and 7 queues are accepted.
2017-04-27T10:57:11.198Z cpu0:65645)NetLB: 2242: Driver claims supporting 7 TX queues, and 7 queues are accepted.
2017-04-27T10:57:11.198Z cpu10:70090)NetSched: 628: vmnic2-4-tx: worldID = 70090 exits
2017-04-27T10:57:11.198Z cpu0:65645)Uplink: 9893: enabled port 0x3000002 with mac 00:1b:21:bc:7f:0c
2017-04-27T10:57:12.198Z cpu0:65645)NetPort: 1879: disabled port 0x3000002
2017-04-27T10:57:12.198Z cpu8:70095)NetSched: 628: vmnic2-0-tx: worldID = 70095 exits
2017-04-27T10:57:12.198Z cpu10:70094)NetSched: 628: vmnic2-1-tx: worldID = 70094 exits
2017-04-27T10:57:12.198Z cpu27:70096)NetSched: 628: vmnic2-2-tx: worldID = 70096 exits
2017-04-27T10:57:12.198Z cpu24:70097)NetSched: 628: vmnic2-3-tx: worldID = 70097 exits
2017-04-27T10:57:12.198Z cpu2:70098)NetSched: 628: vmnic2-4-tx: worldID = 70098 exits
2017-04-27T10:57:12.198Z cpu17:70099)NetSched: 628: vmnic2-5-tx: worldID = 70099 exits
2017-04-27T10:57:12.198Z cpu13:70101)NetSched: 628: vmnic2-7-tx: worldID = 70101 exits
2017-04-27T10:57:12.198Z cpu0:65645)NetLB: 2246: Driver claims supporting 7 RX queues, and 7 queues are accepted.
2017-04-27T10:57:12.198Z cpu0:65645)NetLB: 2242: Driver claims supporting 7 TX queues, and 7 queues are accepted.
2017-04-27T10:57:12.198Z cpu0:65645)Uplink: 9893: enabled port 0x3000002 with mac 00:1b:21:bc:7f:0c
2017-04-27T10:57:12.198Z cpu18:70100)NetSched: 628: vmnic2-6-tx: worldID = 70100 exits
2017-04-27T10:57:19.877Z cpu14:66181)<6>ixgbe 0000:01:00.0: vmnic2: Received unrecoverable ECC Err, please reboot
2017-04-27T10:57:23.799Z cpu31:66175)<3>ixgbe 0000:01:00.0: vmnic2: Detected Tx Unit Hang
Tx Queue <0>
TDH, TDT <4d>, <65>
next_to_use <65>
next_to_clean <4d>
2017-04-27T10:57:23.799Z cpu31:66175)<3>ixgbe 0000:01:00.0: vmnic2: tx_buffer_info[next_to_clean]
time_stamp <2a14f>
jiffies <2a2d8>
2017-04-27T10:57:23.799Z cpu31:66175)<6>ixgbe 0000:01:00.0: vmnic2: tx hang 32 detected on queue 0, resetting adapter
2017-04-27T10:57:23.799Z cpu9:66108)<3>ixgbe 0000:01:00.0: vmnic2: Reset adapter
2017-04-27T10:57:24.030Z cpu9:66108)<6>ixgbe 0000:01:00.0: vmnic2: detected SFP+: 5
2017-04-27T10:57:24.270Z cpu9:66108)<6>ixgbe 0000:01:00.0: vmnic2: NIC Link is Up 10 Gbps, Flow Control: None
2017-04-27T10:57:25.198Z cpu0:65645)NetPort: 1879: disabled port 0x3000002
2017-04-27T10:57:25.198Z cpu2:70102)NetSched: 628: vmnic2-0-tx: worldID = 70102 exits
2017-04-27T10:57:25.198Z cpu10:70103)NetSched: 628: vmnic2-1-tx: worldID = 70103 exits
2017-04-27T10:57:25.198Z cpu19:70104)NetSched: 628: vmnic2-2-tx: worldID = 70104 exits
2017-04-27T10:57:25.198Z cpu7:70105)NetSched: 628: vmnic2-3-tx: worldID = 70105 exits
2017-04-27T10:57:25.198Z cpu13:70106)NetSched: 628: vmnic2-4-tx: worldID = 70106 exits
2017-04-27T10:57:25.198Z cpu14:70107)NetSched: 628: vmnic2-5-tx: worldID = 70107 exits
2017-04-27T10:57:25.198Z cpu20:70108)NetSched: 628: vmnic2-6-tx: worldID = 70108 exits
2017-04-27T10:57:25.198Z cpu24:70109)NetSched: 628: vmnic2-7-tx: worldID = 70109 exits
2017-04-27T10:57:25.198Z cpu0:65645)NetLB: 2246: Driver claims supporting 7 RX queues, and 7 queues are accepted.
2017-04-27T10:57:25.198Z cpu0:65645)NetLB: 2242: Driver claims supporting 7 TX queues, and 7 queues are accepted.
2017-04-27T10:57:25.198Z cpu0:65645)Uplink: 9893: enabled port 0x3000002 with mac 00:1b:21:bc:7f:0c
2017-04-27T10:57:26.196Z cpu0:65645)NetPort: 1879: disabled port 0x3000002
2017-04-27T10:57:26.196Z cpu7:70110)NetSched: 628: vmnic2-0-tx: worldID = 70110 exits
2017-04-27T10:57:26.196Z cpu14:70116)NetSched: 628: vmnic2-6-tx: worldID = 70116 exits
2017-04-27T10:57:26.196Z cpu24:70112)NetSched: 628: vmnic2-2-tx: worldID = 70112 exits
2017-04-27T10:57:26.196Z cpu29:70114)NetSched: 628: vmnic2-4-tx: worldID = 70114 exits
2017-04-27T10:57:26.196Z cpu21:70111)NetSched: 628: vmnic2-1-tx: worldID = 70111 exits
2017-04-27T10:57:26.196Z cpu19:70117)NetSched: 628: vmnic2-7-tx: worldID = 70117 exits
2017-04-27T10:57:26.196Z cpu10:70113)NetSched: 628: vmnic2-3-tx: worldID = 70113 exits
2017-04-27T10:57:26.196Z cpu0:65645)NetLB: 2246: Driver claims supporting 7 RX queues, and 7 queues are accepted.
2017-04-27T10:57:26.196Z cpu0:65645)NetLB: 2242: Driver claims supporting 7 TX queues, and 7 queues are accepted.
2017-04-27T10:57:26.196Z cpu3:70115)NetSched: 628: vmnic2-5-tx: worldID = 70115 exits
2017-04-27T10:57:26.196Z cpu0:65645)Uplink: 9893: enabled port 0x3000002 with mac 00:1b:21:bc:7f:0c
2017-04-27T10:57:28.877Z cpu14:66181)<6>ixgbe 0000:01:00.0: vmnic2: Received unrecoverable ECC Err, please reboot
2017-04-27T10:57:34.382Z cpu31:66175)<3>ixgbe 0000:01:00.0: vmnic2: Detected Tx Unit Hang
Tx Queue <0>
TDH, TDT <25>, <44>
next_to_use <44>
next_to_clean <25>
2017-04-27T10:57:34.382Z cpu31:66175)<3>ixgbe 0000:01:00.0: vmnic2: tx_buffer_info[next_to_clean]
time_stamp <2a4d3>
jiffies <2a6fa>
2017-04-27T10:57:34.382Z cpu31:66175)<6>ixgbe 0000:01:00.0: vmnic2: tx hang 33 detected on queue 0, resetting adapter
2017-04-27T10:57:34.382Z cpu9:66086)<3>ixgbe 0000:01:00.0: vmnic2: Reset adapter
2017-04-27T10:57:34.613Z cpu9:66086)<6>ixgbe 0000:01:00.0: vmnic2: detected SFP+: 5
2017-04-27T10:57:34.853Z cpu9:66086)<6>ixgbe 0000:01:00.0: vmnic2: NIC Link is Up 10 Gbps, Flow Control: None
2017-04-27T10:57:36.198Z cpu0:65645)NetPort: 1879: disabled port 0x3000002
2017-04-27T10:57:36.198Z cpu8:70118)NetSched: 628: vmnic2-0-tx: worldID = 70118 exits
2017-04-27T10:57:36.198Z cpu19:70120)NetSched: 628: vmnic2-2-tx: worldID = 70120 exits
2017-04-27T10:57:36.198Z cpu10:70119)NetSched: 628: vmnic2-1-tx: worldID = 70119 exits
2017-04-27T10:57:36.198Z cpu7:70121)NetSched: 628: vmnic2-3-tx: worldID = 70121 exits
2017-04-27T10:57:36.198Z cpu26:70123)NetSched: 628: vmnic2-5-tx: worldID = 70123 exits
2017-04-27T10:57:36.198Z cpu20:70124)NetSched: 628: vmnic2-6-tx: worldID = 70124 exits
2017-04-27T10:57:36.198Z cpu30:70125)NetSched: 628: vmnic2-7-tx: worldID = 70125 exits
2017-04-27T10:57:36.198Z cpu0:65645)NetLB: 2246: Driver claims supporting 7 RX queues, and 7 queues are accepted.
2017-04-27T10:57:36.198Z cpu0:65645)NetLB: 2242: Driver claims supporting 7 TX queues, and 7 queues are accepted.
2017-04-27T10:57:36.198Z cpu14:70122)NetSched: 628: vmnic2-4-tx: worldID = 70122 exits
2017-04-27T10:57:36.198Z cpu0:65645)Uplink: 9893: enabled port 0x3000002 with mac 00:1b:21:bc:7f:0c
2017-04-27T10:57:37.198Z cpu13:65645)NetPort: 1879: disabled port 0x3000002
2017-04-27T10:57:37.198Z cpu6:70126)NetSched: 628: vmnic2-0-tx: worldID = 70126 exits
2017-04-27T10:57:37.198Z cpu20:70127)NetSched: 628: vmnic2-1-tx: worldID = 70127 exits
2017-04-27T10:57:37.198Z cpu23:70128)NetSched: 628: vmnic2-2-tx: worldID = 70128 exits
2017-04-27T10:57:37.198Z cpu8:70129)NetSched: 628: vmnic2-3-tx: worldID = 70129 exits
2017-04-27T10:57:37.198Z cpu30:70130)NetSched: 628: vmnic2-4-tx: worldID = 70130 exits
2017-04-27T10:57:37.198Z cpu10:70132)NetSched: 628: vmnic2-6-tx: worldID = 70132 exits
2017-04-27T10:57:37.198Z cpu26:70133)NetSched: 628: vmnic2-7-tx: worldID = 70133 exits
2017-04-27T10:57:37.198Z cpu13:65645)NetLB: 2246: Driver claims supporting 7 RX queues, and 7 queues are accepted.
2017-04-27T10:57:37.198Z cpu13:65645)NetLB: 2242: Driver claims supporting 7 TX queues, and 7 queues are accepted.
2017-04-27T10:57:37.198Z cpu6:70131)NetSched: 628: vmnic2-5-tx: worldID = 70131 exits
2017-04-27T10:57:37.198Z cpu13:65645)Uplink: 9893: enabled port 0x3000002 with mac 00:1b:21:bc:7f:0c
2017-04-27T10:57:39.997Z cpu14:67716)<6>ixgbe 0000:01:00.0: vmnic2: Received unrecoverable ECC Err, please reboot
2017-04-27T10:57:44.965Z cpu31:66175)<3>ixgbe 0000:01:00.0: vmnic2: Detected Tx Unit Hang
Tx Queue <0>
TDH, TDT <f>, <1e>
next_to_use <1e>
next_to_clean <f>
2017-04-27T10:57:44.965Z cpu31:66175)<3>ixgbe 0000:01:00.0: vmnic2: tx_buffer_info[next_to_clean]
time_stamp <2a92b>
jiffies <2ab1c>
2017-04-27T10:57:44.965Z cpu31:66175)<6>ixgbe 0000:01:00.0: vmnic2: tx hang 34 detected on queue 0, resetting adapter
2017-04-27T10:57:44.965Z cpu10:66104)<3>ixgbe 0000:01:00.0: vmnic2: Reset adapter
2017-04-27T10:57:45.196Z cpu10:66104)<6>ixgbe 0000:01:00.0: vmnic2: detected SFP+: 5
2017-04-27T10:57:45.436Z cpu10:66104)<6>ixgbe 0000:01:00.0: vmnic2: NIC Link is Up 10 Gbps, Flow Control: None
2017-04-27T10:57:48.198Z cpu17:65645)NetPort: 1879: disabled port 0x3000002
2017-04-27T10:57:48.198Z cpu19:70136)NetSched: 628: vmnic2-2-tx: worldID = 70136 exits
2017-04-27T10:57:48.198Z cpu6:70134)NetSched: 628: vmnic2-0-tx: worldID = 70134 exits
2017-04-27T10:57:48.198Z cpu25:70135)NetSched: 628: vmnic2-1-tx: worldID = 70135 exits
2017-04-27T10:57:48.198Z cpu12:70137)NetSched: 628: vmnic2-3-tx: worldID = 70137 exits
2017-04-27T10:57:48.198Z cpu14:70139)NetSched: 628: vmnic2-5-tx: worldID = 70139 exits
2017-04-27T10:57:48.198Z cpu1:70140)NetSched: 628: vmnic2-6-tx: worldID = 70140 exits
2017-04-27T10:57:48.198Z cpu27:70141)NetSched: 628: vmnic2-7-tx: worldID = 70141 exits
2017-04-27T10:57:48.198Z cpu17:65645)NetLB: 2246: Driver claims supporting 7 RX queues, and 7 queues are accepted.
2017-04-27T10:57:48.198Z cpu17:65645)NetLB: 2242: Driver claims supporting 7 TX queues, and 7 queues are accepted.
2017-04-27T10:57:48.198Z cpu17:65645)Uplink: 9893: enabled port 0x3000002 with mac 00:1b:21:bc:7f:0c
2017-04-27T10:57:48.198Z cpu31:70138)NetSched: 628: vmnic2-4-tx: worldID = 70138 exits

If anyone needs further logs , it will be shared promptly.

Any help on this will be highly appreciated.

Use VMXNet3 as your Ethernet adapter. Support was added to CHR in 6.31.

https://mikrotik.com/download/changelogs/ (search for 6.31 or vmxnet3)

That will give you > 1gbps performance assuming the underlying infrastructure and other components can move at those speeds.

Hi,

I bough license p1 and have same problem.
Mikrotik support tell me that problem will resolve in V7.

One month spent time to try esx 5.5 sp3 without success. No more than 600mb speed.
Kernel driver from mikrotik not good enogh for gigabit....so....10gbit....

Good luck to you

... While others are able to run CHR with 10G and provide it publicly for your testing....

viewtopic.php?f=2&t=104266

... While others are able to run CHR with 10G and provide it publicly for your testing....

viewtopic.php?f=2&t=104266

course ...but how many people install on ESX ? I see 2-3 people and speed ....

This is from Mikrotik support team

> > > > In that case unfortunately that is all how we can help you at this moment.
> > > >
> > > > You can try to change something on drivers but that is not related to RouterOS.
> > > >
> > > > Unfortunately all that we can do is to wait for RouterOS v7 where completely new drivers will be used and performance should be appropriate.
> > > >
> > > > We are sorry for any inconvenience.

Regards,

How does it matter how many people running chr on esxi are you able to see?

How does it matter how many people running chr on esxi are you able to see?

Hi,

Please try and you will see.
Maybe I don't know but the response from support must to clarify that.

P.S. Maybe work on old server with esx installed.

Best Regards,

I am running chr instances in vmware workstation easily maxing the gigabit port on the server. When I am ready to run esxi I will move them there too...

I am running chr instances in vmware workstation easily maxing the gigabit port on the server. When I am ready to run esxi I will move them there too...

Course ....with with vmware WORKSTATION should be work because the driver is from host os.

ESX use vkernel and it is totally different driver...

P.S. If you try a with more network with more vswitch scenario in ESX you will see same little problem like: vlan with vpn in some situation not working, vlan encapsulation it is not fully compatible with all vendor, driver for mikrotik work but not at full speed .......... THIS IS FOR ESX 5.5 UP2. I DON'T KNOW IF WORKS IN ESX 6.5 !!!

Regards

I am running chr instances in vmware workstation easily maxing the gigabit port on the server. When I am ready to run esxi I will move them there too...

Yeah but that's nothing to brag about.

My CCR's battle at more than ~4Gbps (real traffic) - and I'm not the only one, there's many similar posts on this very forum. I am currently testing a CHR on ESX and it's not doing much better - at least BGP's faster though.

Not bragging about that. Just telling that chr is capable more than someone else tells here...

I run a CHR under ESXi with 10-gig interfaces.

it works well and I have no problem sustaining 8 Gig + throughputs with low CPU loads.

In my opinion, to build/configure a high-throughput CHR system, here is what I do:
- On the physical box:
1, disable all unnecessary interfaces (stuff like serial ports and printer ports and CD/floppy drives)
2, fast Xeon CPUs, with lots of processor cache
3, if I can, I prefer to disable hyper-threading
4, Fast SAS hard drives with lots of cache in the drives
5, A single 10-gig 2-port network card -and- disable all other 1-gig networks cards
6, remove/disable as many devices as possible that use interrupts.
7, when possible, never have the 10-gig network card share an interrupt with other devices.

- On the virtual hosted OS --- CHR
a, disable all unnecessary interfaces (stuff like serial ports and printer ports and CD/floppy drives)
b, use fewer virtual network interfaces - and instead use 802.1q vlans instead (if you need many many network interfaces).
c, for very-high throughput CHR systems, try to run other virtual servers on other physical ESXi servers
d, When ever possible, use only hypervisor optimized devices/drivers. ( Paravirtual VMXNET-3 network interfaces & Paravirtual SCSI controllers "VMware Paravirtual" SCSI Controller Type)

You know you have a decent build when you can btest to 127.0.0.1 at near 20-gig throughput --and-- btest between different CHRs on the same ESXi box or other physical ESXi boxes at close to 10-gig.
And - You know you have a decent build when you can sustain 8 gig to your NAS system.

North Idaho Tom Jones

I run a CHR under ESXi with 10-gig interfaces.

it works well and I have no problem sustaining 8 Gig + throughputs with low CPU loads.

In my opinion, to build/configure a high-throughput CHR system, here is what I do:
- On the physical box:
1, disable all unnecessary interfaces (stuff like serial ports and printer ports and CD/floppy drives)
2, fast Xeon CPUs, with lots of processor cache
3, if I can, I prefer to disable hyper-threading
4, Fast SAS hard drives with lots of cache in the drives
5, A single 10-gig 2-port network card -and- disable all other 1-gig networks cards
6, remove/disable as many devices as possible that use interrupts.
7, when possible, never have the 10-gig network card share an interrupt with other devices.

- On the virtual hosted OS --- CHR
a, disable all unnecessary interfaces (stuff like serial ports and printer ports and CD/floppy drives)
b, use fewer virtual network interfaces - and instead use 802.1q vlans instead (if you need many many network interfaces).
c, for very-high throughput CHR systems, try to run other virtual servers on other physical ESXi servers
d, When ever possible, use only hypervisor optimized devices/drivers. ( Paravirtual VMXNET-3 network interfaces & Paravirtual SCSI controllers "VMware Paravirtual" SCSI Controller Type)

You know you have a decent build when you can btest to 127.0.0.1 at near 20-gig throughput --and-- btest between different CHRs on the same ESXi box or other physical ESXi boxes at close to 10-gig.
And - You know you have a decent build when you can sustain 8 gig to your NAS system.

North Idaho Tom Jones

What version and license level of vmware are you using?

Also, have you tested NIC’s above 10gbps? Ie: 40gbps

We are currently testing o. Esxi 6.5 using a Dell R620 with dual e5-2650L v2, 64Gb RAM, 1 x dual QSFP+ 40Gbit Nic, 1 x Quad SFP+ 10Gbit NIC and the onboard NIC with 2 x 10Gbit SFP+ and 2 x 1gbe

ausip

re:

- What version and license level of vmware are you using?
- Also, have you tested NIC’s above 10gbps? Ie: 40gbps
- We are currently testing o. Esxi 6.5 using a Dell R620 with dual e5-2650L v2, 64Gb RAM, 1 x dual QSFP+ 40Gbit Nic, 1 x Quad SFP+ 10Gbit NIC and the onboard NIC with 2 x 10Gbit SFP+ and 2 x 1gbe


I have run several several VMware ESXi versions 5.5 -through- 6.5.x
My VmWare ESXi license levels are the free versions.

I have only tested the Intel 10-Gig nics. I suspect my VmWare ESXi servers ( virtaul switches and virtual hosts) may not have the horse-power to hit/hold 40 gig on a 40-gig interface ( because of CHR loopback btests to 127.0.0.1 only hitting about 20 gig ). I suspect newer faster CPUs & faster CPU memory would boost my performance.

What ever the brand/model of the physical server running VmWare ESXi , I suspect it all comes down to all of the following to produce 40-gig throughput speeds:
- CPU clock speed
- CPU cache memory ( larger is always better )
- CPU version
- Memory speed
- Interrupts (shut all non-used devices for both physical and virtual)
- little of few or no wait-states for all devices on the physical system buss
- Paravirtual device drivers (such as VMNIC3 for Ethernet interfaces)
- HyperThreading ( disabled ... )
- Number of other guest host systems running (for 40-gig, I would try running only a single guest and see what that does)
- How complex are the firewall rules and other configurations
- Delayed-ACK (I often perfer delayed_ACK = 1)
- All power management (physical and virtual machines) disabled
- If lots of disk I/O , then fast SAS or SSD drives (for SAS , I suggest lots of built-in cache)
- If my VmWare ESXI is running a remote datastore, then NFS and zfs sync disabled
- If my NAS is FreeNAS, then auto-tune and also delayed_ack with zfs sync disabled (my older FreeNAS uses 6-Gig SAS interfaces and can hold/sustain 8-Gig transfers to/from my lz4 NFS mounted datastores when performing a sequential copy (about 1/2 that when multiple R/W operations are hitting my NAS at the same time).


I suspect 40-gig throughput just might require hardware which is the best of the best

Hope this helps.

North Idaho Tom Jones