I have a standard local network on RB750GL with latest updates. There is a default NAT with masquerade on Ethernet 1 port (WAN). Internet works fine.

Problem is computers on local network can't access Samba v2 or v3 share over internet - simple linux samba server at OVH dedicated machine. Windows just says Error 0x80070035 "The network path was not found"
BUT! when i force SMB v1 on said server - everything works just fine, computers can connect without a problem. Obviously i dont want to keep this server with SMBv1 protocol, i want to use SMBv2 or newer.

Samba server have no firewall, just hosts file with one allowed IP (internet IP of this mikrotik router).
Is this problem with my router or do i have to search answers somewhere else? Do SMBv2/v3 need some special rules, port forwarding, different NAT settings?

SMBv1-v3 use the same TCP port 445 I think. If SMBv1 works, then you must have port forwarding set up correctly, and using an IP or FQDN to connect (like \\host.domain.local\share)

The Mikrotik router won't be concerned with the content or protocols used. Your issue probably lies elsewhere.

On Windows, when your network connection is set to "Private/Work" or "Public/Guest" then there is an outbound firewall rule which restricts outbound windows file sharing packets to the local subnet only. Check for the "Windows File Sharing" rule.

After that you can use Winbox, Tools > Packet Sniffer and capture traffic going in and out of your WAN. Then open the caputre file with Wireshark. Wireshark decodes SMB packets. Maybe you'll see something useful in there. Maybe you won't see the request leaving your network, or maybe you'll find that SAMBA server is not replying.

This problem is caused by the "Secure Negotiate" feature that was added to SMB 3.0 for Windows Server 2012 and Windows 8. This feature depends upon the correct signing of error responses by all SMBv2 servers, including servers that support only protocol versions 2.0 and 2.1. Some third-party file servers do not return a signed error response. Therefore, the connection fails.

From Microsoft: https://support.microsoft.com/en-au/hel ... -windows-8

You cannot connect to Samba v2+ shares with Windows 8+. This has been brought up before and all you get is your thread being hijacked by anti-Microsoft people. Hopefully RouterOS 7 will have proper v2 support at a minimum.

So friend, let's talk ... SMB over the Internet ... really ...

Basically don't do it. I know SMB 3 has encryption support but I personally wouldn't rely on the server and client negotiating it all correctly. You indicated you are running a VM at OVH. Have you thought about going a different route. Since you already have a VM maybe spin up an OwnCloud or NextCloud box. You may find this a much nicer tool to share files over and via the Internet. It has integration with the Linux, Mac and Windows desktop environments along with support for your phones. You'd also get all of the Dropbox like features to leverage as well.

https://owncloud.org/features/

Alternatively and only if you are truly stuck on making SMB work, update the SMB server process on your OVH VM to a version of 4.3+. Fedora 25/26 default packages in the repo will get you there. Alternatively there is an extra repo you can add to CentOS to get the latest packages. I'm not sure how friendly that is though. Personally I run a lot of Fedora so I'd just go that route.

https://apps.fedoraproject.org/packages/samba

I am thinking about tunelling this with VPN but it will slow down because of this.

OR. Can I create another interface in router with different WAN IP and create a 'rule' where specified local network clients will be bypassing NAT and connecting directly to SMB server from different WAN IP? (i have a few spare IPs) So whole regular internet connection will be NATted on WAN1 but if someone want to connect to IP xxx.xxx.xxx.xxx it will bypass NAT and go through WAN2 port? Can i do that?

I don't see a benefit in doing that. You wouldn't need to chew up a physical interface. Could you respond more directly to the questions I posed in my initial response?

You indicated you are trying to connect to a host via SMB, the host is a simple Linux server. Can you verify that the server is hosted at a cloud provider? The client machine is behind the MikroTik and connects to the host via the Internet through NAT? Is your version of Samba on the Linux server is at least 4.3 or greater and configured appropriately to support newer Windows clients?

Have you put any thought into my posts about the security risks using SMB especially SMBv1 over the Internet? A solution like VPN would definitely help regardless of the protocol version of SMB. If you can tell us more about why you want to use SMB over the Internet we can help you design and choose the best solution to your problem.

IMHO
A. You should look at MSS parameter and tailor MTU sizes.
B. I think that problem is in packet fragmenting
C. Could be out of order problem.

Could you respond more directly to the questions I posed in my initial response?

I need to mount a drive in windows clients so people can work on this files like it was a NAS share. I thought samba would be good for this if i make sure noone else will have access to this (smb server will allow only this specific IP client). I have 500 Mbps symmetrical internet connection but download from this server is only 16MB/s so i thought NAT is maybe slowing it down somehow? In meantime i changed mikrotik router to RB3011 for more power but this didnt help.

So thats why i want to bypass NAT specifically for connection with that server but i dont know how. I am still in test phase, we are just testing this possibility but i can take suggestions if there is better way to achieve that - i just need a samba share.

BTW samba version on server is 4.2.14

If you read my post you'll see, Samba needs to be upgraded for the clients to connect. Either way it's unsafe. Adding a VPN will alleviate the risk of transmitting credentials over the Internet as will the encryption features in newer protocols. But will only continue to the lower performance.

NAT can affect performance. You can only bypass NAT by making the server local, either by physically moving the server or by VPN. If you VPN the CPU hit will much higher than NAT for the traffic although much safer. Also the 16MBps could be the CPU of the VM capping out. I'd check that too. Alternatively placing IPv6 on the Samba server as well as the clients would bypass NAT without a VPN.

You may find wholly better performance with an OwnCloud / NextCloud solution. The files are stored locally on the machines and synced between the client and server. You can also choose which files are synced locally. Just a thought. I try to lean away from SMB shares whenever possible. Give everyone an account on the OwnCloud server and let them share files as needed amongst themselves like you would with DropBox. You still control the data and it is kept local for fast editing, diverse client support and secure.