Agreed, but it should be better than whats available currently. I get terribly wrong code atm.Count me as doubtful that "accurate and effective configurations" be as simple as plugging in more specific training data, but IDK.
Thank you normis, I will use this and provide feedback.It would be interesting to see if training a model on the documentation and forum would work.
here is the PDF, it's on the documentation page: https://box.mikrotik.com/f/7d18a1be34c241b4adaa/?dl=1
Not sure what you have against AI but whats been compiled could geniunely be helpful to someone new to Mikrotik and networking in general like me. AI is clearly useful in certain things and it has certainly helped me so just sharing what I could. If you don't like it, feel free to ignore?Please do not throw in the AI generated crap asking to review that "job".
I disagree, all you will do is increase the illiteracy of MT users.Not sure what you have against AI but whats been compiled could geniunely be helpful to someone new to Mikrotik and networking in general like me. AI is clearly useful in certain things and it has certainly helped me so just sharing what I could. If you don't like it, feel free to ignore?Please do not throw in the AI generated crap asking to review that "job".
What I shared is not a "job" and I don't get anything from your review, the community benefits.
I don't see many Cheat Sheets and Commands listed floating around. If AI has done half the job, even if its "crap" we could improve it further and make it useful.
You describe AI outputs as "s--t" multiple times but if it were truly worthless, it wouldn't pose any threat to the community. What I think is actually bothering you (and it's a legitimate concern) is that AI has become good enough to:The problem with posting s–t generated here on the forum is that, in addition to the s–t generated by real people,
it adds to the material with which intelligence is trained and users (the few) who search for something on the forum also find the remains of s–t from artificial intelligence.
Not to mention that people, that freely help, often find out later that the configuration or script for which they donated free time,
was actually made initially by artificial intelligence and must correct its mistakes,
instead of having those who use it learn something...
There is a risk that the forum, from a place of comparison and education between people,
becomes a place to correct the errors of artificial intelligence...
Really? How so? The AI has essentially just summarized information from the official documentation. Making concise information more accessible doesn't create illiteracy - if anything, it helps people get started who might otherwise be overwhelmed by dense technical documentation. Many people learn better with simplified explanations before diving into more complex details.I disagree, all you will do is increase the illiteracy of MT users.
Not sure what you have against AI but whats been compiled could geniunely be helpful to someone new to Mikrotik and networking in general like me. AI is clearly useful in certain things and it has certainly helped me so just sharing what I could. If you don't like it, feel free to ignore?
What I shared is not a "job" and I don't get anything from your review, the community benefits.
I don't see many Cheat Sheets and Commands listed floating around. If AI has done half the job, even if its "crap" we could improve it further and make it useful.
Exactly, many users, not all users...[*]Create content that many users can't distinguish from human expertise
Maybe, never seen (well enough).[*]Produce solutions that work well enough to be implemented
I am among the many who have written a ton of scripts in the forum,[*]Capture value from your knowledge and corrections without compensation
I would also like to explain that I am not against AI, I am against those who do not understand it,doesn't justify dismissing the technology outright.
Ah It's refreshing to hear from someone who's actually explored what good AI can do! Claude 3.7 is indeed remarkable.I very much don't want to inflame, or feed the existing flames on the topic of AI, but, in case it is of value to anyone, for the past week or so I have been working with claude.ai (their paid 3.7 version) and it is, IMHO, fantastic.
Claude.ai explains topics truly beautifully, is infinitely patient with my questions, and generates fully commented and (I believe) good quality code.
With the help of others here (you know who you are), claude.ai, a little test lab, and herculean efforts (to overcome my substantial learning disorder), I am progressing nicely in my learning.
I would be happy to share some of the config, but only on request, as I am sensitive to the admonitions against posting AI generated code.
How many people I found going the wrong way,Experienced travelers still bring their expertise to interpret the GPS suggestions, understanding when to follow them and when human judgment should override.
Which some time ago I asked you about (and again, it was and still is a genuine honest question) and you said there was no such problem ?With the help of others here (you know who you are), claude.ai, a little test lab, and herculean efforts (to overcome my substantial learning disorder), I am progressing nicely in my learning.
I understand the point you're making about knowing the basics, and I agree that foundational knowledge is valuable. However, I think you're focusing on edge cases to dismiss tools that benefit the vast majority of users in the vast majority of situations.As it happened to be I used CoPilot today to get some user info with specific parameters out of an Active Directory environment.
But it had some flaws which I was easily able to correct, because I know my basics.
Funny you mention GPS ... there are parts in the world where GPS doesn't work. You wouldn't believe it but it's true.
I would think France is not considered a 3th world country, however there are plenty of blind spots there in the woods.
What if you do not have a normal map with you ?
I know plenty of people really getting LOST without GPS. They don't know about the sun, where it is more or less at a given time of day. They have no sense of directions. They never learned to read a map either (not even when it's on their smartphone offline).
The basics.
But it's RouterOS scripting where these LLM falter. Stuff like JavaScript is 10000x more common & on top there is plenty of linter to pre-check output of LLM for JavaScript validity. I'd have to imagine more LLM take advantage of "LSP" (see https://langserver.org), while open standard for presenting code in editor it can be consulted at any point in AI workflows. But since there isn't a LSP, all the LLM have to "fly blind" on what's actually valid RouerOS script/config.
Well, you started it!What a discussion !!!![]()
The smiley emoji indicates it was a joke. And yes, it referenced your comments.Which some time ago I asked you about (and again, it was and still is a genuine honest question) and you said there was no such problem ?With the help of others here (you know who you are), claude.ai, a little test lab, and herculean efforts (to overcome my substantial learning disorder), I am progressing nicely in my learning.
Interesting idea to try the dude chat.If you're messing with Claude, it might be worth it to try "The Dude" chatbot at https://mikrotik.com/support/. It can answer network questions like "what a trunk port?", and it was also trained on Mikrotik's docs. While I don't think pre se be "better" that just using Claude/etc, but it's starting context may be better (i.e. it know you're likely going to ask a networking or Mikroitk question).
But it's RouterOS scripting where these LLM falter. Stuff like JavaScript is 10000x more common & on top there is plenty of linter to pre-check output of LLM for JavaScript validity. I'd have to imagine more LLM take advantage of "LSP" (see https://langserver.org), while open standard for presenting code in editor it can be consulted at any point in AI workflows. But since there isn't a LSP, all the LLM have to "fly blind" on what's actually valid RouerOS script/config.
So if you ask me... better AI with RouterOS config start with an LSP. And LSP be useful since a real person can use it check their own code and get "hints" etc. Win, win.
+1. I'd like to here an experts analysis of this which is from Claude after incorporating @CGGXANNX's suggestions: viewtopic.php?p=1130715#p1130715I am in no way an expert at ros but I would really like to hear an expert’s analysis of Claude’s configuration advice.
It sure seems to me that it is getting the config correct, and I’ve been asking it for more and more complex solutions.
Not "old," but rather "wise and experienced."Only to show how old I am, something very similar to this AI thingy was once called an "expert system".
You feed the system with correct data, and then it can make every kind of cheatsheets, digests and provide correct answers to correct questions.
The approach works just fine.
Most of these new AI's have been instead fed with anything that was available, shaken and mixed well, and then the expected result is correct answers.
It is a logical impossibility.
The answers may be correct (or they may be not) but not only there is no way (without checking directly) to know, but there isn't even an approximate estimation of the probabilities, instead everything is provided with a somehow "absolute" tone, the AI - probably in good faith - thinks it is right, but it doesn't really know.
Very likely - before or later - it will evolve (actually better programmed) to the point of being capable of understanding when it is talking bu**sh*t and shut up or becoming capable to simply admit it is not so sure about something, but till then its only meaningful use is for anything that is:
a. not important (like funny or not funny pictures)
b. not otherwise "exact" (like programming or scripting)
Not that humans are largely particularly good at programming or scripting, let alone document what they do.
So if you ask me... better AI with RouterOS config start with an LSP. And LSP be useful since a real person can use it check their own code and get "hints" etc. Win, win.
You both clearly know far far more than I do about AI, LSP, LLM, ROS, etc.So if you ask me... better AI with RouterOS config start with an LSP. And LSP be useful since a real person can use it check their own code and get "hints" etc. Win, win.
Since today's LLMs also understand syntax definitions and can be trained accordingly, it would be great if Mikrotik could publish an official BNF for RouterOS scripting for this purpose. If Mikrotik uses Yacc/Bison or a similar tool internally, it might not be too much work to provide a BNF.
If MikroTik published an official BNF for RouterOS scripting, it would help developers and tools, including LLMs, better understand and work with the scripting language. This would lead to smoother implementations of automation tools for network management and monitoring, enable better code analysis in external IDEs, and provide more useful learning resources for the MikroTik community.
Ps..
I plan to open a thread and make it an official feature fequest.
No better than the references already provided by others, I think you are more comfortable/receptive with a sexy AI presence ;-P )I remain very happily surprised at how well it understands and explains VLANs and ROS.
I have an opinion nonetheless (they're free, after all): Why not check out claude.ai's current level of understanding before concluding how to make AI more accurate and useful for ROS config help? My analysis on how well claude.ai does with ROS is not worth the electrons used to express them here, but you both, and many others here, could form a more much valuable analysis of the current state -- and thereby a better path forward.
I was using the same definition of script as used here when we have separate forum for scripting.configuration=script
Not really, configuration = tiny subset of scripting language (without variables, conditional execution, etc.).configuration=script
This statement is two-edged.
- Create content that many users can't distinguish from human expertise
Santa Vergine, Dio del cielo ... COBOLLA reinventati !!!
No worries, it was gibberish, I didnt understand the meaning of the Italian translation either '=)Incidentally, the quick AI translation did a terrible job of dealing with @rextended's "Maremma cinghiala", even wondering whether it could be Indonesian rather than Italian.
That's ok, also many Italians not born in Tuscany would have troubles understanding It, they would of course understand the words, but not fully appreciate the meanings (that sentence can have more than one depending on context and tone).No worries, it was gibberish, I didnt understand the meaning of the Italian translation either '=)
..In response to "AI will soon replace rextended, the baddass script guru"
If I tried to translate it into English it would be "Come on, I don't believe it!"
....In response to "I met a woman at a bar last night, nudge nudge wink wink, the next morning she told me her name was Giorgi M"
If I tried to translate it into English it would be "Come on, I don't believe it!"
Yep, but it all depends on the tones, and how they can be rendered with punctuation, that would probably be:If I tried to translate it into English it would be "Come on, I don't believe it!"
Left or right? Canadians firstLuv those that complain but dont offer anything better............. booo!
You got that backwards -- if it wasn't sh*t then it wouldn't pose a treat but would be genuinely useful.You describe AI outputs as "s--t" multiple times but if it were truly worthless, it wouldn't pose any threat to the community.
No, more like create content that many users can't verify at the speed and scale that is prohibitively expensive even for actual domain experts to verify. In this case it is also placing a burden on existing MT professionals that now have to waste time verifying AI answers and dealing with people complaining that AI "solutions" don't work.- Create content that many users can't distinguish from human expertise
In networking and especially in network security there's no such thing as "well enough" -- you either have a 100% correct setup or you are risking intrusion, becoming part of botnet, or worse, being a victim of identity theft or your resources being used for criminal activity which can even make you liable or at least greatly inconvenience you until you have proven it wasn't you.Produce solutions that work well enough to be implemented
Everything the LLM models were trained on was STOLEN. Not a dime of compensation was paid to anyone, nor has anyone's consent been asked before STEALING. And I am talking here about copyright which you do not relinquish when you publish something -- this post of mine is copytighted by me despite being posted on the public message board. I am not giving permission to train ML on its contents. If you do it, you are breaking existing copyright law. The law may be bad, but that's how it currently works, and all those LLMs have broken it on a massive scale.Capture value from your knowledge and corrections without compensation
First, the solution to "making AI even better" isn't throwing more data in it. Every process has its limits and LLMs have reached their limit in what they can do. What should justify dismissing the technology is that whatever it produces is the fruit of the poisonous tree or if you prefer more colorful analogies then money they earn by selling subscriptions to it is blood money -- ill-gotten gains and all that.It's not that AI content is worthless, it's that your expertise, which took years to develop, is being approximated without proper recognition or compensation. And when you freely offer corrections, those improvements get fed back into making AI even better. That's a legitimate concern but doesn't justify dismissing the technology outright.
It still reeks of someone pissed about compensation. Read your own words again - it's all about compensation and stolen data. Yet here's what you're missing: AI is indeed good enough - or I would say damn near close to perfect in many domains.You got that backwards -- if it wasn't sh*t then it wouldn't pose a treat but would be genuinely useful.You describe AI outputs as "s--t" multiple times but if it were truly worthless, it wouldn't pose any threat to the community.
No, more like create content that many users can't verify at the speed and scale that is prohibitively expensive even for actual domain experts to verify. In this case it is also placing a burden on existing MT professionals that now have to waste time verifying AI answers and dealing with people complaining that AI "solutions" don't work.- Create content that many users can't distinguish from human expertise
In networking and especially in network security there's no such thing as "well enough" -- you either have a 100% correct setup or you are risking intrusion, becoming part of botnet, or worse, being a victim of identity theft or your resources being used for criminal activity which can even make you liable or at least greatly inconvenience you until you have proven it wasn't you.Produce solutions that work well enough to be implemented
Everything the LLM models were trained on was STOLEN. Not a dime of compensation was paid to anyone, nor has anyone's consent been asked before STEALING. And I am talking here about copyright which you do not relinquish when you publish something -- this post of mine is copytighted by me despite being posted on the public message board. I am not giving permission to train ML on its contents. If you do it, you are breaking existing copyright law. The law may be bad, but that's how it currently works, and all those LLMs have broken it on a massive scale.Capture value from your knowledge and corrections without compensation
I know that the people like you who evangelize AI will now jump to defend this act by saying that it doesn't differ from how humans learn and let me tell you right away that's not true. We humans buy a book to read its contents or borrow it from a public library which is paying a compensation to authors to allow that to happen. LLMs like Meta's llama were trained on PIRATED books downloaded from the torrent sites.
First, the solution to "making AI even better" isn't throwing more data in it. Every process has its limits and LLMs have reached their limit in what they can do. What should justify dismissing the technology is that whatever it produces is the fruit of the poisonous tree or if you prefer more colorful analogies then money they earn by selling subscriptions to it is blood money -- ill-gotten gains and all that.It's not that AI content is worthless, it's that your expertise, which took years to develop, is being approximated without proper recognition or compensation. And when you freely offer corrections, those improvements get fed back into making AI even better. That's a legitimate concern but doesn't justify dismissing the technology outright.
Finally, the output is as good as input -- it will never get better or otherwise alchemists would have been able to turn lead into gold long time ago.
YepWhich means it needs to be regulated 100%
And your argument seems to imply the issues I pointed out are nothing more than sour grapes.It still reeks of someone pissed about compensation. Read your own words again - it's all about compensation and stolen data. Yet here's what you're missing: AI is indeed good enough - or I would say damn near close to perfect in many domains.
Thing is, those 80% of setups can be covered by default manufacturer configuration. People who come here and use Mikrotik do not fall in those 80%.If they did train AI thoroughly on networking protocols and MikroTik configurations, it would definitely provide near-perfect solutions to 80% of setups.
The difference is that a human posting a configuration example will do so based on their experience so it will be a copy of a working and tested setup. Decent human beings will also check what they share before posting, perhaps even make a quick lab setup to make sure it works. Not to mention that if you pay someone for advice like you are paying the LLM subscription they are obligated to provide a proper service.Your "100% correct or nothing" argument ignores that humans make configuration errors constantly - the difference is AI can be systematically improved.
It's not either / or -- it's a causative relationship. AI is sh*t because it is hard to verify even for people who have the domain knowledge, let alone for people who are clueless.Your argument about verification is just moving goalposts - first it's "AI is shit," then it's "well, it's too hard to verify."
That's not even remotely similar. If you type 2+2 into a calculator it WILL always output the same result 4. Ask the AI the same thing several times and confront it and suddenly you will have several differently worded responses. In order to understand why that's a problem you'd need to understand how LLMs work which seems to be out of reach of your mental capabilities or you wouldn't be singing this blind praise.That's like saying calculators are worthless because someone needs to verify the inputs.
So your argument is "the end justifies the means"?The theft argument might have legal merit, but it doesn't change the technical reality. AI is getting better regardless of how you feel about its training.
My expertise is not in danger at all as what you call "AI" (i.e. a LLM) will never be able to write the code like I do -- my sanity and patience are because AI generated trash has permeated all spheres of life.And honestly, if your expertise can be "approximated" this easily, maybe the real concern isn't the AI.
You're really good at putting words in my mouth while ignoring the actual technical reality. The Dunning Kruger accusation is ironic coming from someone dismissing an entire technology because it threatens their worldview.And your argument seems to imply the issues I pointed out are nothing more than sour grapes.It still reeks of someone pissed about compensation. Read your own words again - it's all about compensation and stolen data. Yet here's what you're missing: AI is indeed good enough - or I would say damn near close to perfect in many domains.
As for the "good enough" claim -- that depends on how much you know about those many domains which, judging by how you praise AI, doesn't seem to be much. In short, typical Dunning-Kruger in effect.
Thing is, those 80% of setups can be covered by default manufacturer configuration. People who come here and use Mikrotik do not fall in those 80%.If they did train AI thoroughly on networking protocols and MikroTik configurations, it would definitely provide near-perfect solutions to 80% of setups.
The difference is that a human posting a configuration example will do so based on their experience so it will be a copy of a working and tested setup. Decent human beings will also check what they share before posting, perhaps even make a quick lab setup to make sure it works. Not to mention that if you pay someone for advice like you are paying the LLM subscription they are obligated to provide a proper service.Your "100% correct or nothing" argument ignores that humans make configuration errors constantly - the difference is AI can be systematically improved.
It's not either / or -- it's a causative relationship. AI is sh*t because it is hard to verify even for people who have the domain knowledge, let alone for people who are clueless.Your argument about verification is just moving goalposts - first it's "AI is shit," then it's "well, it's too hard to verify."
That's not even remotely similar. If you type 2+2 into a calculator it WILL always output the same result 4. Ask the AI the same thing several times and confront it and suddenly you will have several differently worded responses. In order to understand why that's a problem you'd need to understand how LLMs work which seems to be out of reach of your mental capabilities or you wouldn't be singing this blind praise.That's like saying calculators are worthless because someone needs to verify the inputs.
So your argument is "the end justifies the means"?The theft argument might have legal merit, but it doesn't change the technical reality. AI is getting better regardless of how you feel about its training.
My expertise is not in danger at all as what you call "AI" (i.e. a LLM) will never be able to write the code like I do -- my sanity and patience are because AI generated trash has permeated all spheres of life.And honestly, if your expertise can be "approximated" this easily, maybe the real concern isn't the AI.
For example, some 3D asset creatores have started using AI generated images as textures for 3D assets and are selling those assets for $15. In reality, the "quality" of those generated textures makes them not worth buying, let alone paying that much. Worse yet, those assets are not labelled as AI generated and I now have to wade through a quagmire of that crap to find useful stuff.
Did you try Google search recently? Not just the AI summaries, but AI answers (most of which are flat out wrong), but the veritable torrent of AI written "articles" abusing SEO to farm ad revenue making it all but impossible to find what you are looking for on the first page or even at all.
And you call that progress? All I see is ensh*tification of everything.
typists had toward word processors
switchboard operators had toward automated exchanges
typists had toward word processors
switchboard operators had toward automated exchanges
Two examples that require determinism, against something not determinate, as you yourself wrote.
I don't want a word processor to correct my sentences as someone else likes, just because someone else whose LLM has stolen the work has to put it in mine...
I don't want a switchboard to call me from a different number than the one I dialed, because usually at that time I call someone and they think I've got the wrong number...
AI is unreliable in principle precisely because in the exact same context it gives different solutions.
To do 2+2, you have to do 2+2 (ignoring how it is actually done by the calculator circuit, always the same).
I don't care that today 2+2 is 16/8*32/16 and tomorrow it tells me that it is √16 leaving me to check why it didn't write directly 4.
Most of the produced codes are full of useless operations, it looks like Windows source code...
Not to mention that he never admits that he doesn't know how to do it, making things up,
if an command doesn't exist, and no document provided has it, he makes it up from scratch... just to please the writer.
They are just chat machines, and it is obvious that you are into them, you probably have no one else to talk to.
How cute, the emotions, the love, the envy, the joy...Your emotional resistance won't change that trajectory.
There is no need for me to explain the sentence, it is clear what is written.I don't want a word processor to correct my sentences as someone else likes, just because someone else whose LLM has stolen the work has to put it in mine...
@3zzy, your example is perfect, in the sense that millions of people commit daily blunders of spelling and grammar simply because they accept what the software tells them; they know no better, or do not stop to consider.... you don't want a word processor correcting your sentences? That's literally what millions of people rely on daily with spell check and grammar tools.
Exactly wrong. They "probably" work meaning there is an error rate. You can count on that, it is practically determined.And they work because they're probabilistic, not deterministic.
@rextended, you are in the right TRACKThis thing made me think so much,
.............
.............
It turns out in reality that in the spaceship only two families were left alive (coincidentally him & her) and they will have to revive humanity on that new planet...
It's a perfect example showing how simple human abilities are falling down for millions of people.@3zzy, your example is perfect, in the sense that millions of people commit daily blunders of spelling and grammar simply because they accept what the software tells them; they know no better, or do not stop to consider.... you don't want a word processor correcting your sentences? That's literally what millions of people rely on daily with spell check and grammar tools.
...
I truly and profoundly HATE youtube instruction videos.Reading ... oh man ... what are the YT, TT and other platforms for? "Do not want to read!, I WANT TO WATCH AND LISTEN TO!"
I did no such thing -- I quoted what you said and responded to each specific bit unlike you.You're really good at putting words in my mouth while ignoring the actual technical reality.
"If all you have is a hammer, everything looks like a nail." -- Abraham Maslow, 1966 (but there are variations dating back to 1868.The 80% figure isn't about "default configurations" - it's about the complex, custom setups that MikroTik users actually need. What you're missing is that pattern recognition is exactly what AI excels at, and networking configurations follow patterns.
And where, pray tell, will your "AI" read the correct information to improve itself once it ingested the incorrect one? Unless the original author updates their post and "AI" is retrained on it how do you expect it to improve? Also, what incenitive will there be for anyone to post anything useful if it will all be gobbled up by a machine without compensation? Me thinks your "AI" is an idealistic fantasy, not what I said. You fail to understand that if "AI" is trained on a sum of human knowledge it can only be as-good-as that knowledge, never better. But by all means, keep drinking the cool aid from "Open" "AI" and the likes.Your "working and tested" human example is idealistic fantasy. Have you seen the garbage configurations people post online? At least AI improves systematically over time, whereas human errors persist indefinitely.
It wasn't my calculator comparison -- it was yours. I just made it make sense, but you don't seem to like the result. Second, what you call "AI" is deterministic -- if you pass the same random seed you will get the same output every time.Your calculator comparison misses the point entirely. AI isn't deterministic by design - that's a feature, not a bug. It's meant to adapt to different contexts and requirements, just like human expertise does.
Even if the technology works (and it definitely doesn't work the way you seem to believe hence my Dunning-Kruger remark), there's no legal framework to justify Meta pirating 82TB of books to train it.The "end justifies the means" strawman is deflection. The legal framework for data usage is still evolving, but that's separate from whether the technology works.
I am dismissive because I understand the limitations of LLM since I had to understand how it works in order to properly evaluate its usefulness (or the lack of it).Your skills may be safe for now, but your attitude reeks of the same dismissiveness that typists had toward word processors or switchboard operators had toward automated exchanges.
Good luck with that, can't wait for you to tell us your IP address after "AI" creates your "secure" configuration.So feel free to keep raging against the machine while the rest of us figure out how to use it productively.
I don't mind the YT videos per se. What's annoying is I know between scripting and editing, it a lot more work than maintain good docs and other WRITTEN communication. Just for example... take @normis video on Let's Encrypt, from two years ago. Anyone who watched that video be left with expired certificates today. So instead of update the docs with some canonical "Mikrotik approved" way/script of renewing certs, which isn't very hard (and while some ACME TLS auto-renew is ideal — docs can fill in those gaps BUT don't). Instead, we've got more videos and device-mode* (*instead of being able to use TLS out-of-the-box).I truly and profoundly HATE youtube instruction videos.Reading ... oh man ... what are the YT, TT and other platforms for? "Do not want to read!, I WANT TO WATCH AND LISTEN TO!"
Total waste of time in most cases.
I can read faster then they can explain it in a video. Even in English.
And when it's a digital manual, even better. Then I can search.