Using two arrays, process the text and create a third array

DyadyaGenya · Wed May 24, 2023 11:53 pm

I have text. It repeats a phrase, which consists of an alphabetic part and a digital part. The letter part does not change, but the digital part changes and the numbers in it are separated by commas. Something like this: "bla-bla 10,2,,125" or "bla-bla 2,1,,80" or "bla-bla 7,0,,8080". There is one array that contains the indexes of the occurrence of the searched phrase. There is a second array with the length of the phrase. Tell me how to use these arrays to process the text in such a way that a new multilevel array is obtained, in which there is the first number before the comma from the phrase and two other digits from this phrase are attached to it. Unfortunately, I don’t understand at all how multilevel arrays are written in Mikrotik.
It is clear that you can cut off the phrase using

:local numbers [:pick "text" <start>[<count>]]

, where "start" is the data from the first array plus the number of letters in the phrase (bla-bla = 7 characters), and "count" is the data from the second array minus the number of letters in the phrase, i.e. 7.
The cut phrase is also easy to put in an array like this

:local newNumbers [:toarray $numbers]

But how to create a new array using this? And the array should turn out something like this (unfortunately I don’t know how to write a multilevel array correctly)

[
[10[2,125]]
[2[1,80]]
[7[0,8080]]
]

That is, so that the first numbers are of the index type

DyadyaGenya · Thu May 25, 2023 12:14 am

Perhaps that will be clearer. Here is a piece of text in which you need to do a search

bla-bla 10,2,,125
43800200437043C043E04360435043C043E002107918350000014F0600ED0D63739500030C07070438043C04300439043C043E0441044F00210020042004300437043E043C0020043C06C7EBBCB0008221120513141803C0
bla-bla 2,1,,80
BBCB0008221120513141808C0500030C070104140440044304370456002C0020044707918350000014F0400ED0D637396C7E04350440043504370020043F043E043204420443E0440043E0433043000200432002004340435044F043A0438044500200440043504330456043E043D0430044500200423043A044004300457043D043800200432045604340441044304423E0440043D04560020043004420430043A0438002004320
bla-bla 7,0,,8080
120513141808C0500030C070207918350000014F0400ED0D637396C7EBBCB0008221043D045400200435043B0435043A004470430043D043D044F002E0020041C043E0436044304420440043E043F043E044104420430442044C002004320438043D0438043A043004420438002004423E0432045600200441043A043B04300434043D043E044904560020043704560020043704322019044F0437043A0438043C04470430044104

Here are two arrays obtained from this segment.

:local start [0,197,552]
:local len [18,17,19]

rextended · Thu May 25, 2023 12:28 am

It's not clear at all, you don't give its purpose or usefulness, so I don't care.

I've already answered this:
viewtopic.php?p=1004108#p1004085
but I still don't see the usefulness for everyone, so if you need such a particular script with a purpose unknown to us in the forum,
I suggest you hire a consultant, if you really need it...

optio · Thu May 25, 2023 12:37 am

Code: Select all
[
[10[2,125]]
[2[1,80]]
[7[0,8080]]
]
That is, so that the first numbers are of the index type

For your example:

# init main array
:local arr {"10"={2;125}; "2"={1;80}; "7"={0;8080}}
# extracts inner array for index 10 (actually is key)
:local arr10 ($arr->"10")
# adds 1345 into inner array
:set arr10 ($arr10, 1345)
# sets updated inner array to main array
:set ($arr->"10") $arr10

If you need new inner array at some index, ex. 44:

# init new inner array
:local arr44 {77;0}
# sets new inner array to main array at index 44
:set ($arr->"44") $arr44

And you can go in depth as like you want...:

:local arr {"10"={"1"={2;125}; "2"={3;34}}};
:put ($arr->"10"->"1")
:put ($arr->"10"->"2")

etc...

rextended · Thu May 25, 2023 12:56 am

This is example of little "nested" (

) array of arays of arrays and other non array inside other arrays...
viewtopic.php?p=973659&hilit=json#p973295

for convert this
[
[10[2,125]]
[2[1,80]]
[7[0,8080]]
]
to
new array with on index 2 another array of values 1 and 80, on index 7 another array of values 0 and 8080, on index 10 another array of values 2 and 125
there are some ways to convert that on one array of arrays:

examples code

{
:local newarr {2,125;1,80;0,8080}
:put ($newarr->0->0)
:put ($newarr->0->1)
:put ($newarr->1->0)
:put ($newarr->1->1)
:put ($newarr->2->0)
:put ($newarr->2->1)
}

{
:local newarr [:toarray ""]
:set ($newarr->10) {2;125}
:set ($newarr->2) {1;80}
:set ($newarr->7) {0;8080}
:put ($newarr->10->0)
:put ($newarr->10->1)
:put ($newarr->2->0)
:put ($newarr->2->1)
:put ($newarr->7->0)
:put ($newarr->7->1)
}

{
:local newarr {"10"=2,125;"2"=1,80;"7"=0,8080}
:put ($newarr->"10"->0)
:put ($newarr->"10"->1)
:put ($newarr->"2"->0)
:put ($newarr->"2"->1)
:put ($newarr->"7"->0)
:put ($newarr->"7"->1)
}

DyadyaGenya · Thu May 25, 2023 12:58 am

I've already answered this:

Your answer helped a lot to create an array that contains the length of the phrase. That is, the array len
The general meaning of the future script is to learn how to parse SMS. In a long-standing situation, this will be the SMS number, its status, length. Based on it, I hope to learn how to parse the rest of the SMS. I know that there are topics on the forum in which SMS is sent, for example, to telegrams or mail. But I don't need it. I just want to get some information about sms.

optio · Thu May 25, 2023 1:06 am

This is example of little "nested" ( ) array of arays of arrays and other non array inside other arrays...
viewtopic.php?p=973659&hilit=json#p973295

for convert this
[
[10[2,125]]
[2[1,80]]
[7[0,8080]]
]
to
new array with on index 2 another array of values 1 and 80, on index 7 another array of values 0 and 8080, on index 10 another array of values 2 and 125
there are some ways to convert that on one array of arrays:
...

Yeah, I was editing my post with additional examples, you were faster... Arrays and maps can be created and manipulated in ros script in various ways.

rextended · Thu May 25, 2023 1:09 am

I just want to get some information about sms.

Good fortune...
viewtopic.php?p=797583#p797583

I just done this two parts:
Function to convert Hex GSM7 to CP1252 string (for decode the PDU message part on GSM7)
viewtopic.php?t=177551#p992792
and
Function to convert one CP1252 (or simply ASCII-7-bit) string to one Hex GSM7 string (for create the PDU message part on GSM7)
viewtopic.php?t=177551#p992677

and also this two for UCS-2
Function to convert UCS-2 to UTF-8 (for decode the PDU message part on UCS-2)
viewtopic.php?p=983695#p983695
Function to convert UTF-8 to UCS-2 (for create the PDU message part on UCS-2)
viewtopic.php?p=985041#p985041

But only the author on the first link have patience to decode the rest of the PDU message....

optio · Thu May 25, 2023 1:24 am

I just want to get some information about sms.
Good fortune...
viewtopic.php?p=797583#p797583

I just done this two parts:
Function to convert Hex GSM7 to CP1252 string (for decode the PDU message part on GSM7)
viewtopic.php?t=177551#p992792
and
Function to convert one CP1252 (or simply ASCII-7-bit) string to one Hex GSM7 string (for create the PDU message part on GSM7)
viewtopic.php?t=177551#p992677

and also this two for UCS-2
Function to convert UCS-2 to UTF-8 (for decode the PDU message part on UCS-2)
viewtopic.php?p=983695#p983695
Function to convert UTF-8 to UCS-2 (for create the PDU message part on UCS-2)
viewtopic.php?p=985041#p985041

But only the author on the first link have patience to decode the rest of the PDU message....

I guess I need to update my sms2email script with this.

Maybe also checking UDH header for joining splitted messages will be nice

DyadyaGenya · Thu May 25, 2023 1:27 am

Good fortune...

Thank you. I have seen this thread. It contains a lot of redundant information. And there is no one that interests me. But I already said that I don't need someone else's function. I want to figure it out myself. Maybe later I will want to forward the SMS somewhere, but for now it's enough for me to read the SMS. And in my opinion it would be much more correct to put the read SMS in an array. So it would be more convenient for further work with him. The author of that project does not do this. The structure is more rigid than I would like. Maybe I'm wrong and I don't understand all the code in that thread. In any case, I would like to learn how to read SMS myself.
Maybe I'll use your code snippets later for transcoding, for now I just want to split the sms and put it in an array.

DyadyaGenya · Thu May 25, 2023 1:30 am

For your example:

Thanks for your examples. I'm starting to understand a little. Of course, I'm still far from fully understanding, but these examples are beginning to help.

optio · Thu May 25, 2023 1:56 am

For your example:
Thanks for your examples. I'm starting to understand a little. Of course, I'm still far from fully understanding, but these examples are beginning to help.

In ROS script you have one dimensional array in format

{val1; val2; val3; ...}

and key=value maps in format

{"key1"=val1; "key2"=val2; "key3"=val3; ...}

, multidimensional arrays can be achieved with maps if key is string of dimension index and value is array of that dimension or map with inner dimension, not optimal and simple as in common programing languages but possible to go in depth like that.

DyadyaGenya · Thu May 25, 2023 1:06 pm

# extracts inner array for index 10 (actually is key)
:local arr10 ($arr->"10")

I more or less understand this, but how can I access the internal data index in key 10 directly?
If I do like this:

foreach a,k in=$arr do={
  :put "$a=$k";
  }

then I get the following:

10=2;10=125
2=1;2=80
7=0;7=8080

If I do like this:

:local key10 ($arr->"10")
:put $key10

then I get the following:

2;125

But how do I get only 2 or only 125?

DyadyaGenya · Thu May 25, 2023 5:17 pm

At the moment, I have more or less figured out how to access a specific index of a nested array. There seem to be two ways, but I'm not sure which one is correct.

# init 2 main multy array
:local multy {"10"={k1={2};k2={125}}; "2"={k1={1};k2={80}}; "7"={k1={0};k2={8080}}}
:put $multy

:local multy1 {"10"={k1=2;k2=125}; "2"={k1=1;k2=80}; "7"={k1=0;k2=8080}}
:put $multy1

Both options will produce the same result:

10=k1=2;k2=125;2=k1=1;k2=80;7=k1=0;k2=8080
10=k1=2;k2=125;2=k1=1;k2=80;7=k1=0;k2=8080

We can refer to any of them like this:

:local mu ($multy->"10"->"k1")
:put $mu
:local mu1 ($multy1->"10"->"k1")
:put $mu1

Both options will produce the same result:

2
2

At the moment, I can’t figure out how to iterate over such an array, for example, using the foreach loop. The following options do nothing

foreach aa,kk in=$multy do={
  :put "$aa=$kk";
  }

foreach aaa,kkk in=$multy1 do={
  :put "$aaa=$kkk";
  }

rextended · Thu May 25, 2023 6:03 pm

You have to use the 3rd example here.
Why lost time to call first value (index 0) k1 and second value (index 1) k2?
Simply call the value...
viewtopic.php?t=196458#p1004142

optio · Thu May 25, 2023 7:25 pm

@DyadyaGenya
I'm wondering why is this all needed for processing sms inbox.
If you need some model map per sms message to process messages outside :foreach sms in=[/tool/sms/inbox find] do={...} loop, you can just create array of maps with properties of sms messages stored in that loop, which will look like:

:local smsList {{"phone"="+123456787"; "message"="sms message 1"; "timestamp"="May/25/2023 18:08:09 GMT -0"; "type"="class-0"}; {"phone"="+4356546456"; "message"="sms message 2"; "timestamp"="May/25/2023 18:08:09 GMT -0"; "type"="class-0"}}

#to get single sms at index 0
:local sms ($smsList->0)
#to get sms phone property
:put ($sms->"phone")
#to get sms message property
:put ($sms->"message")
# etc...

#or to process whole list:
:foreach sms in=$smsList do={
  :put ($sms->"phone")
  :put ($sms->"message")
}

You can achieve that with inner array instead of using properties map and get certain property by index of inner array, but for me this approach has less cognitive complexity code.

DyadyaGenya · Thu May 25, 2023 8:08 pm

You can achieve that with inner array instead of using properties map and get certain property by index of inner array, but for me this approach has less cognitive complexity code.

Maybe you're right. I haven't decided yet how it will be. In this tutorial, I'm trying to understand what arrays can do in Mikrotik scripts. One of the goals is to display a table like this:

--------------------------------------------------------------------------------|
№ |	SCA	|   timestamp  |status|			Body			|
--------------------------------------------------------------------------------|

And I understand that your foreach loop can theoretically accomplish this.
But I also want to have access to any part of the SMS from anywhere and using any index, to understand which SMS it belongs to. Perhaps your solution will be sufficient. Am I understanding correctly that it is a two-dimensional array?

DyadyaGenya · Thu May 25, 2023 8:09 pm

You have to use the 3rd example here.
Why lost time to call first value (index 0) k1 and second value (index 1) k2?
Simply call the value...
viewtopic.php?t=196458#p1004142

Unfortunately, I haven't had time to take a closer look at your example yet. Hope to do it tonight.

optio · Thu May 25, 2023 8:30 pm

--------------------------------------------------------------------------------|
№ |	SCA	|   timestamp  |status|			Body			|
--------------------------------------------------------------------------------|

then for each column you can have map property key per sms message like in my example, either you are using sms tool inbox or extracting PDU directly from modem.

And I understand that your foreach loop can theoretically accomplish this.
But I also want to have access to any part of the SMS from anywhere and using any index, to understand which SMS it belongs to.

also in example: ($smsList->0), index can be variable also, ($smsList->$index)

Perhaps your solution will be sufficient. Am I understanding correctly that it is a two-dimensional array?

Depends how you look at maps

https://www.geeksforgeeks.org/differenc ... y-and-map/. In this case its one-dimensional array of maps, but maps are linked structures in memory with unique key (which is a kind of array) and its elements are commonly accessed by keys, not by index (you can by index also with :pick in ROS) to have better cognitive readable code or unique value per key. Also maps are not ordered (in ROS), ROS is ordering map elements by key name as I see when printing, which means order can change depending which key name you add to map not sequentially when is added, that's why accessing by index doesn't make much sense.
For your table example, row is array index and column is map key at that index (except for №, which can be value of index+1).

DyadyaGenya · Fri May 26, 2023 7:04 pm

I can't say for certain which arrays I will use, but I wanted to try something very basic, specifically, obtaining the message number, its status, and length. To do this, I need to find all phrases of the "+CMGL: 15,1,,155" type and trim the alphabetic part. It seems like I've gone through all the necessary steps: found all occurrences of such phrases, created an array of these occurrences, found the length of each phrase, and created an array with their values. However, when I try to trim them, only the first phrase gets trimmed for some reason, even though there's an array of occurrence positions created next to it and some other unclear chunk. Could you please take a look and let me know where my mistake might be?

#The variable where all the unprocessed SMS messages go
:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
#Four variables for the counter.
:local matchPosition -1;
:local matchCount 0;
:local previous 0
:local continue true;
#The array where all occurrences of the found line endings are collected.
:local whereEnd ([:toarray ""]);
#The array where all the values of the phrase length, such as "+CMGL: 15,1,,155", are collected
:local arrayLen ({})
#An array where all positions of the occurrence of the phrase go
:local arrayHead ({})
#An array where all trimmed data from a phrase like "15,1,,155" should go
:local arrayStat ({})

#The loop for finding all the identified line ending characters.
while ( $continue ) do={
    :set matchPosition [ :find $modemOutput $search $matchPosition ];
    :if ( [ :typeof $matchPosition ] != "nil" ) do={
        :set matchCount ( $matchCount + 1 );
#        :put ( "Find position #" . [ :tostr $matchCount ] . " at: " . $matchPosition );
        :set $whereEnd ($whereEnd, $matchPosition)
    } else={ :set continue false; };
};

#The loop for creating an array where all the calculated lengths of phrases like "+CMGL: 15,1,,155" are stored.
:foreach key,num in=$whereEnd do={
    if (($key % 2) = 0) do={
        :put "$($num - $previous)"
        :set $arrayLen ($arrayLen, $num - $previous)
    } else={
        :set previous $num
        #:put "Find position #$key at:$num"
    }
}

#Checking the result
:put $whereEnd
:put $arrayLen



#The loop for searching and creating an array from the results of trimming the phrase "+CMGL: 15,1,,155".
:local continue true;
while ( $continue ) do={
    :set matchPosition [ :find $modemOutput $searchHead $matchPosition ];
   :put $arrayLen
    :if ( [ :typeof $matchPosition ] != "nil" ) do={
        :set matchCount ( $matchCount + 1 );
        :put ( "Find position #" . [ :tostr $matchCount ] . " at: " . $matchPosition );
        :set $arrayHead ($arrayHead, $matchPosition)
        :set $arrayStat ($arrayStat, [:pick $modemOutput ($matchPosition+7) ($arrayLen->$matchCount) ] )
    } else={ :set continue false; };
};

#Checking the result
:put $arrayHead
:put $arrayStat

Perhaps I have chosen not the most optimal approach, but I am just starting to learn scripts, and I am trying to go from simple to complex in order to develop a solid understanding.

I want to add that all the screen outputs are done for result verification purposes, to make it easier to understand where and at which stage the data exists.

rextended · Fri May 26, 2023 7:53 pm

As already written on another post, never use ({}) the result is unexpected.
The only way to create empty array is [:toarray ""] and without ( ).

DyadyaGenya · Fri May 26, 2023 7:56 pm

As already written on another post, never use ({}) the result is unexpected.
The only way to create empty array is [:toarray ""] and without ( ).

Okay, I'll fix this. I have inserted a few lines for clarification in the code. I forgot to add them when I did the translation

rextended · Fri May 26, 2023 8:09 pm

concrete example code

+CMGL: 0,1,"",48\r\n07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72\r\n
+CMGL: 1,1,"",40\r\n07912180958739F1040B917120069876F000009140503223438A17D4F29C9E769F4141EA6AD83C3241C377BB1D769341\r\n
+CMGL: 2,1,"",38\r\n07912180958739F1040B917120069876F000009140503233048A15D272980C0AB3D9A069730A6A97E7F3F0B93C07\r\n
OK

Do not split at \n (or more precisely to \r\n), split at +CMGL:

Array obtained afer filtering out the last two characters (\r\n) and the end OK at the same time:
0,1,"",48\r\n07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72
1,1,"",40\r\n07912180958739F1040B917120069876F000009140503223438A17D4F29C9E769F4141EA6AD83C3241C377BB1D769341
2,1,"",38\r\n07912180958739F1040B917120069876F000009140503233048A15D272980C0AB3D9A069730A6A97E7F3F0B93C07

Using :toarray on each line can be possible have one
array of arrays: on each array the 1st value (0) is the index number, the 2nd (1) the message status, the 3rd (2) the used memory, and on the 4th (3) the PDU message.
and now is possible to create a function that replace the PDU with one array with converted PDU contents.
the PDU start with the DECIMAL number total PDU length, the \r\n and the hex code of the PDU. The length of the hex code is the double of the decimal number that represent each hex pair.

example of one SMS array inside the main array
48\r\n07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72
==>>
+12085978931 ; SMS-DELIVER ; +17026089670 ; GSM-7 ; 19-04-05 23:32:12 TZ GMT-7 ; 21 ; "Test message to AT Command Tester"

07 91 2180958739F1 (sms center, length [7 x 2 bytes] and international format, F is a filler without value)
PDU HEADER: 04 (SMS-DELIVER)
TP-OA: 0B 91 7120069876F0 (sender number length [B = 11 characters] and international format, F is a filler without value)
TP-PID: 00 (message class 0)
TP-DCS: 00 (message text on GSM-7 format)
TP-SCTS 9140503223218A (date and time) (19-04-05 23:32:12 TZ GMT-7 [negative timezone and 28 "times" 15 minutes = 420 minutes = 7 hours ] )
TP-UDL: 21 (message lenght on septept, because is on GSM-7 format. Are octects if the mesage is on UCS-2)
TP-UD: D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72
(message encoded on octect, but internally is on septept for GSM-7, is on octect if is ASCII-8 or UCS-2)

Decode the GSM-7 or eventually the UCS-2 part is easy with my already existent functions.
Function to convert Hex GSM7 to CP1252 string (for decode the PDU message part on GSM7)
viewtopic.php?t=177551#p992792
Function to convert UCS-2 to UTF-8 (for decode the PDU message part on UCS-2)
viewtopic.php?p=983695#p983695

DyadyaGenya · Fri May 26, 2023 9:04 pm

Do not split at \n (or more precisely to \r\n)

Unfortunately, I did not have time to notice that there is a carriage return symbol here. This makes the job easier to some extent.

split at +CMGL:

As far as I understand, in MikroTik there is no built-in way to split by a specific character. Maybe I haven't found it, but it seems that it only splits by comma.
Currently, I haven't been able to split the overall SMS string into a multidimensional array using anything other than commas. However, in this case, all the results are essentially obtained in a single column. That's the approach I have used so far.

:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local matches [:toarray $modemOutput]
:foreach chast in=$matches do={
    :put $chast;
}

That's why I chose the method of searching for the length of the phrase.

array of arrays: on each array the 1st value (0) is the index number, the 2nd (1) the message status, the 3rd (2) the used memory, and on the 4th (3) the PDU message.

....................

07 91 2180958739F1 (sms center, length and international format, F is a filelr without value)
PDU HEADER: 04 (SMS-DELIVER)
TP-OA: 0B 91 7120069876F0 (sender number length and international format, F is a filelr without value)
TP-PID: 00 (no extra data)
TP-DCS: 00 (message text on GSM-7 format)
TP-SCTS 09140503223218A (date and time) (19-04-05 23:32:12 TZ
TP-UDL: 21 (message lenght)
TP-UD: D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72 (message)

I actually know what parts a SMS consists of. I even know that the phone number needs to be rearranged in pairs of digits. At the moment, my goal is to learn how to split the SMS into parts. However, I didn't understand two things: 1. How you split the modem response into a multidimensional array, and 2. Why there is an error in my code, as the other arrays are being filled.

DyadyaGenya · Sat May 27, 2023 12:05 am

I have realized my mistake. It turns out I was translating and applying the ":pick" command incorrectly. The second parameter of the command is not the length of the segment to be cut from the beginning (from the first parameter), but rather the length from the beginning of the string. Therefore, I was performing unnecessary calculations for the segment length. It is possible to directly replace the string with the message number 52

       :set $arrayStat ($arrayStat, [:pick $modemOutput ($matchPosition+7) ([ :find $modemOutput "\n" $matchPosition ]  -1 ) ] )

Thus, the entire code will now look like this:

:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search1 "+CMGL:";
:local search2 "\r\n";
:local matchPosition -1;
:local matchCount 0;
:local arrayStat [:toarray ""]

:local continue true;
while ( $continue ) do={
    :set matchPosition [ :find $modemOutput $search1 $matchPosition ];
    :if ( [ :typeof $matchPosition ] != "nil" ) do={
        :set matchCount ( $matchCount + 1 );
        :put ( "Find position #" . [ :tostr $matchCount ] . " at: " . $matchPosition );
        :set $arrayStat ($arrayStat, [:pick $modemOutput ($matchPosition+7) ([ :find $modemOutput $search2 $matchPosition ] ) ] )
    } else={ :set continue false; };
};

:put $arrayStat

Let's consider that I have figured out the first step

rextended · Sat May 27, 2023 12:18 pm

Replaced with:
viewtopic.php?p=1005348#p1005348

DyadyaGenya · Sat May 27, 2023 5:12 pm

Ok, if you want one hand, since now the goal is sufficently precise:

I'm still thinking and deciding what would be more convenient. But your option looks if not exactly how I want it, then very close to what I want
I have a couple of clarifications regarding your code, although they are not directly related to it.
1. How did you determine that the excerpt '\r\n+CMGL: ' at the beginning of the string contains carriage return and line feed characters, and that the excerpt '\r\nOK' contains some 'OK' and what does that 'OK' mean? I haven't seen such Constant Escape Sequences.
2. I used to see the 'alpha' field before. It was always empty. At least I haven't seen it non-empty anywhere. And I haven't come across any explanations about what it is. In the screen output, you wrote 'phonebook name=$($item->"alpha")'. So, 'alpha' seems to be some kind of phonebook name. Can you suggest where I can read about this phonebook name and what it means?

rextended · Sat May 27, 2023 6:51 pm

1)
Nothing to determine, is the GSM standard.
Just read the "concrete example code" on previous post "\r\n+CMGL: ", included the space, is what eactly separate fields (go to new line are just for forum formatting).
at the start is :local start -2 because the first record start with "+CMGL: " that have 2 less characters: \r\n
"\r\nOK" is the end of AT output, so must be removed at the end.
You do not have noticed the "OK" at the end of AT+CMGL on your tests?

2)
this field is for the SIM that have in the internal SIM phonebook the number, also return the phonebook name... not all modem support that.

DyadyaGenya · Sat May 27, 2023 10:40 pm

at the start is :local start -2 because the first record start with "+CMGL: " that have 2 less characters: \r\n

I did not have time to check, but does it find the first occurrence in this form?

"\r\nOK" is the end of AT output, so must be removed at the end.
You do not have noticed the "OK" at the end of AT+CMGL on your tests?

If I output an AT command, then of course there is, but here in the script I did not notice it. This "OK" does not seem to be present when displaying the contents of the SMS.

2)
this field is for the SIM that have in the internal SIM phonebook the number, also return the phonebook name... not all modem support that.

That is, if I save a number on the SIM card, for example, John Carter, and receive SMS from this number, will this field be filled in as John Carter?

I have a request. Below I'm showing the code. It seems to be working and generates two arrays. The first array is the one I made yesterday, which contains variations of the CMGL phrase. And the second array is the body of the SMS. At the end, I print both arrays for verification. There are no spaces/gaps between elements of the first array. The next element starts right after the semicolon. But when I print the second array, there is a huge space between elements, and I don't understand where it comes from and if it's normal.

:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search1 "+CMGL:";
:local search2 "\r\n";
:local matchPosition -1;
:local matchCount 0;
:local arrayStat [:toarray ""]
:local smsArray [:toarray ""]

:local continue true;
while ( $continue ) do={
    :set matchPosition [ :find $modemOutput $search1 $matchPosition ];
    :if ( [ :typeof $matchPosition ] != "nil" ) do={
        :set matchCount ( $matchCount + 1 );
        :set $arrayStat ($arrayStat, [:pick $modemOutput ($matchPosition+7) [ :find $modemOutput $search2 $matchPosition ] ] )
        :set matchPosition [ :find $modemOutput $search2 $matchPosition ];
        :set smsArray ( $smsArray , [ :pick $modemOutput ( $matchPosition + 1 ) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] );
    } else={ :set continue false; };
};

:put $arrayStat
:put $smsArray

DyadyaGenya · Sun May 28, 2023 12:07 am

If I do such a check, then these gaps are also clearly visible.

foreach key,sms in=$smsArray do={
    :put "$key=$sms";
}

Even if I replace I will do this replacement:

        :set smsArray ( $smsArray , [ :pick $modemOutput ( $matchPosition+1) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] );
 to
         :set smsArray ( $smsArray , [ :pick $modemOutput $matchPosition ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] );

rextended · Sun May 28, 2023 12:25 am

at the start is :local start -2 because the first record start with "+CMGL: " that have 2 less characters: \r\n
I did not have time to check, but does it find the first occurrence in this form?

I don't understand what you mean, in the first line only "+CMGL: " must be removed, because there is no \r\n at the start, so skip "\r\n+CMGL: " length - 2

You do not have noticed the "OK" at the end of AT+CMGL on your tests?
If I output an AT command, then of course there is, but here in the script I did not notice it. This "OK" does not seem to be present when displaying the contents of the SMS.

Try with
:put ([ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" )
and you see the "OK" at the end.

this field is for the SIM that have in the internal SIM phonebook the number, also return the phonebook name... not all modem support that.
That is, if I save a number on the SIM card, for example, John Carter, and receive SMS from this number, will this field be filled in as John Carter?

It seems to me that I have already written everything clearly.

DyadyaGenya · Sun May 28, 2023 10:05 am

Try with
:put ([ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" )
and you see the "OK" at the end.

This is what I was talking about. In such a situation, I see "OK". But if I do this

:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search "\r\nOK";
:local matchPosition -1;
:local matchCount 0;

:local continue true;
while ( $continue ) do={
    :set matchPosition [ :find $modemOutput $search $matchPosition ];
    :if ( [ :typeof $matchPosition ] != "nil" ) do={
        :set matchCount ( $matchCount + 1 );
        :put ( "Find position #" . [ :tostr $matchCount ] . " at: " . $matchPosition );
    } else={ :set continue false; };
};

The search doesn't find anything.

It seems to me that I have already written everything clearly.

The thing is, I'm not sure if Google translate is correct.

I don't understand what you mean, in the first line only "+CMGL: " must be removed, because there is no \r\n at the start, so skip "\r\n+CMGL: " length - 2

Later I will try to check and formulate my question in a different way to be more confident in the google translation.
Now I can say that I found an error in my version and now there are no extra characters in the array elements. The error is that I forgot that now two characters "\r\n" are participating in the search instead of one "\n", so the offset should be two characters. It turned out like this:

:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search1 "+CMGL:";
:local search2 "\r\n";
:local matchPosition -1;
:local matchCount 0;
:local arrayStat [:toarray ""]
:local smsArray [:toarray ""]

:local continue true;
while ( $continue ) do={
    :set matchPosition [ :find $modemOutput $search1 $matchPosition ];
    :if ( [ :typeof $matchPosition ] != "nil" ) do={
        :set matchCount ( $matchCount + 1 );
        :put ( "Find position #" . [ :tostr $matchCount ] . " at: " . $matchPosition );
        :set $arrayStat ($arrayStat, [:pick $modemOutput ($matchPosition+7) [ :find $modemOutput $search2 $matchPosition ] ] )
        :set matchPosition [ :find $modemOutput $search2 $matchPosition ];
        :set smsArray ( $smsArray , [ :pick $modemOutput ( $matchPosition+2) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] );
    } else={ :set continue false; };
};

:put $arrayStat
:put "-----------";
:put $smsArray

DyadyaGenya · Sun May 28, 2023 1:07 pm

And when I try to find only "OK", it is found only once at the very end

optio · Sun May 28, 2023 9:24 pm

@DyadyaGenya
If you still have troubles with parsing PDU, here is mine script if you find it useful:

sms-read.rsc

and sms2email script which is using it:

sms2email.rsc

sms-read script is mostly created from scripts at viewtopic.php?p=797583#p797583 with some mine modifications and code formatting.

DyadyaGenya · Mon May 29, 2023 2:34 pm

@DyadyaGenya
If you still have troubles with parsing PDU, here is mine script if you find it useful:

Thanks, I'll take a look anyway. But now I have a problem with arrays. I can't figure out how to work with them.

DyadyaGenya · Mon May 29, 2023 2:40 pm

Ok, if you want one hand, since now the goal is sufficently precise:

Of course your code works. But I'm learning and trying to do the same and use my experience. Below is my attempt to do something similar without taking into account phonebook data. How to substitute them I understand. But I can't put the SMS text into the array. Although I can either place it in a separate array or print it.

:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search1 "+CMGL:";
:local search2 "\r\n";
:local matchPosition -1;
:local matchCount 0;
:local arrayStat [:toarray ""]

:local continue true;
while ( $continue ) do={
    :set matchPosition [ :find $modemOutput $search1 $matchPosition ];
    :if ( [ :typeof $matchPosition ] != "nil" ) do={
        :set matchCount ( $matchCount + 1 );

        :local tmpStat [:toarray [:pick $modemOutput ($matchPosition+7) [ :find $modemOutput $search2 $matchPosition ] ] ]
        :local subarray [:toarray ""]
        :set ($subarray->"index" ) [:tonum ($tmpStat->0)]
        :set ($subarray->"status") [:tonum ($tmpStat->1)]
        :set ($subarray->"length") [:tonum ($tmpStat->2)]


        :set matchPosition [ :find $modemOutput $search2 $matchPosition ];
#The line below is just for checking        
    :put [ :pick $modemOutput ( $matchPosition+2) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ];
        :local tmpStat2 [:toarray [ :pick $modemOutput ( $matchPosition+2) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] ];
        :set ($subarray->"sms" ) [:tonum ($tmpStat2->0)]
        :set ($arrayStat->$matchCount) $subarray
    } else={ :set continue false; };
};

:put $arrayStat

rextended · Mon May 29, 2023 3:21 pm

You want to do too many things at once without knowing how to code.
Even I who "get by" don't do it this way (although I don't post the intermediate steps).
The final goal must be divided into subproblems and only when the subproblem is solved, do we move on to the next one.

Problem 1)
Understand what is the final scope: "Have one array with mixed arrays and values that represent all the info possible on SMS inside the SIM"
*** done ***

Problem 2)
How to read the SMS (on unaltered default modem config) and put in one variable?
new problem 2a) after tests, the result can't be directly one array
:put ([/interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output")
new problem 2b) and if I want to read only what I do not have already readed, etc.?
I can add one array that represent all possible parameters:

partial example code

{
:local smsstatus [:toarray "REC UNREAD,REC READ,STO UNSENT,STO SENT,ALL"]
:put ([/interface lte at-chat lte1 wait=yes input="AT+CMGL=$[:find $smsstatus "ALL" -1]" as-value ]->"output")
}

Problem 3)
Is not array, ok, now identifying the data sctructure.
The data sctructure usually is:
+CMGL: 0,1,,48\r\n07[…]72\r\n+CMGL: 1,1,TOBY WORK,40\r\n07[…]41\r\n+CMGL: 2,1,,38\r\n07[…]07\r\nOK
So, if contain at least one record, contain +CMGL
Everytime the data end with \r\nOK
on RouterOS the space is \_
Each SMS start with "+CMGL\_" and end with \r\n, so the separator between the SMS are "\r\n+CMGL\_",
except the first that start with two character less: "+CMGL\_" so the length of "\r\n+CMGL\_" -2
and the last that end with "\r\nOK"
Thw "wile" must be executed until "\r\n+CMGL\_" or "\r\nOK" are present [NOT until I set inside one variable to something...]
With all this info just create a cycle that split raw SMS reply:

partial example code

{
:local smsstatus [:toarray "REC UNREAD,REC READ,STO UNSENT,STO SENT,ALL"]
:local txtSMSpdu ([/interface lte at-chat lte1 wait=yes input="AT+CMGL=$[:find $smsstatus "ALL" -1]" as-value ]->"output")
:local endsrch1 "\r\n+CMGL: " ; :local endsrch2 "\r\nOK"
:local start -2; :local tmppos 0 ; :local index 0

:while (([:typeof [:find $txtSMSpdu $endsrch1 $start]] = "num") or ([:typeof [:find $txtSMSpdu $endsrch2 $start]] = "num")) do={
    :if ([:typeof [:find $txtSMSpdu $endsrch1 $start]] = "num") do={
        :set tmppos [:find $txtSMSpdu $endsrch1 $start]
    } else={
        :set tmppos [:find $txtSMSpdu $endsrch2 $start]
    }

:put [:pick $txtSMSpdu ($start + [:len endsrch1] + 1) $tmppos]

    :set index    ($index + 1)
    :set start    $tmppos
}
}

Problem 4)
split each value (already separated with ,) inside one array of values
*** to be tested later ***: if the phonebook entry already containing "," parser do wrong result
for example splitting "0,1,"CARLO",48\r\n07[…]72" on one array:
{0 ; 1 ; "CARLO" ; 48\r\n07[…]72 }
BUT, if the array do not containing inside the ponebook name, RouterOS ignore the doube ","
{0 ; 1 ; 48\r\n07[…]72 }

partial example code

{
:local smsstatus [:toarray "REC UNREAD,REC READ,STO UNSENT,STO SENT,ALL"]
:local txtSMSpdu ([/interface lte at-chat lte1 wait=yes input="AT+CMGL=$[:find $smsstatus "ALL" -1]" as-value ]->"output")
:local endsrch1 "\r\n+CMGL: " ; :local endsrch2 "\r\nOK"
:local start -2; :local tmppos 0 ; :local index 0
:while (([:typeof [:find $txtSMSpdu $endsrch1 $start]] = "num") or ([:typeof [:find $txtSMSpdu $endsrch2 $start]] = "num")) do={
    :if ([:typeof [:find $txtSMSpdu $endsrch1 $start]] = "num") do={
        :set tmppos [:find $txtSMSpdu $endsrch1 $start]
    } else={
        :set tmppos [:find $txtSMSpdu $endsrch2 $start]
    }
    :local tmparray [:toarray [:pick $txtSMSpdu ($start + [:len endsrch1] + 1) $tmppos]]

:put $tmparray

    :set index    ($index + 1)
    :set start    $tmppos
}

}

Problem 5a)
Manage the different length of the returned array if phonebook entry is not present.
So, if the length of obained array is 3, the name is missing, if is 4 is present.
If length is 3, add a 4th empty value o array.
Create one array that map the values to the right position.
5b) Also the length and PDU part are on the same variable. What divide the length and the PDU is just "\r\n", so just split at it.
5c) Add a print cycle at the end for display the values.
Full code here:
viewtopic.php?p=1004891#p1004540

The previous problems are solved, except test the script on one SIM with one received SMS from one name phonebook containing "," in name.
You are at the
Problem 6)
create one SEPARATE function $PDU2arr to be putted here:
:set ($subarray->"PDU" ) [$PDU2arr [:tostr [:pick ($tmparray->$idx3) ([:find ($tmparray->$idx3) "\r\n" -1] + 2) [:len ($tmparray->$idx3)]]] ]
for add one array of PDU values.

Problem 6) Subproblem A)
Create one function $PDU2arr that split each SMS part and return it as array.
example:
[$PDU2arr "07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72"]
return one array containing

splitted PDU code

{
"smsc"="07912180958739F1";
"type"="04"; [this determines what other fields there will be after, for type="04" they are:]
"sender"="0B917120069876F0";
"extra-data"="00";
"aplhabet-class-and-others"="00";
"date"="9140503223218A";
"timezone"="8A";
"length"="21";
"message"="D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72"
}

Problem 6) Subproblem B)
decode the values for return instead one aray of decoded vallues:

wanted values code

{
"smsc"="+12085978931";
"type"="SMS-DELIVER"; [this determines what other fields there will be after, for type="SMS-DELIVER" they are:]
"sender"="+17026089670";
"class"="standard"; [from aplhabet-class-and-others]
"aplhabet"="GSM-7"; [from aplhabet-class-and-others]
....
.... (some other decoded values from aplhabet-class-and-others and extra-data are possible, but I do not studied the full list)
.... for example "SMS ID" and "SMS 1 of 2" for SMS with lenght > 160 characters.
.....
"date"="19-04-05 23:32:12"
"timezone"=-28"; (is GMT-7, so 28 times -15 minutes)
"length"=21;
"message"="Test message to AT Command Tester"
}

DyadyaGenya · Mon May 29, 2023 7:26 pm

You want to do too many things at once without knowing how to code.

Apparently all the same Google translator translates incorrectly. You misunderstood me. I'm just trying to do it step by step. And I built the steps for myself in my head approximately the same as you described. In my last post, I complained that I was missing one step. It was approximately at your step 3. Now I made it and went through all the steps that you wrote for step 6.
Here is my version of the code, how do I get to step 6

:local smsStatus [:toarray "REC UNREAD,REC READ,STO UNSENT,STO SENT,ALL"]
:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search1 "+CMGL:";
:local search2 "\r\n";
:local matchPosition -1;
:local matchCount 0;
:local arrayStat [:toarray ""]

:local continue true;
while ( $continue ) do={
    :set matchPosition [ :find $modemOutput $search1 $matchPosition ];
    :if ( [ :typeof $matchPosition ] != "nil" ) do={


        :local tmpStat [:toarray [:pick $modemOutput ($matchPosition+7) [ :find $modemOutput $search2 $matchPosition ] ] ]
        :local subarray [:toarray ""]
            :if ( [:len $tmpStat]=3 ) do={
                :set ($subarray->"index" ) [:tonum ($tmpStat->0)]
                :set ($subarray->"status") ($smsStatus->[:tonum ($tmpStat->1)])
                :set ($subarray->"length") [:tonum ($tmpStat->2)]
                :set ($subarray->"alpha") ("")
            } else={
                :set ($subarray->"index" ) [:tonum ($tmpStat->0)]
                :set ($subarray->"status") ($smsStatus->[:tonum ($tmpStat->1)])
                :set ($subarray->"length") [:tonum ($tmpStat->3)]
                :set ($subarray->"alpha") [:tonum ($tmpStat->2)]
            }

        :set matchPosition [ :find $modemOutput $search2 $matchPosition ];
#    :put [ :pick $modemOutput ( $matchPosition+2) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ];
        :set ($subarray->"sms" ) ( [ :pick $modemOutput ( $matchPosition+2) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] )
        :set ($arrayStat->$matchCount) $subarray
        :set matchCount ( $matchCount + 1 );
    } else={ :set continue false; };
};

:put $arrayStat
:put "-----------";

Now I can split the body of the SMS into parts, select ASC, TP-MTI & Co, TP-UDHI and other parts. And already further to do the transcoding of the SMS text.

DyadyaGenya · Mon May 29, 2023 7:29 pm

sms-read script is mostly created from scripts at

Please see my version.

optio · Mon May 29, 2023 8:20 pm

sms-read script is mostly created from scripts at
Please see my version.

You need to decode PDU line from sms key in $arrayStat values map from offsets described at post #23 or see here: https://www.gsmfavorites.com/documents/sms/pdutext/. A lot of work is a front of you if you want to create all from scratch (there is a 1077 lines in sms-read script!). In that script this is handled by several functions, $extracted array contains PDU line for each sms (like from your array under value sms key), examine rest of a code from that point (

:local extracted [$smsReadExtractPDUs]

) and you will get a picture what all you need to implement decoding...

rextended · Mon May 29, 2023 8:23 pm

Step 1: clean frills, fix concept erros, and add something that display the result:

pass 1 clean code

{
:local smsStatus     [:toarray "REC UNREAD,REC READ,STO UNSENT,STO SENT,ALL"]
:local modemOutput   ([/interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value]->"output")
:local search1       "+CMGL:"
:local search2       "\r\n"
:local matchPosition -1
:local matchCount    0
:local arrayStat     [:toarray ""]

while ([:typeof [:find $modemOutput $search1 $matchPosition]] != "nil") do={
    :set matchPosition [:find $modemOutput $search1 $matchPosition]
    :local tmpStat  [:toarray [:pick $modemOutput ($matchPosition + 7) [:find $modemOutput $search2 $matchPosition]]]
    :local subarray [:toarray ""]
    :set ($subarray->"index" ) [:tonum ($tmpStat->0)]
    :set ($subarray->"status") ($smsStatus->[:tonum ($tmpStat->1)])
    :if ([:len $tmpStat] = 3) do={
        :set ($subarray->"alpha" ) ""
        :set ($subarray->"length") [:tonum ($tmpStat->2)]
    } else={
        :set ($subarray->"alpha" ) [:tonum ($tmpStat->2)]
        :set ($subarray->"length") [:tonum ($tmpStat->3)]
    }
    :set matchPosition [:find $modemOutput $search2 $matchPosition]
    :set ($subarray->"sms") ([:pick $modemOutput ($matchPosition + 2) [:find $modemOutput $search2 $matchPosition]])
    :set ($arrayStat->$matchCount) $subarray
    :set matchCount ($matchCount + 1)
}

:foreach item in=$arrayStat do={
    :put "SMS index=$($item->"index"), status=$($item->"status"), phonebook name=$($item->"alpha"), pdu length=$($item->"length")\r\n*** START PDU ***"
    :put ($item->"sms")
    :put "*** END PDU ***\r\n\r\n"
}

}

If I continue to fix your code, at the end is identical to the mine.
Better if you go to Problem 6) Subproblem A), you need to write one separate function.....
At the start try to convert just the string
07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72
ignoring all previous code, is useless at this step....

optio · Mon May 29, 2023 8:55 pm

If I continue to fix your code, at the end is identical to the mine.

And to continue to do that for several hundreds of lines for proper implementation... this will be a long thread

DyadyaGenya · Wed May 31, 2023 12:15 am

Results:
ERROR: missing everytime last PDU hex value...

Very strange. I check the result of your script and mine and they have identical results in my opinion. Could you show what exactly is missing in my version? What symbols?

At the start try to convert just the string
07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72
ignoring all previous code, is useless at this step....

I have not yet begun to decode the body of the SMS itself, that is, such an excerpt as you showed. But I can even translate it a little with my hands. For example, the phone is SCA 12085978931, and the sender is 17026089670, TP-DCS is a 7-bit compressed encoding, the possible date of sending is 2019/04/05 23:32:12 A 8 time zone (probably China). Naturally, I'm not an expert and I'm not very good at decoding and I can make mistakes in this data, but I just started doing it.
I can select all these parts of the SMS using the script, including the SMS text itself, and at this stage I cannot yet decode the SMS text itself. The rest are no problems so far.

rextended · Wed May 31, 2023 12:43 am

Results:
ERROR: missing everytime last PDU hex value...
Very strange. I check the result of your script and mine and they have identical results in my opinion. Could you show what exactly is missing in my version? What symbols?

Example:
using my code
07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72
usin your (unaltered or clean, no difference) code
07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB7
do not notice the mising 2 at the end?

( 8A is GMT -7 (28 times -15) as explained before, is on USA , not China... )

optio · Wed May 31, 2023 1:01 am

Very strange. I check the result of your script and mine and they have identical results in my opinion. Could you show what exactly is missing in my version? What symbols?

Could be a problem at line that matches expression

.*\$matchPosition[[:space:]]][[:space:]]-[[:space:]]1[[:space:]]\)[[:space:]]][[:space:]]\)$

which extracts one character less at the end...

DyadyaGenya · Wed May 31, 2023 1:57 pm

If I do such a check, then these gaps are also clearly visible.

foreach key,sms in=$smsArray do={
    :put "$key=$sms";
}

Even if I replace I will do this replacement:

        :set smsArray ( $smsArray , [ :pick $modemOutput ( $matchPosition+1) ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] );
 to
         :set smsArray ( $smsArray , [ :pick $modemOutput $matchPosition ( [ :find $modemOutput $search2 $matchPosition ] - 1 ) ] );

DyadyaGenya · Wed May 31, 2023 2:08 pm

Example:
using my code
07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72
usin your (unaltered or clean, no difference) code
07912180958739F1040B917120069876F000009140503223218A21D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB7
do not notice the mising 2 at the end?

You are really right. This is my carelessness. I corrected the starting point in one place, because I began to use "\r\n" instead of "\n" and forgot in another. This is how it works right

:global printValue;
:local smsStatus [:toarray "REC UNREAD,REC READ,STO UNSENT,STO SENT,ALL"]
:local modemOutput ( [ /interface lte at-chat lte1 wait=yes input="AT+CMGL=4" as-value ]->"output" );
:local search1 "+CMGL:";
:local search2 "\r\n";
:local matchPosition -1;
:local matchCount 0;
:local arrayStat [:toarray ""]

:local continue true;
while ( $continue ) do={
    :set matchPosition [ :find $modemOutput $search1 $matchPosition ];
    :if ( [ :typeof $matchPosition ] != "nil" ) do={


        :local tmpStat [:toarray [:pick $modemOutput ($matchPosition+7) [ :find $modemOutput $search2 $matchPosition ] ] ]
        :local subarray [:toarray ""]
            :if ( [:len $tmpStat]=3 ) do={
                :set ($subarray->"index" ) [:tonum ($tmpStat->0)]
                :set ($subarray->"status") ($smsStatus->[:tonum ($tmpStat->1)])
                :set ($subarray->"length") [:tonum ($tmpStat->2)]
                :set ($subarray->"alpha") ("")
            } else={
                :set ($subarray->"index" ) [:tonum ($tmpStat->0)]
                :set ($subarray->"status") ($smsStatus->[:tonum ($tmpStat->1)])
                :set ($subarray->"length") [:tonum ($tmpStat->3)]
                :set ($subarray->"alpha") [:tonum ($tmpStat->2)]
            }

        :set matchPosition [ :find $modemOutput $search2 $matchPosition ];
        :set ($subarray->"sms" )  [ :pick $modemOutput ( $matchPosition+2) ( [ :find $modemOutput $search2 $matchPosition ]  ) ]
        :set ($arrayStat->$matchCount) $subarray
        :set matchCount ( $matchCount + 1 );
    } else={ :set continue false; };
};

:put $arrayStat

Unfortunately, I have not yet had time to see what my conceptual errors are, which you noticed in the redone version. I hope to be able to take a look later tonight.

Could be a problem at line that matches expression
Code: Select all
.*\$matchPosition[[:space:]]][[:space:]]-[[:space:]]1[[:space:]]\)[[:space:]]][[:space:]]\)$
which extracts one character less at the end...

I think I went down the same path that you suggested looking for my mistake.

DyadyaGenya · Wed May 31, 2023 3:58 pm

:set ($subarray->"index" ) [:tonum ($tmpStat->0)]
:set ($subarray->"status") ($smsStatus->[:tonum ($tmpStat->1)])
:if ([:len $tmpStat] = 3) do={
:set ($subarray->"alpha" ) ""
:set ($subarray->"length") [:tonum ($tmpStat->2)]
} else={
:set ($subarray->"alpha" ) [:tonum ($tmpStat->2)]
:set ($subarray->"length") [:tonum ($tmpStat->3)]
}

Managed to watch. Here you are absolutely right. So the code will be more correct and shorter. I should have guessed it myself, even though I did a draft.

while ([:typeof [:find $modemOutput $search1 $matchPosition]] != "nil") do={
:set matchPosition [:find $modemOutput $search1 $matchPosition]

I read somewhere that the "find" command is the most expensive and energy intensive. In my version, it is used only once.

rextended · Wed May 31, 2023 4:12 pm

I read somewhere that the "find" command is the most expensive and energy intensive. In my version, it is used only once.

We are talking about at most 25 SMS, not a log of 1000 items or a firewall with 10,000 items...
But is easy solveable...

Fixed the cleaned version with same fix you do at your code
viewtopic.php?p=1004951#p1004951

rextended · Wed May 31, 2023 4:47 pm

I've updated my script by changing the variable names to be more consistent,
I've reformatted the code for clarity,
optimized the internal operations to execute as few instructions as possible
and make everything easier to understand how it works.

convert all sms inside one unique array of arrays of subvalues code

{
:local CMGLstatus [:toarray "REC UNREAD,REC READ,STO UNSENT,STO SENT,ALL"]
:local CMGLoutput ([/interface lte at-chat ([find]->0) input="AT+CMGF=0;+CMGL=$[:find $CMGLstatus "ALL" -1]" wait=yes as-value ]->"output")
# Future check: If CMGLoutput value is "OK" only, there is no results.
:set   CMGLoutput "$[:pick $CMGLoutput 7 ([:len $CMGLoutput] - 2)]+CMGL: "
:local SMSarray   [:toarray ""]
:local separator  "\r\n+CMGL: "
:local lenCMGLsep [:len $separator]
:local findStart  (0 - $lenCMGLsep) ; :local tempPos 0 ; :local smsIndex 0 ; :local alphaIndex 0 ; :local rawIndex 0

:local findPosSeparator [:find $CMGLoutput $separator $findStart]
:while ([:typeof $findPosSeparator] = "num") do={
    :set tempPos $findPosSeparator
    :local tempArray [:toarray [:pick $CMGLoutput ($findStart + $lenCMGLsep) $tempPos]]
    # Future check: if on SIM phonebook the name containing comma, and the Modem reply with the name on SIM phonebook, prevent this problem
    :if ([:len $tempArray] = 4) do={:set alphaIndex 2 ; :set rawIndex 3} else={:set alphaIndex 4 ; :set rawIndex 2  ; :set ($tempArray->4) ""}
    :local subArray [:toarray ""]
    :set ($subArray->"index"  ) [:tonum ($tempArray->0)]
    :set ($subArray->"status" ) ($CMGLstatus->[:tonum ($tempArray->1)])
    :set ($subArray->"alpha"  ) "\"$($tempArray->$alphaIndex)\""
    :set ($subArray->"length" ) [:tonum [:pick ($tempArray->$rawIndex) 0 [:find ($tempArray->$rawIndex) "\r\n" -1]]]
    :set ($subArray->"PDU"    ) [:pick ($tempArray->$rawIndex) ([:find ($tempArray->$rawIndex) "\r\n" -1] + 2) [:len ($tempArray->$rawIndex)]]
    :set ($SMSarray->$smsIndex) $subArray
    :set smsIndex  ($smsIndex + 1)
    :set findStart $tempPos
    :set findPosSeparator [:find $CMGLoutput $separator $findStart]
}

:foreach item in=$SMSarray do={
    :put "SMS index=$($item->"index"), status=$($item->"status"), phonebook name=$($item->"alpha"), pdu length=$($item->"length")"
    :put "*** START PDU ***"
    :put ($item->"PDU")
    :put "*** END PDU ***\r\n"
}

}

DyadyaGenya · Wed May 31, 2023 9:49 pm

You need to decode PDU line from sms key in $arrayStat values map from offsets described at post #23 or see here

Do you have more examples explaining how to decipher some of the meanings? I can't figure out how to deal with the sender number and SMS type (SMS-DELIVER,SMS-STATUS REPORT,SMS-SUBMIT REPORT,RESERVED)
Or, for example, what is the type of SMS if it is encrypted as 44 or 60 or 40?

DyadyaGenya · Wed May 31, 2023 9:55 pm

I've updated my script by changing the variable names to be more consistent,
I've reformatted the code for clarity,
optimized the internal operations to execute as few instructions as possible
and make everything easier to understand how it works.

Your code is really good, I didn't argue with that. But my task was first of all to learn different ways of working with arrays using the example of working with SMS. And I always keep your option in front of my eyes. If I understand correctly, then with the correction of my mistake about cutting off the last SMS character, there are no special comments on my version?
If so, then it remains for me to deal with some of the nuances of recoding. I already know how to make an additional local function that would turn an encrypted phone into a normal one. Still need to figure out how to work with the letter number. And some other things.

DyadyaGenya · Wed May 31, 2023 11:53 pm

As already written on another post, never use ({}) the result is unexpected.
The only way to create empty array is [:toarray ""] and without ( ).

By the way, can you tell me the link where it is written about this?

Amm0 · Thu Jun 01, 2023 12:07 am

As already written on another post, never use ({}) the result is unexpected.
The only way to create empty array is [:toarray ""] and without ( ).

Just try it. It's a syntax error in recent V7 at least. But {} is [:nothing] (or nil), not an array.

By the way, can you tell me the link where it is written about this?

https://wiki.mikrotik.com/wiki/Manual:S ... mpty_array

rextended · Thu Jun 01, 2023 12:31 am

Do you have more examples explaining how to decipher some of the meanings? I can't figure out how to deal with the sender number and SMS type (SMS-DELIVER,SMS-STATUS REPORT,SMS-SUBMIT REPORT,RESERVED)
Or, for example, what is the type of SMS if it is encrypted as 44 or 60 or 40?

I know only what is already wrote on Problem 6) here. Google is your friend...
viewtopic.php?t=196458#p1004891

(44? 60? 40? in those case is better if the number is > of 9 specify if are decimal numbers or hexadecimal, etc. but I do not know what correspondence is...)

DyadyaGenya · Thu Jun 01, 2023 1:24 am

TP-UDL: 21 (message lenght on septept, because is on GSM-7 format. Are octects if the mesage is on UCS-2)
TP-UD: D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72
(message encoded on octect, but internally is on septept for GSM-7, is on octect if is ASCII-8 or UCS-2)

Decode the GSM-7 or eventually the UCS-2 part is easy with my already existent functions.
Function to convert Hex GSM7 to CP1252 string (for decode the PDU message part on GSM7)
viewtopic.php?t=177551#p992792
Function to convert UCS-2 to UTF-8 (for decode the PDU message part on UCS-2)
viewtopic.php?p=983695#p983695

It's not entirely clear how I'm supposed to figure out which encoding to choose for my language. Here, it seems that the message is written in English, but it can only be translated by your UCS2toUTF8 function into some other numbers and percent signs, and the second function says that there is an error. Although the text in the SMS is written in English.
Here is the SMS text: "C8329BFD065DDF72363904"
Before that, it says that some kind of 7-bit encoding is used, but I don’t understand what that means.
SMS from your example also does not display in the terminal in normal English or something else. Only numbers and percent signs

rextended · Thu Jun 01, 2023 2:42 am

Before it can be used on my program for encode/decode GSM-7 the data must be converted from 7 bit PDU to 8 bit hex....

RouterOS can display only 7 bit ascii standard characters, not GSM-7 UCS-2 or UTF-8
The %encode is for send the message using fetch to twitter, whatsapp, etc.

optio · Thu Jun 01, 2023 3:01 am

RouterOS can display only 7 bit ascii standard characters, not GSM-7 UCS-2 or UTF-8

Even from utf8 terminal over ssh? I've seen characters correctly displayed from scripts which are not in 7 bit ascii when connecting over ssh, still cli input is ascii always...

rextended · Thu Jun 01, 2023 3:04 am

viewtopic.php?t=177551#p993440

C8329BFD065DDF72363904 => 48656C6C6F20576F726C6421 => Hello World!

:put [$pdutogsm7 ("\C8\34\88\FE\06\05\D9\EC\50\28\04")]
:put [$HexGSM7toCP1252 ("48656C6C6F20576F726C6421")]

rextended · Thu Jun 01, 2023 3:08 am

RouterOS can display only 7 bit ascii standard characters, not GSM-7 UCS-2 or UTF-8
Even from utf8 terminal over ssh? I've seen characters correctly displayed from scripts which are not in 7 bit ascii when connecting over ssh, still cli input is ascii always...

I'm not talking about third-party programs.
It's that program that shows them to you, not RouterOS...

rextended · Thu Jun 01, 2023 3:10 am

In the end I made a lot of functions to manage SMS... More than meets the eye...

optio · Thu Jun 01, 2023 3:10 am

Even from utf8 terminal over ssh? I've seen characters correctly displayed from scripts which are not in 7 bit ascii when connecting over ssh, still cli input is ascii always...
I'm not talking about third-party programs.
It's that program that shows them to you, not RouterOS...

Ok, just saying, if you use remote terminal for cli connection (I'm always using cli like that) you can get proper output from script if terminal encoding is setup correctly.

rextended · Thu Jun 01, 2023 3:12 am

I know, thanks

, but it depends too much on the program used, and in general when you want to send an SMS via email or telegram, etc., don't go through SSH...

optio · Thu Jun 01, 2023 3:25 am

I know, thanks , but it depends too much on the program used, and in general when you want to send an SMS via email or telegram, etc., don't go through SSH...

With mine script when UCS-2 is converted to UTF8, that string variable is directly passed to /tool/e-mail/send and in received email body UTF8 characters are displayed correctly, doesn't matter if script is triggered over terminal, winbox or scheduler. For telegram I guess is needed to be url escaped, did not send over it to know, but I guess if is http api...

optio · Thu Jun 01, 2023 3:32 am

... <when using reply instead edit> ...

rextended · Thu Jun 01, 2023 3:32 am

Yep, it was basically born for telegram and anything else fetchable,
which needs to be encoded in the GET part of the URL.
But of course it can also be sent by mail as a data string, etc.

DyadyaGenya · Thu Jun 01, 2023 11:45 am

Just try it. It's a syntax error in recent V7 at least. But {} is [:nothing] (or nil), not an array.

That is, it only concerns the creation of an empty array. If the array is created not empty, then it can be written in this way?

rextended · Thu Jun 01, 2023 12:37 pm

Just try it. It's a syntax error in recent V7 at least. But {} is [:nothing] (or nil), not an array.
That is, it only concerns the creation of an empty array. If the array is created not empty, then it can be written in this way?

Please don't try to interpret what I write like someone do with the bible and the like.
Who mentioned non-empty arrays?

I already show you this:
viewtopic.php?p=973659#p973295

Amm0 · Thu Jun 01, 2023 5:16 pm

If the array is created not empty, then it can be written in this way?

Sure, if there are values inside in {} – that's allowed. It's just the specific case of "{}" is NOT an empty array.

DyadyaGenya · Sat Jun 03, 2023 7:18 pm

Maybe also checking UDH header for joining splitted messages will be nice

Can you provide an example of multiple SMS messages that consist of several parts?

optio · Sat Jun 03, 2023 8:51 pm

Can you provide an example of multiple SMS messages that consist of several parts?

You can create examples from here: https://www.smsdeliverer.com/online-sms ... coder.aspx, there is also PDU decoder: https://www.smsdeliverer.com/online-sms ... coder.aspx. These are for sms-submit PDUs, but decoding TP-UHDI and UHD part from TP-UD is the same as for deliver.

DyadyaGenya · Wed Jun 07, 2023 1:02 am

viewtopic.php?t=177551#p993440

C8329BFD065DDF72363904 => 48656C6C6F20576F726C6421 => Hello World!

:put [$pdutogsm7 ("\C8\34\88\FE\06\05\D9\EC\50\28\04")]
:put [$HexGSM7toCP1252 ("48656C6C6F20576F726C6421")]

Unfortunately I don't understand your example. I see that it works for you, but I can't figure out how to use it myself. If possible, explain from the beginning. Here I received a PDU fragment "C8329BFD065DDF72363904". I write in the PDU that 7-bit encryption is used. What should I do next?

DyadyaGenya · Sat Jun 10, 2023 9:10 pm

@DyadyaGenya
If you still have troubles with parsing PDU, here is mine script if you find it useful:

I can already parse SMS. I have two problems left: re-encode and glue if the SMS consists of several parts. These parts I can draw out separately. And when I print the array, they are clearly visible and the unique SMS number and serial number. But still unable to connect. In any case, the first priority is to recode. And for starters, I would like to understand the algorithm, if it is encoded using GSM 7, that is, the DCS field is equal to "00"

For example excerpt: "C8329BFD065DDF72363904" or "D4F29C0E6A97E7F3F0B90CA2BF41412A68F86EB7C36E32885A9ED3CB72" or "050003590303DCE532B94C06CDDF6F37". In the last example of a composite SMS, where 05 - UDH LENGTH, 00 - IEI, 03 - IEDL, 59 - IED1 (unique number), 03 - IED2 (number of parts), 03 - IED3 (serial number of the SMS part)

I understand that you can use functions from @rextended, but I do not understand how and in what order.

optio · Sat Jun 10, 2023 11:17 pm

59 - IED1 (unique number), 03 - IED2 (number of parts), 03 - IED3 (serial number of the SMS part

IED3 is actually part number, join messages with same IED1 ordered by IED3.

DyadyaGenya · Sat Jun 10, 2023 11:58 pm

IED3 is actually part number, join messages with same IED1 ordered by IED3.

I understand it. I can't write a function for this yet. Or insert the procedure into an existing function. And I have not yet wondered how to make such a function. For me, it's in the background. This was said only to demonstrate that the rest of the tasks have already been dealt with. Only these two remained. Although it still needs to be printed beautifully. But that's later.

But I'm more concerned about the decoding issue.
I am trying to do something like this:

:put [$pdutogsm7 ("C8329BFD065DDF72363904") ]

or

:put [$HexGSM7toCP1252 ([$pdutogsm7 ("C8329BFD065DDF72363904") ] )]

But I'm getting bullshit.

optio · Sun Jun 11, 2023 12:04 am

But I'm more concerned about the decoding issue.
I am trying to do something like this:
Code: Select all
:put [$pdutogsm7 ("C8329BFD065DDF72363904") ]
or
Code: Select all
:put [$HexGSM7toCP1252 ([$pdutogsm7 ("C8329BFD065DDF72363904") ] )]
But I'm getting bullshit.

how about

:put [$HexGSM7toCP1252 "C8329BFD065DDF72363904"]

?

DyadyaGenya · Sun Jun 11, 2023 12:57 pm

how about
Code: Select all
:put [$HexGSM7toCP1252 "C8329BFD065DDF72363904"] 
?

> :put [$HexGSM7toCP1252 "C8329BFD065DDF72363904"] 
Invalid 7-bit value (200)

Looks like I need to start a new thread.

optio · Sun Jun 11, 2023 2:34 pm

Code: Select all
> :put [$HexGSM7toCP1252 "C8329BFD065DDF72363904"] 
Invalid 7-bit value (200)
Looks like I need to start a new thread.

You did not implement something right, I tried with function

smsReadConvert7bitToUtf8

in my script which converts GSM-7 to UTF8:

> :put [$smsReadConvert7bitToUtf8 instring="C8329BFD065DDF72363904"];
Hello World!

rextended · Sun Jun 11, 2023 5:55 pm

I already wrote it. it would be enough for him to read and understand what is written there...

:put [$pdutogsm7 ("\C8\34\88\FE\06\05\D9\EC\50\28\04")]

DyadyaGenya · Sun Jun 25, 2023 12:28 am

Earlier, I mentioned that there shouldn't be any major issues with concatenating SMS messages. However, I was mistaken. I can't figure out how to handle the situation when not all parts of a composite SMS are received. Here's an example of a simplified array that I receive when processing SMS messages, along with the code that should process this array and create a new array with concatenated composite SMS messages. I have come up with a solution for handling the situation when SMS parts are duplicated due to delivery delays. However, I lack the imagination to come up with a way to concatenate the parts if not all of them are received, so that instead of the missing part, there would be a phrase like "This part of the SMS is still pending.

:local arrayStat ( {
    {
        "01_index"=1;
        "16_phoneSender"="+380000000000";
        "19_SCTStime"="23-06-09   00:23:10   UTC = 3:0";
        "20_TP-UDL"=2;
        "21_UDH LENGTH"="0";
        "22_UDH"=[];
        "23_IEI"=[];
        "24_IEDL"=[];
        "25_IED1"=[];
        "26_IED2"=[];
        "27_IED3"=[];
        "28_TEXT"="SMS01/"
    };
    {
        "01_index"=2;
        "16_phoneSender"="+380000000000";
        "19_SCTStime"="23-06-09   14:45:41   UTC = 3:0";
        "20_TP-UDL"=160;
        "21_UDH LENGTH"=5;
        "22_UDH"="0003590301";
        "23_IEI"="00";
        "24_IEDL"="03";
        "25_IED1"="59";
        "26_IED2"="03";
        "27_IED3"="01";
        "28_TEXT"="SMS02/"
    };
    {
        "01_index"=3;
        "16_phoneSender"="+380000000000";
        "19_SCTStime"="23-06-09   14:45:43   UTC = 3:0";
        "20_TP-UDL"=160;
        "21_UDH LENGTH"=5;
        "22_UDH"="0003590302";
        "23_IEI"="00";
        "24_IEDL"="03";
        "25_IED1"="59";
        "26_IED2"="03";
        "27_IED3"="02";
        "28_TEXT"="SMS03/"
    };
    {
        "01_index"=4;
        "16_phoneSender"="+380000000000";
        "19_SCTStime"="23-06-09   14:45:44   UTC = 3:0";
        "20_TP-UDL"=18;
        "21_UDH LENGTH"=5;
        "22_UDH"="0003590303";
        "23_IEI"="00";
        "24_IEDL"="03";
        "25_IED1"="59";
        "26_IED2"="03";
        "27_IED3"="03";
        "28_TEXT"="SMS04/"
    };
    {
        "01_index"=7;
        "16_phoneSender"="+380000000000";
        "19_SCTStime"="23-06-12   19:22:55   UTC = 3:0";
        "20_TP-UDL"=34;
        "21_UDH LENGTH"="0";
        "22_UDH"=[];
        "23_IEI"=[];
        "24_IEDL"=[];
        "25_IED1"=[];
        "26_IED2"=[];
        "27_IED3"=[];
        "28_TEXT"="SMS05/"
    };
    {
        "01_index"=8;
        "16_phoneSender"="+380000000000";
        "19_SCTStime"="23-06-12   19:31:09   UTC = 3:0";
        "20_TP-UDL"=40;
        "21_UDH LENGTH"="0";
        "22_UDH"=[];
        "23_IEI"=[];
        "24_IEDL"=[];
        "25_IED1"=[];
        "26_IED2"=[];
        "27_IED3"=[];
        "28_TEXT"="SMS06/"
    }
} );

:local parsedArray [ :toarray "" ];

:local tempReadedSplitedSmsID [ :toarray "" ];

:local smsCounter 0;
:local successCounter 0;
:while ( ( [ :typeof ( $arrayStat->$smsCounter ) ] = "array" ) and ( [ :len ( $arrayStat->$smsCounter ) ] > 5 ) ) do={
	:if ( [ :typeof ( $arrayStat->$smsCounter->"25_IED1" ) ] = "nil" ) do={
		# full sms in one
		:local splitedSms ( {
			"16_phoneSender"=( $arrayStat->$smsCounter->"16_phoneSender" );
			"19_SCTStime"=( $arrayStat->$smsCounter->"19_SCTStime" );
			"TEXT"=( $arrayStat->$smsCounter->"28_TEXT" )
		} );
		:set ( $parsedArray->$successCounter ) $splitedSms;
		:set successCounter ( $successCounter + 1 );
	} else={
		# splitted sms ( 25_IED1 - id of full, 26_IED2 - parts count, 27_IED3 - current part )
		:local smsID ( $arrayStat->$smsCounter->"25_IED1" );
		# check if this ID already readed
		:local readed false; 
		:foreach readedID in=$tempReadedSplitedSmsID do={ :if ( $smsID = $readedID ) do={ :set readed true; }; };
		:if ( !$readed ) do={
			:local partsCountFull [ :tonum ( $arrayStat->$smsCounter->"26_IED2" ) ];
			:local partsCountReal 0;
			:foreach partOfSms in=$arrayStat do={
				:if ( ( $partOfSms->"25_IED1" ) = $smsID ) do={ :set partsCountReal ( $partsCountReal + 1 ); };
			};
			:if ( $partsCountFull = $partsCountReal ) do={
				# if received all parts of full sms
				:local splitedSms ( {
					"16_phoneSender"=( $arrayStat->$smsCounter->"16_phoneSender" );
					"19_SCTStime"="";
					"TEXT"=""
				} );
				# read every part from 1
				:for partcounter from=1 to=$partsCountFull do={
					:foreach partOfSms in=$arrayStat do={
						:if ( ( ( $partOfSms->"25_IED1" ) = $smsID ) and ( [ :tonum ( $partOfSms->"27_IED3" ) ] = $partcounter ) ) do={
							:set ( $splitedSms->"19_SCTStime" ) ( $partOfSms->"19_SCTStime" );
							:set ( $splitedSms->"TEXT" ) ( ( $splitedSms->"TEXT" ) . ( $arrayStat->$partcounter->"28_TEXT" ) );
						};
					};
				};
				:set ( $parsedArray->$successCounter ) $splitedSms;
				:set successCounter ( $successCounter + 1 );
			} else={ 
				:if ( $partsCountFull < $partsCountReal ) do={
				:put "Have duble"
				# read every part from 1
				:for partcounter from=1 to=$partsCountFull do={
				:local uniqueElements [ :toarray "" ]
					:foreach partOfSms in=$arrayStat do={
						:if ( ( ( $partOfSms->"25_IED1" ) = $smsID ) and ( [ :tonum ( $partOfSms->"27_IED3" ) ] = $partcounter ) and ([:len ($uniqueElements->( [ :tonum ( $partOfSms->"27_IED3" ) ] ) )] = 0) ) do={
							:set ( $splitedSms->"19_SCTStime" ) ( $partOfSms->"19_SCTStime" );
							:set ( $splitedSms->"TEXT" ) ( ( $splitedSms->"TEXT" ) . ( $arrayStat->$partcounter->"28_TEXT" ) );
							:set ($uniqueElements->( [ :tonum ( $partOfSms->"27_IED3" ) ] ) ) 1
						};
					};
				};
				:set ( $parsedArray->$successCounter ) $splitedSms;
				:set successCounter ( $successCounter + 1 );
    } else={
        :put "Need wait"
        }
};
			:set tempReadedSplitedSmsID ( $tempReadedSplitedSmsID , $smsID );
		};
	};
	:set smsCounter ( $smsCounter + 1 );
};

I have marked the place where, in theory, this should be implemented.

    } else={
        :put "Need wait"
        }

Those who wish can experiment, for example, by duplicating the SMS with index 4 or by removing it.

examples code

concrete example code

partial example code

partial example code

partial example code

splitted PDU code

wanted values code

pass 1 clean code

convert all sms inside one unique array of arrays of subvalues code

Who is online