How to convert a HEX value to a char?

jeroenp · Fri May 06, 2016 9:18 am

I'm trying to fill an associative array with mappings from characters to their HEX encoding.

The below fails: it doesn't like the `("\0".$hexChar)` construct.

So: how do I convert a string containing the characters "\", "0" and "1" (where "0" and "1" are dynamically generated) so it parses as "\01"?

:global convert ({});
#:set ($convert->"\00") "\\00"
:set ($convert->"\7F") "\\7F"
:set ($convert->"\81") "\\81"
:local hexChars "01234567890ABCDEF"
:local groupChars "01890ABCDEF"
  :for hexCharsIndex from=0 to=([:len $hexChars] - 1) do={ 
    :local hexChar [:pick $hexChars $hexCharsIndex]
    :set ($convert->("\0".$hexChar)) ("\\0".$hexChar)
  }

--jeroen

Fri May 06, 2016 2:04 pm

Doing a single slash means the next character or two are expected to form a valid escape sequence. "\0" on its own is not a valid escape sequence, unless you have a literal second hex character there, e.g. "\0F".

To put this another way, on a parser level,

"\0".$hexChar

is three main tokens: a string, a concatenation operator, and a variable access operator. Within a string, each character other than a slash is valid, but a slash must either be followed by one of the second characters from the constant escape sequences, or the next two characters must form a valid hex number.

jeroenp · Fri May 06, 2016 4:49 pm

...To put this another way, on a parser level,
Code: Select all
"\0".$hexChar
is three main tokens: a string, a concatenation operator, and a variable access operator.

That I already figured. Hence the above question: How can I do this despite the parser?
Is there a kind of `eval` or `parse` function that I could use? And if so: where should I start?

--jeroen

Fri May 06, 2016 8:34 pm

Yes. There's the :parse command. It takes a string, and returns a callable command. If you store it in a variable, then referencing the variable will trigger the command.
(that one's been in use from before :local and :global could declare functions with the "do" argument)

There's also :execute, which is similar, but instead of returning the command, is executes it right away. It's "the" equivalent of eval().

jeroenp · Fri May 06, 2016 10:04 pm

Thanks. Any good examples to get started (before I find myself in quoting hell).

jeroenp · Sat May 07, 2016 7:46 am

Thanks. Any good examples to get started (before I find myself in quoting hell).

Google Search isn't exactly helpful, as these hardly return anything:

And I know from experience in other languages that these kind of constructs are really tricky.

Another question: how bad is the performance of `:parse` and `:execute` compared to normal script execution?

It's about replacing ~150 lines of initialisation code with a double loop.

--jeroen

Sat May 07, 2016 5:04 pm

Obviously, it's slower than executing the actual code directly, but AFAIK, it's not THAT much slower (I haven't done any measurements, but there's certainly not a noticeable difference). The penalty adds up if you call :parse/:execute multiple times, but if you compose everything, before eventually calling it once, there shouldn't be much different, compared to executing the generation, and then executing "real code equivalent".

The larger parts of your performance concerns should be at the actual generation, as there's no "append" operation for strings. Every concatenation produces a new value, and then you overwrite the old value, and you do that each time you want to append to a string, which is not exactly very efficient, and adds up on large loops. I don't think 150 overwrites add THAT much time, but still, it would be noticeably slower if you were comparing 1000 times here with 1000 time in any language supporting operators like "+=" or ".=".

jeroenp · Sun May 08, 2016 8:48 pm

In that case I will leave it as is, unless you have comments on the current script at https://gist.github.com/jpluimers/f667a ... f1eeda2c2f

rextended · Sun Aug 08, 2021 1:48 am

search tag # rextended hex2chr hexadecimal to char

Mmm... but simply:

[rextended@MATRIX] > :global hex2chr do={:return [[:parse "(\"\\$1\")"]]}
[rextended@MATRIX] > :put [$hex2chr 64]
d
[rextended@MATRIX] >

just do not pass lowercase or invalid hex digit, only uppercase.

see here for the function to convert a string of HEX value to a string of Chars, it also check lower/upper/invalid hex:
viewtopic.php?f=9&t=129693&p=871742#p871742

Sertik · Thu Jan 26, 2023 12:04 pm

You can still do this. Get any character by code (in $key):

:local char [[:parse "(\"\\$[:pick "0123456789ABCDEF" (($key >> 4) & 0xF)]$[:pick "0123456789ABCDEF" ($key & 0xF)]\")"]]

rextended · Thu Jan 26, 2023 2:32 pm

You can still do this. Get any character by code (in $key):
Code: Select all
:local char [[:parse "(\"\\$[:pick "0123456789ABCDEF" (($key >> 4) & 0xF)]$[:pick "0123456789ABCDEF" ($key & 0xF)]\")"]]

Something do not work on your script, this topic "How to convert a HEX value to a char?" is fo convert hexadecimal values to characters.
On this test I use equal sign "=" that is 0x3D on hex, my function works, your not.
Probably your line is for convert a 0-255 number to a 8-bit character.

{
:global hex2chr do={:return [[:parse "(\"\\$1\")"]]}
:put [$hex2chr "3D"]


:local key "3D"
:local char [[:parse "(\"\\$[:pick "0123456789ABCDEF" (($key >> 4) & 0xF)]$[:pick "0123456789ABCDEF" ($key & 0xF)]\")"]]
:put $char
}

Can be used for create a function:

numer to character code

:global dec2chr do={
    :local input [:tonum $1]
    :local hexchars "0123456789ABCDEF"
    :local convert [:pick $hexchars (($input >> 4) & 0xF)]
    :set convert ($convert.[:pick $hexchars ($input & 0xF)])
    :return [[:parse "(\"\\$convert\")"]]
}

:put [$dec2chr 61]

Sertik · Mon Jan 30, 2023 10:34 am

:global fsAlphabitCP1251 do={
:if ([:typeof $0]="lookup") do={ 
:if ([:typeof $1]!="nil") do={:return [[:parse "(\"\\$[:pick "0123456789ABCDEF" (([:tonum $1] >> 4) & 0xF)]$[:pick "0123456789ABCDEF" ([:tonum $1] & 0xF)]\")"]]}
:local string
:for key from=0 to=255 do={
    :local char [[:parse "(\"\\$[:pick "0123456789ABCDEF" (($key >> 4) & 0xF)]$[:pick "0123456789ABCDEF" ($key & 0xF)]\")"]]
    :set string ("$string"."$char")
}
:return $string
  }
 }

:put [$fsAlphabitCP1251 "\2B"]

:put [$fsAlphabitCP1251]

Sertik · Mon Jan 30, 2023 11:27 am

The function is written clearly for analysis and study:

:global dec2chr do={
    :local input [:tonum $1]
    :local hexchars "0123456789ABCDEF"
    :local convert [:pick $hexchars (($input >> 4) & 0xF)]
    :set convert ($convert.[:pick $hexchars ($input & 0xF)])
    :return [[:parse "(\"\\$convert\")"]]
}

:put [$dec2chr 61]

Same function. Nothing is clear, but short and works

:

:global dec2chrNew do={
:return [[:parse "(\"\\$[:pick "0123456789ABCDEF" (([:tonum $1] >> 4) & 0xF)]$[:pick "0123456789ABCDEF" ([:tonum $1] & 0xF)]\")"]]
}

:put [$dec2chrNew 61]

rextended · Mon Jan 30, 2023 12:57 pm

I prefer write code than after some month I keep understand what is wrote without go mad,
and I avoid rewriting the same strings and redoing the same operations several times ("0123...", [:tonum $1])

Code: Select all
:global fsAlphabitCP1251 do={
[...]
 }
:put [$fsAlphabitCP1251 "\2B"]

:put [$fsAlphabitCP1251]

What is this for? Everytime return NULL (0x00)

Sertik · Thu Feb 02, 2023 12:28 pm

I apologize for not answering for a long time. The function is used to get a character by code, and if $1 is not set, then it returns a character string of all codes of page.

:global fsAlphabitCP1251 do={
:if ([:typeof $0]="lookup") do={ 
:if ([:typeof $1]!="nothing") do={:return [[:parse "(\"\\$[:pick "0123456789ABCDEF" (([:tonum $1] >> 4) & 0xF)]$[:pick "0123456789ABCDEF" ([:tonum $1] & 0xF)]\")"]]}
:local string
:for key from=0 to=255 do={
    :local char [[:parse "(\"\\$[:pick "0123456789ABCDEF" (($key >> 4) & 0xF)]$[:pick "0123456789ABCDEF" ($key & 0xF)]\")"]]
    :set string ("$string"."$char")
}
:return $string
  }
 }

:log error [$fsAlphabitCP1251 43]

:log warning [$fsAlphabitCP1251]

rextended · Thu Feb 02, 2023 12:37 pm

The function is used to get a character by code, and if $1 is not set, then it returns a character string of all codes of page.

Sorry, but can not be called with CP 1251 name, because do not do any conversion, and for different users provide for different characters same code...

How to convert a HEX value to a char? [SOLVED]

How to convert a HEX value to a char?

Re: How to convert a HEX value to a char?

Re: How to convert a HEX value to a char?

Re: How to convert a HEX value to a char?

Re: How to convert a HEX value to a char?

Re: How to convert a HEX value to a char?

Re: How to convert a HEX value to a char?

Re: How to convert a HEX value to a char?

Re: How to convert a HEX value to a char? [SOLVED]

Re: How to convert a HEX value to a char?

Re: How to convert a HEX value to a char?

numer to character code

Re: How to convert a HEX value to a char?

Re: How to convert a HEX value to a char?

Re: How to convert a HEX value to a char?

Re: How to convert a HEX value to a char?

Re: How to convert a HEX value to a char?