[feature request] Add support for UCS-2 encoded SMS

[dscp46]

Hi,

When UCS-2 encoded SMS are received, the text variable is empty, and replaced by ucs-2 in the json payload.

Here is a sample message extracted with the uqmi -s -d $device --get-message $messageID --storage $storage command (sensitive data redacted):

{"smsc":"+336xxxxxxxx","sender":"Free Mobile","timestamp":"2024-02-20 20:58:04","concat_ref":67,"concat_part":4,"concat_parts":4,"ucs-2":"00200070006100730073006500200064006500200076006f0074007200650020004500730070006100630065002000410062006f006e006e00e900200065007400200063006f006e00740061006300740065007a002d006e006f0075007300200061007500200033003200340034002e"}

As a custom fix, I've added the following lines between fi and echo "$sender" > $receiveFolder/$sms_file in /usr/bin/uqmi_d.sh:

if [ ! -n "$text" ]
then
	json_get_var ucs_2 ucs-2
	text=$(echo $ucs_2 | sed -r 's/00([0-9a-f][0-9a-f])/\1/g' | xxd -r -p)
fi 

It's a quick and ugly fix as it depends on xxd, which is not part of your dependencies, and the charset conversion is not done in a proper way.

Kind regards,

[mrhaav]

Hi
Do you know how to send an SMS coded with ucs-2?

[dscp46]

Sorry, I wasn't clear enough with my initial idea:
The feature request focuses on appropriately decoding SMSes when their payload is UCS-2 encoded.
Sending is less of an issue, unless your consider international usage.

Hi Do you know how to send an SMS coded with ucs-2?

[rwalli]

Maybe it is better to save the raw messages with "/usr/bin/uqmi_d.sh".
So you could decide what to do with ucs-2 content in e.g. "/usr/bin/uqmi_sms.sh" and you don't lose all messages.

@dscp46 solution will help languages with fewer special characters, but how to deal with Greek or Chinese?

[cyrozap]

If you skip the sed command, it should be possible to use iconv -f UCS-2BE to convert from big-endian UCS-2 to UTF-8 and preserve special characters. For example:

if [ ! -n "$text" ]
then
	json_get_var ucs_2 ucs-2
	text=$(echo $ucs_2 | xxd -r -p | iconv -f UCS-2BE -t UTF-8)
fi