I'm getting some strange behavior when reading a .txt file with [text define], where the output from [text get] isn't routeable like a normal list. See the attached file for an example. Do other users experience the same thing?
[text fromsymbol] reveals that there are some extended ASCII characters junking around in front of the problematic term. Is PD accidentally reading some meta-data from the text file?
Currently, it only seems to be happening on the first line of the file. I think I remember having the same problems throughout a textfile, although I can't replicate this now.
Any suggestions?
Hi Liam
On Die, 2017-02-07 at 07:34 +0000, Liam Goodacre wrote:
I'm getting some strange behavior when reading a .txt file with [text define], where the output from [text get] isn't routeable like a normal list. See the attached file for an example. Do other users experience the same thing?
[text fromsymbol] reveals that there are some extended ASCII characters junking around in front of the problematic term. Is PD accidentally reading some meta-data from the text file?
Currently, it only seems to be happening on the first line of the file. I think I remember having the same problems throughout a textfile, although I can't replicate this now.
Any suggestions?
It seems that Pd reads the BOM (Byte Order Mark) of the Unicode text file as part of the content instead of skipping it.
However, you don't necessarily need your text file to have a BOM. In your example there aren't even any non-ASCII characters. You can remove the BOM by doing:
tail --bytes=+4 strange.txt > strange_new.txt
Roman
Your text file has been UTF8 encoded, and starts with a "byte order mark" (BOM) {0xEF 0xBB 0xBF}, which then is rejected by [route].
see https://en.wikipedia.org/wiki/Byte_order_mark#UTF-8 :
The UTF-8 https://en.wikipedia.org/wiki/UTF-8 representation of the BOM
is the byte sequence 0xEF,0xBB,0xBF. A text editor or web browser misinterpreting the text as ISO-8859-1 https://en.wikipedia.org/wiki/ISO-8859-1 or CP1252 https://en.wikipedia.org/wiki/CP1252 will display the characters  for this.
Antoine Rousseau http://www.metalu.net http://metalu.net __ http://www.metaluachahuter.com/ http://www.metaluachahuter.com/compagnies/al1-ant1/
2017-02-07 8:34 GMT+01:00 Liam Goodacre liamg_uw@hotmail.com:
I'm getting some strange behavior when reading a .txt file with [text define], where the output from [text get] isn't routeable like a normal list. See the attached file for an example. Do other users experience the same thing?
[text fromsymbol] reveals that there are some extended ASCII characters junking around in front of the problematic term. Is PD accidentally reading some meta-data from the text file?
Currently, it only seems to be happening on the first line of the file. I think I remember having the same problems throughout a textfile, although I can't replicate this now.
Any suggestions?
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/ listinfo/pd-list
Thanks Roman and Antoine--looks like you both go the answer at the same time!
This explanation makes sense, however I am using non ASCII characters in the textfile (not the one attached, but the one I'm working on), so I guess that I need the BOM to stay there. I've rigged up a system which will split the dead characters off beginning of the first term, so it's working for now.
Would this be considered normal behavior for PD, or should I file a bug report?
From: Antoine Rousseau antoine@metalu.net Sent: 07 February 2017 08:11 To: Liam Goodacre Cc: PD list Subject: Re: [PD] un-routable output from [text get]
Your text file has been UTF8 encoded, and starts with a "byte order mark" (BOM) {0xEF 0xBB 0xBF}, which then is rejected by [route].
see https://en.wikipedia.org/wiki/Byte_order_mark#UTF-8 : Byte order mark - Wikipediahttps://en.wikipedia.org/wiki/Byte_order_mark#UTF-8 en.wikipedia.org The byte order mark (BOM) is a Unicode character, U+FEFF BYTE ORDER MARK (BOM), whose appearance as a magic number at the start of a text stream can signal several ...
The UTF-8https://en.wikipedia.org/wiki/UTF-8 representation of the BOM is the byte sequence 0xEF,0xBB,0xBF. A text editor or web browser misinterpreting the text as ISO-8859-1https://en.wikipedia.org/wiki/ISO-8859-1 or CP1252https://en.wikipedia.org/wiki/CP1252 will display the characters  for this.
Antoine Rousseau http://www.metalu.nethttp://metalu.net __ http://www.metaluachahuter.com/http://www.metaluachahuter.com/compagnies/al1-ant1/
2017-02-07 8:34 GMT+01:00 Liam Goodacre <liamg_uw@hotmail.commailto:liamg_uw@hotmail.com>:
I'm getting some strange behavior when reading a .txt file with [text define], where the output from [text get] isn't routeable like a normal list. See the attached file for an example. Do other users experience the same thing?
[text fromsymbol] reveals that there are some extended ASCII characters junking around in front of the problematic term. Is PD accidentally reading some meta-data from the text file?
Currently, it only seems to be happening on the first line of the file. I think I remember having the same problems throughout a textfile, although I can't replicate this now.
Any suggestions?
Pd-list@lists.iem.atmailto:Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Hi Liam,
I suggest that you use:
https://notepad-plus-plus.org/
as a very good replacement to “ms-windows notepad”.
You can encode as “utf-8” or “utf-8-BOM”,
And much more.
Salutti, Lucarda.
Mensaje telepatico asistido por maquinas.
From: Pd-list pd-list-bounces@lists.iem.at on behalf of Liam Goodacre liamg_uw@hotmail.com Sent: Tuesday, February 7, 2017 8:24 AM To: PD list Subject: Re: [PD] un-routable output from [text get]
Thanks Roman and Antoine--looks like you both go the answer at the same time!
This explanation makes sense, however I am using non ASCII characters in the textfile (not the one attached, but the one I'm working on), so I guess that I need the BOM to stay there. I've rigged up a system which will split the dead characters off beginning of the first term, so it's working for now.
Would this be considered normal behavior for PD, or should I file a bug report?
From: Antoine Rousseau antoine@metalu.net Sent: 07 February 2017 08:11 To: Liam Goodacre Cc: PD list Subject: Re: [PD] un-routable output from [text get]
Your text file has been UTF8 encoded, and starts with a "byte order mark" (BOM) {0xEF 0xBB 0xBF}, which then is rejected by [route].
see https://en.wikipedia.org/wiki/Byte_order_mark#UTF-8 : Byte order mark - Wikipediahttps://en.wikipedia.org/wiki/Byte_order_mark#UTF-8 en.wikipedia.org The byte order mark (BOM) is a Unicode character, U+FEFF BYTE ORDER MARK (BOM), whose appearance as a magic number at the start of a text stream can signal several ...
The UTF-8https://en.wikipedia.org/wiki/UTF-8 representation of the BOM is the byte sequence 0xEF,0xBB,0xBF. A text editor or web browser misinterpreting the text as ISO-8859-1https://en.wikipedia.org/wiki/ISO-8859-1 or CP1252https://en.wikipedia.org/wiki/CP1252 will display the characters  for this.
Antoine Rousseau http://www.metalu.nethttp://metalu.net __ http://www.metaluachahuter.com/http://www.metaluachahuter.com/compagnies/al1-ant1/
2017-02-07 8:34 GMT+01:00 Liam Goodacre <liamg_uw@hotmail.commailto:liamg_uw@hotmail.com>:
I'm getting some strange behavior when reading a .txt file with [text define], where the output from [text get] isn't routeable like a normal list. See the attached file for an example. Do other users experience the same thing?
[text fromsymbol] reveals that there are some extended ASCII characters junking around in front of the problematic term. Is PD accidentally reading some meta-data from the text file?
Currently, it only seems to be happening on the first line of the file. I think I remember having the same problems throughout a textfile, although I can't replicate this now.
Any suggestions?
Pd-list@lists.iem.atmailto:Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Btw,
Your example becomes “routable” if converted from “utf-8-BOM” to “utf-8”
Mensaje telepatico asistido por maquinas.
From: Pd-list pd-list-bounces@lists.iem.at on behalf of Lucas Cordiviola lucarda27@hotmail.com Sent: Tuesday, February 7, 2017 8:34 AM To: Liam Goodacre; PD list Subject: Re: [PD] un-routable output from [text get]
Hi Liam,
I suggest that you use:
https://notepad-plus-plus.org/
as a very good replacement to “ms-windows notepad”.
You can encode as “utf-8” or “utf-8-BOM”,
And much more.
Salutti, Lucarda.
Mensaje telepatico asistido por maquinas.
From: Pd-list pd-list-bounces@lists.iem.at on behalf of Liam Goodacre liamg_uw@hotmail.com Sent: Tuesday, February 7, 2017 8:24 AM To: PD list Subject: Re: [PD] un-routable output from [text get]
Thanks Roman and Antoine--looks like you both go the answer at the same time!
This explanation makes sense, however I am using non ASCII characters in the textfile (not the one attached, but the one I'm working on), so I guess that I need the BOM to stay there. I've rigged up a system which will split the dead characters off beginning of the first term, so it's working for now.
Would this be considered normal behavior for PD, or should I file a bug report?
On 2017-02-07 09:24, Liam Goodacre wrote:
This explanation makes sense, however I am using non ASCII characters in the textfile (not the one attached, but the one I'm working on), so I guess that I need the BOM to stay there.
no. the BOM was just another useless invention. UTF-8 (unlike UTF-16) is a byte-stream oriented protocol. it is unconcerned by the notion of byte-order.
gfmasdr IOhannes
-
Antoine Rousseau -
IOhannes m zmoelnig -
Liam Goodacre -
Lucas Cordiviola -
Roman Haefeli