hello
please, can somebody explain why does pure data round big numbers stored in tables, when I save my patch?
- the same for messageboxes, but for tables I find it even stranger.
what's the point of such feature that only waste you time, when you can't switch it off when you need to, so you need to spend hours finding workarounds (that still make your patches slower. (last time it made my patch load for 20s (instead of 2s) because of workarounds in abstractions that were used >333 times....)
am I the only one who finds the unneccesary rounding of numbers annoying? when we want our numbers rounded, we can use some object to round them. why is pure data rounding stuff automatically? i just don't get it. is it a bug? or is it supposed to be like this?
are developers reading this mailing list too, or is it better to post to their mailing list also?
regards, daniel
that's not what i mean.
i mean, that when i write 12345678 (must be 100% accurate within 32bit float with 24bit mantissa) into an array and read it from there, it's still 12345678. but when i save that patch, close it and reload it, and i read from the array, i get 12345700.
nothing to do with limitations of floating point number.
On Sun, Apr 8, 2012 at 4:23 AM, i go bananas hard.off@gmail.com wrote:
It's because Pd saves the value by printing it as text into the patch file using a reduced precision format specifier (%g instead of %f, or %0.6f) so that the numbers look good on screen, with no extra zeros for example. I don't like it either.
Martin
On 2012-04-07 22:40, Angakok Thoth wrote:
that's not what i mean.
i mean, that when i write 12345678 (must be 100% accurate within 32bit float with 24bit mantissa) into an array and read it from there, it's still 12345678. but when i save that patch, close it and reload it, and i read from the array, i get 12345700.
nothing to do with limitations of floating point number.
On Sun, Apr 8, 2012 at 4:23 AM, i go bananas <hard.off@gmail.com mailto:hard.off@gmail.com> wrote:
http://en.wikipedia.org/wiki/Floating_point
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
If this happens, then it should be really given some thought.
<thinking out loud>
I guess one of the downsides of graphical dataflow is that we see what we get (kinda like WYSIWIG editors), therefore there should be a way to always get dirty and non rounded numbers on screen and onto loaded patches. Otherwise we are missing something essential. I see multiple ways pd could do it:
- saving to file is always preserving precision and no rounding; GUI by
default shows rounded versions for space sake or to avoid messy patches; tough the user has the possibility to choose a overall setting or per GUI object that enables him to see the full thing.
or conversely
- saving to file is always preserving precision and no rounding; GUI by
default shows the real stored number; tough the user has the possibility to choose a overall setting or per GUI object that enables him to see only see rounded version on the object.
</thinking out loud>
On Sun, Apr 8, 2012 at 4:58 AM, Martin Peach martin.peach@sympatico.cawrote:
It's because Pd saves the value by printing it as text into the patch file using a reduced precision format specifier (%g instead of %f, or %0.6f) so that the numbers look good on screen, with no extra zeros for example. I don't like it either.
Martin
On 2012-04-07 22:40, Angakok Thoth wrote:
that's not what i mean.
i mean, that when i write 12345678 (must be 100% accurate within 32bit float with 24bit mantissa) into an array and read it from there, it's still 12345678. but when i save that patch, close it and reload it, and i read from the array, i get 12345700.
nothing to do with limitations of floating point number.
On Sun, Apr 8, 2012 at 4:23 AM, i go bananas <hard.off@gmail.com mailto:hard.off@gmail.com> wrote:
http://en.wikipedia.org/wiki/**Floating_pointhttp://en.wikipedia.org/wiki/Floating_point
______________________________**_________________ Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/** listinfo/pd-list http://lists.puredata.info/listinfo/pd-list
______________________________**_________________ Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/** listinfo/pd-list http://lists.puredata.info/listinfo/pd-list
On 04/08/2012 04:58 AM, Martin Peach wrote:
It's because Pd saves the value by printing it as text into the patch file using a reduced precision format specifier (%g instead of %f, or %0.6f) so that the numbers look good on screen, with no extra zeros for example. I don't like it either.
I wonder how can anyone possibly like that
On Sun, Apr 8, 2012 at 1:33 PM, Matteo Sisti Sette matteosistisette@gmail.com wrote:
On 04/08/2012 04:58 AM, Martin Peach wrote:
It's because Pd saves the value by printing it as text into the patch file using a reduced precision format specifier (%g instead of %f, or %0.6f) so that the numbers look good on screen, with no extra zeros for example. I don't like it either.
I wonder how can anyone possibly like that
I've once compiled (vanilla) Pd with the format specifiers changed to print up to 8 significant digits, and soon found why it is normally done with 6 digits max. You get things like this:
33 * 0.3 = 9.900001
That is because binary numbers round differently than decimal numbers. The default format with maximum 6 significant digits avoids some of the confusion, but some (rare) inconsistencies still happen with 6 significant digits, like this one:
7.1 - 7 = 0.0999999
In MaxMsp this is handled like so: if the numbox is too small to show the 0.0999999, it shows 0.1 followed by however many trailing zero's fit in the numbox. Pd instead, shows 0.09 if the numbox width is 4. Also in MaxMsp you can create for example [* 1073741824.] (2^30, a number which can be expressed with exactness in single precision float format). If you try to create [* 1073741825.], it automatically creates [* 1073741824.], because 1073741825. can't be expressed in single precision.
Would be super to have such careful handling and storage of floats in Pd. MaxMsp demonstrates that it is somehow possible.
Katja
On 04/08/2012 04:27 PM, katja wrote:
I've once compiled (vanilla) Pd with the format specifiers changed to print up to 8 significant digits, and soon found why it is normally done with 6 digits max. You get things like this:
33 * 0.3 = 9.900001
That is completely unrelated. That is an issue intrinsic in floating point numbers.
Reducing the precision of numbers when writing them to files (or when parsing messages or whatever) is an issue in Pd.
One thing is rounding numbers for _displaying_ them, another thing is rounding them when _storing_ them (whether in a file or wherever).
Consider this: I create a patch like this:
[9.900001( | [== 9.9] | [print]
This prints 0, as expected.
Then I save and close it. I load it, and it has been transformed into this:
[9.9( | [== 9.9] | [print]
which obviously prints 1.
This is WRONG, there's no reason why it could be good or even acceptable. I do understand why it happens.
By the way, if you write into an object box:
[f 9.90001]
it is immediately changed into: [f 9.9]
Which is also WRONG (why shouldn't I be able to create an object with a parameter value which _can_ be represented without loss of information?), but it is much "less wrong" than the previous case in that this is CONSISTENT. In this case, at least, "what you have is what you save" (note that "what you see" is not an issue).
The case of message boxes (not to mention arrays) is devastating, because you may not realise you're going to loose information until you save the patch, close and reopen it.
Whops, I should have read the other replies first :$
On 04/09/2012 01:23 AM, Matteo Sisti Sette wrote:
On 04/08/2012 04:27 PM, katja wrote:
I've once compiled (vanilla) Pd with the format specifiers changed to print up to 8 significant digits, and soon found why it is normally done with 6 digits max. You get things like this:
33 * 0.3 = 9.900001
That is completely unrelated. That is an issue intrinsic in floating point numbers.
Reducing the precision of numbers when writing them to files (or when parsing messages or whatever) is an issue in Pd.
One thing is rounding numbers for _displaying_ them, another thing is rounding them when _storing_ them (whether in a file or wherever).
Consider this: I create a patch like this:
[9.900001( | [== 9.9] | [print]
This prints 0, as expected.
Then I save and close it. I load it, and it has been transformed into this:
[9.9( | [== 9.9] | [print]
which obviously prints 1.
This is WRONG, there's no reason why it could be good or even acceptable. I do understand why it happens.
By the way, if you write into an object box:
[f 9.90001]
it is immediately changed into: [f 9.9]
Which is also WRONG (why shouldn't I be able to create an object with a parameter value which _can_ be represented without loss of information?), but it is much "less wrong" than the previous case in that this is CONSISTENT. In this case, at least, "what you have is what you save" (note that "what you see" is not an issue).
The case of message boxes (not to mention arrays) is devastating, because you may not realise you're going to loose information until you save the patch, close and reopen it.
On Mon, Apr 9, 2012 at 1:23 AM, Matteo Sisti Sette matteosistisette@gmail.com wrote:
On 04/08/2012 04:27 PM, katja wrote:
I've once compiled (vanilla) Pd with the format specifiers changed to print up to 8 significant digits, and soon found why it is normally done with 6 digits max. You get things like this:
33 * 0.3 = 9.900001
That is completely unrelated. That is an issue intrinsic in floating point numbers.
Reducing the precision of numbers when writing them to files (or when parsing messages or whatever) is an issue in Pd.
One thing is rounding numbers for _displaying_ them, another thing is rounding them when _storing_ them (whether in a file or wherever).
You're right. In Pd however, the format specifier %g is used for display (except for message boxes), and the same reformatted value is eventually saved to disk as text. Therefore, changing the format specifiers affect both representation and storage. The code is just simplistic on this aspect.
Doing it better would require a lot of modifications, more than changing some format specifiers. It's a pity we can't see MaxMsp's code, the issues seem to be neatly solved there, like:
- You can type, use and store very large floats if they are
representable, and if not representable, the nearest representable value is automatically displayed and stored.
- On the other hand, you can for example create [* 0.01] without it
being changed into [* 0.0999999] which would be the nearest representable value.
- The representation in a numbox reckons with the size of the numbox,
for example the result of 7.1 - 7 becomes 0.100 if the numbox can only hold 5 characters (while Pd gives 0.099 in that case).
It seems to me that they have had a thorough discussion on the matter, decided about what would be most useful in practice, and implemented it no matter how many conditional checks this would bring into the code.
Katja
On 04/09/12 12:39, katja wrote:
Doing it better would require a lot of modifications, more than changing some format specifiers. It's a pity we can't see MaxMsp's code, the issues seem to be neatly solved there, like:
i haven't looked at the actual behaviour, but max has a (default) binary forma, where it can store numbers without precision loss.
this is of course way easier to implement than trying to save numbers at a "reasonable2 precision in a textfile.
fgmadr IOhannes
2012/4/9 IOhannes m zmölnig zmoelnig@iem.at:
On 04/09/12 12:39, katja wrote:
Doing it better would require a lot of modifications, more than changing some format specifiers. It's a pity we can't see MaxMsp's code, the issues seem to be neatly solved there, like:
i haven't looked at the actual behaviour, but max has a (default) binary forma, where it can store numbers without precision loss.
this is of course way easier to implement than trying to save numbers at a "reasonable2 precision in a textfile.
At least they did implement detailed rules for representation of numbers as text in boxes. A set of number/text conversion rules for storage may not in itself be more complicated. But you'd need two conversion methods, one for representation and one for storage, both fairly detailed.
I would expect that Mathieu Bouchard could have proposed improvements in this field. I had a quick look into desiredata, but can't find files m_atom.c, g_numbox.c etc. Seems it all went into a big blob desiredata.c.
Katja
On 2012-04-09 07:31, IOhannes m zmölnig wrote:
On 04/09/12 12:39, katja wrote:
Doing it better would require a lot of modifications, more than changing some format specifiers. It's a pity we can't see MaxMsp's code, the issues seem to be neatly solved there, like:
i haven't looked at the actual behaviour, but max has a (default) binary forma, where it can store numbers without precision loss.
this is of course way easier to implement than trying to save numbers at a "reasonable2 precision in a textfile.
Here is part of a Max5 patch, a multiply object:
"box" : { "maxclass" : "newobj", "text" : "* -57.29578", "numoutlets" : 1, "fontname" : "Arial", "outlettype" : [ "float" ], "id" : "obj-19", "fontsize" : 12.0, "patching_rect" : [ 294.0, 382.0, 73.0, 20.0 ], "numinlets" : 2 }
It is stored as a text file in a format not unlike xml. It may be that floats are stored as doubles inside Max5 but the patcher is saved as text.
Martin
On Apr 9, 2012, at 11:34 AM, Martin Peach wrote:
On 2012-04-09 07:31, IOhannes m zmölnig wrote:
On 04/09/12 12:39, katja wrote:
Doing it better would require a lot of modifications, more than changing some format specifiers. It's a pity we can't see MaxMsp's code, the issues seem to be neatly solved there, like:
i haven't looked at the actual behaviour, but max has a (default) binary forma, where it can store numbers without precision loss.
this is of course way easier to implement than trying to save numbers at a "reasonable2 precision in a textfile.
Here is part of a Max5 patch, a multiply object:
"box" : { "maxclass" : "newobj", "text" : "* -57.29578", "numoutlets" : 1, "fontname" : "Arial", "outlettype" : [ "float" ], "id" : "obj-19", "fontsize" : 12.0, "patching_rect" : [ 294.0, 382.0, 73.0, 20.0 ], "numinlets" : 2 }
It is stored as a text file in a format not unlike xml. It may be that floats are stored as doubles inside Max5 but the patcher is saved as text.
With Max5 they switched to a JSON file format. Previously, there was the .mxb format, which is binary.
We could still store numbers as ASCII and not lose precision. For example, we could store the actual bits as base64 or hex. Let's say it'll store 64-bits to have one number format for both single and double precision. Using base64, there would have to be 32 characters to support 64 bits.
While we are at it, it would be good to make the comments totally unparsed, so that things like +14322342342 (phone numbers) or 1. (numbered lists) don't get converted by Pd's type system. Comments do not need to be parsed into types, they should just be treated as UTF-8 text.
.hc
http://at.or.at/hans/
On Mon, Apr 9, 2012 at 6:14 PM, Hans-Christoph Steiner hans@at.or.at wrote:
We could still store numbers as ASCII and not lose precision. For example, we could store the actual bits as base64 or hex. Let's say it'll store 64-bits to have one number format for both single and double precision. Using base64, there would have to be 32 characters to support 64 bits.
While we are at it, it would be good to make the comments totally unparsed, so that things like +14322342342 (phone numbers) or 1. (numbered lists) don't get converted by Pd's type system. Comments do not need to be parsed into types, they should just be treated as UTF-8 text.
For a storage format to accommodate single and double precision numbers, I would suggest %.14lg. This is a generous format - even if it truncates some bits from double precision it is sufficient for all practical purposes in Pd. This is already tested in a way. I used the format in Pd-double, and patches saved from Pd-double load without problem in regular Pd. A non-decimal ASCII format instead, would make existing patches unreadable.
Not sure though how to separate code for representation and storage. In the IEM gui's there is a separate line doing the conversion for representation, while storage is achieved via binbuf_addv(), which in the end calls atom_text() which does the conversion. But for object boxes and Pd's built-in numbox, I could not identify separate conversion routines for representation and storage. Changing the format in atom_text() did it all, if I remember well. With format %.14lg, that would make a problem for object boxes in the single precision case.
Katja
On 04/09/12 20:06, katja wrote:
On Mon, Apr 9, 2012 at 6:14 PM, Hans-Christoph Steinerhans@at.or.at wrote:
We could still store numbers as ASCII and not lose precision. For example, we could store the actual bits as base64 or hex. Let's say it'll store 64-bits to have one number format for both single and double precision. Using base64, there would have to be 32 characters to support 64 bits.
yes; but a binary format is much easier to implement. it takes exactly 8 bytes to represent any double precision number. always.
While we are at it, it would be good to make the comments totally unparsed, so that things like +14322342342 (phone numbers) or 1. (numbered lists) don't get converted by Pd's type system. Comments do not need to be parsed into types, they should just be treated as UTF-8 text.
oh yes.
For a storage format to accommodate single and double precision numbers, I would suggest %.14lg. This is a generous format - even if it truncates some bits from double precision it is sufficient for all practical purposes in Pd. This is already tested in a way. I used the format in Pd-double, and patches saved from Pd-double load without problem in regular Pd.
Pd's text2atom converter is rather generous and accepts loads of things, including higher precision numbers and symbols with spaces. it is the atom2text converter that makes all the trouble.
A non-decimal ASCII format instead, would make existing patches unreadable.
right; i think this is also the reason why Pd doesn't do any binary storage/transmission: it makes debugging so easy if you can actually understand what is going on without an assembler book besides the keyboard.
Not sure though how to separate code for representation and storage. In the IEM gui's there is a separate line doing the conversion for representation, while storage is achieved via binbuf_addv(), which in the end calls atom_text() which does the conversion. But for object boxes and Pd's built-in numbox, I could not identify separate conversion routines for representation and storage. Changing the format in atom_text() did it all, if I remember well. With format %.14lg, that would make a problem for object boxes in the single precision case.
i think a good start would be to let pd-gui decide how to properly display a number.
the major drawback i see right now is, that raising the number of bytes used to transmit a single number will make table updates even more costy than they already are.
so i still think that a way to transmit/save/load numbers in binary form would be the easiest way; something like "d:0x400921fb54442d18" (at a pre-defined endianness for all platforms) rather than "3.1415926".
mfgasrd IOhannes
2012/4/9 IOhannes m zmölnig zmoelnig@iem.at:
A non-decimal ASCII format instead, would make existing patches unreadable.
right; i think this is also the reason why Pd doesn't do any binary storage/transmission: it makes debugging so easy if you can actually understand what is going on without an assembler book besides the keyboard.
I mean to say that switching to any format other than decimal ASCII would make it impossible for Pd to interpret patch files using the current format.
the major drawback i see right now is, that raising the number of bytes used to transmit a single number will make table updates even more costy than they already are.
Large tables are mostly stored in an audio format, rather than text. But indeed, saving and loading large tables as text is already time-consuming with the current format. Text is also used by the [pd] object to transmit numbers / signals. And how about sending over UDP or TCP?
Katja
hi IOhannes, Katja,
On 04/10/2012 10:33 AM, katja wrote:
2012/4/9 IOhannes m zmölnigzmoelnig@iem.at:
A non-decimal ASCII format instead, would make existing patches unreadable.
...
for the record: Max abandoned binary patch storage format quite some time ago. They tend to use JSON now for pretty much everything, and it works well.
Declarative format is more flexible and easier to extend than procedural, explicitly stack-based representation. Moreover, with JSON come tools and standards. One example: ECMA Script 5 defines very precisely, how to stringify numbers...
Krzysztof
On Tue, Apr 10, 2012 at 12:50 PM, Krzysztof Czaja czaja@chopin.edu.pl wrote:
for the record: Max abandoned binary patch storage format quite some time ago. They tend to use JSON now for pretty much everything, and it works well.
Declarative format is more flexible and easier to extend than procedural, explicitly stack-based representation. Moreover, with JSON come tools and standards. One example: ECMA Script 5 defines very precisely, how to stringify numbers...
Good info. The ECMA standard describes the number type as a double, and a number value as chosen from the set of representable numbers in this type, with a rounding to nearest method as in IEEE 754. See section 8.5 of:
http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf
Then in section 9 the conversion rules are presented in greatest detail, with 'number to string' in section 9.3.1. Krzysztof, do you think that MaxMsp uses the same rules for printing numbers in boxes? If so, it could be used as a guideline for Pd as well.
Katja
On 04/10/2012 02:20 PM, katja wrote: ...
http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf
Then in section 9 the conversion rules are presented in greatest detail, with 'number to string' in section 9.3.1. Krzysztof, do you think that MaxMsp uses the same rules for printing numbers in boxes? If so, it could be used as a guideline for Pd as well.
not really. The old, six decimals after point limit still remains all over Max6. But if you force something more sane into Pd, they'll likely follow...
Krzysztof
Hi all -
this section 9.3.1 describes how to convert strings to numbers - but isn't the real problem how Pd converts numbers to strings?
I think the ideal solution when the number of characters isn't an issue is to specify that whatever prints out should be a string that, when scanned using scanf("%f", ...) (or scanf("%lf", ...)) returned the exact 32-bit number. I'm not 100% sure this is possible, since there might be 32 bit values that scanf can't ever return, but I bet there's a way to get really really close :)
Miller
On Wed, Apr 11, 2012 at 09:35:14PM +0200, Krzysztof Czaja wrote:
On 04/10/2012 02:20 PM, katja wrote: ...
http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf
Then in section 9 the conversion rules are presented in greatest detail, with 'number to string' in section 9.3.1. Krzysztof, do you think that MaxMsp uses the same rules for printing numbers in boxes? If so, it could be used as a guideline for Pd as well.
not really. The old, six decimals after point limit still remains all over Max6. But if you force something more sane into Pd, they'll likely follow...
Krzysztof
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Hello,
I did another array-update performance test, following Jonathan's earlier suggestion. A 800 pt array (this is also the visual size) is updated every 50 milliseconds. This is very cpu-intensive anyhow. For Pd-double ~4% more than for Pd-extended 0.43.1. Pd-double translates a number to maximally 20 characters (like in -0.00029718922544271) and Pd-extended to a maximum of 12 characters (like in -0.000958107). The small performance difference indicates that the conversions are relatively fast, compared to other aspects of the array update. Indeed, Tcl code is responsible for most of the cpu load in array updates. I've checked it with Shark.app performance profiler.
It seems to me that an ASCII decimal representation with more decimal digits, like the %.14lg used in Pd-double, does not seriously hurt (the performance of) storage and transmission to Pd-gui. When printing numbers in boxes on screen however, simply using more than six significant decimals would not give the desired effect for single precision numbers.
Since printing numbers in boxes happens relatively seldom, a more detailed routine could be spent on that, to display the desired precision. In the sense of Miller's suggestion, convert a number as typed in a box into a number representable by Pd's float type. Allow numbers to be printed with full precision up to a reasonable number of significant decimal digits For example, %lg14 no matter what float type, would be sufficient for all practical purposes. However, numbers should not be printed with more characters than originally typed in the box. If you'd type 4294967297 it would be converted to 4294967296 (in single precision case). But if you typed 0.01, Pd should not print 0.0099999999977648.
Katja
On Wed, Apr 11, 2012 at 10:01 PM, Miller Puckette msp@ucsd.edu wrote:
Hi all -
this section 9.3.1 describes how to convert strings to numbers - but isn't the real problem how Pd converts numbers to strings?
I think the ideal solution when the number of characters isn't an issue is to specify that whatever prints out should be a string that, when scanned using scanf("%f", ...) (or scanf("%lf", ...)) returned the exact 32-bit number. I'm not 100% sure this is possible, since there might be 32 bit values that scanf can't ever return, but I bet there's a way to get really really close :)
Miller
On Wed, Apr 11, 2012 at 09:35:14PM +0200, Krzysztof Czaja wrote:
On 04/10/2012 02:20 PM, katja wrote: ...
http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf
Then in section 9 the conversion rules are presented in greatest detail, with 'number to string' in section 9.3.1. Krzysztof, do you think that MaxMsp uses the same rules for printing numbers in boxes? If so, it could be used as a guideline for Pd as well.
not really. The old, six decimals after point limit still remains all over Max6. But if you force something more sane into Pd, they'll likely follow...
Krzysztof
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On 04/10/12 10:33, katja wrote:
I mean to say that switching to any format other than decimal ASCII would make it impossible for Pd to interpret patch files using the current format.
why?
Large tables are mostly stored in an audio format, rather than text.
i was talking about pd/pd-gui communication (and keep the number format for both saving and pd/gui communication the same). when displaying/updating a table every single number is converted to text using printf, send over the wire and then converted back to a number for drawing the table. it makes a difference if you have to transmit 44100 *4 bytes or 44100*12 bytes.
fgmasdr IOhannes
2012/4/10 IOhannes m zmölnig zmoelnig@iem.at:
On 04/10/12 10:33, katja wrote:
i was talking about pd/pd-gui communication (and keep the number format for both saving and pd/gui communication the same). when displaying/updating a table every single number is converted to text using printf, send over the wire and then converted back to a number for drawing the table. it makes a difference if you have to transmit 44100 *4 bytes or 44100*12 bytes.
Ah I see. It is not uncommon to display complete audio files, much more than 44100 samples. So all these samples are converted to text and back to numbers, as they go over the network? (While in the end, only a couple hundred values are displayed). And every character goes through the loop in binbuf_text() with all it's cases... well that is a bottleneck which should not be further aggravated. At least this performance issue can be quickly tested, using Pd vs Pd-double. I'll make a test patch for that.
Katja
----- Original Message -----
From: katja katjavetter@gmail.com To: pd-list@iem.at Cc: Sent: Tuesday, April 10, 2012 9:45 AM Subject: Re: [PD] why does PD round numbers? (in tables, in messageboxes, etc)
2012/4/10 IOhannes m zmölnig zmoelnig@iem.at:
On 04/10/12 10:33, katja wrote:
i was talking about pd/pd-gui communication (and keep the number format for both saving and pd/gui communication the same). when displaying/updating a table every single number is converted to text using printf, send over the wire and then converted back to a number for drawing the table. it makes a difference if you have to transmit 44100 *4 bytes or 44100*12 bytes.
Ah I see. It is not uncommon to display complete audio files, much more than 44100 samples.
I've never seen a patch that _displays_ all the data for an array that large. To
transmit 44,100 drawing instructions you need a _canvas_ size of "44100" and
if anyone has actually needed to do that in a patch I'd really like to see it.
So all these samples are converted to text and back to numbers, as they go over the network?
No. See plot_vis inside g_template.c. There is not a 1-to-1 correspondence between # of array elements and rectangles/polygon-coords on the tk canvas (at least for garrays, not sure about data structures). If you create a 44100 element array and make the "size" field in that arrays canvas dialog "2", Pd will only send data to the gui for those 2 pixels, not for the entire array. However, it will loop through the entire array on the c side _every_ time plot_vis is called, in order to figure out what info should be sent to the gui.
For example: running with -d 3, create an array with 10,000,000 elements. Now make the "size" field in its canvas dialog "2". Select the array. Click an array key to move it. Notice the lag, but also notice Pd is only sending two commands to the gui to draw the elements. It's because Pd must loop through 5,000,000 elements before it hits the next pixel where it needs to send another drawing instruction to the gui!
(While in the end, only a couple hundred values are displayed). And every character goes through the loop in binbuf_text() with all it's cases... well that is a bottleneck which should not be further aggravated. At least this performance issue can be quickly tested, using Pd vs Pd-double. I'll make a test patch for that.
Keep in mind that you're using a horribly implemented feature of Pd to do your test-- that is, if you're using garrays. For example, moving an array shouldn't
send _any_ element data to the gui. It doesn't in Pd-l2ork because it just moves
everything by tag, using one line of tk, thus there is no bottleneck in that case.
A practical test I can think of to compare 4byte vs 12byte payload is
something like [metro 100]--[tabwrite~] animation for a visible garray. I'd be
curious to know if there is a significant performance difference there.
-Jonathan
Katja
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Thanks for your tips, Jonathan. Anyway, while you were writing the mail I was already doing a test patch where update times of a 32 million samples array and table are tested, see attached.
I ran the test with Pd-extended 0.43.1 and Pd-double on OSX. For an array with that length, update takes ~3 seconds. For a table displayed graphically, ~3.1 seconds. Surprisingly, drawing as points (instead of polygon) takes almost 5 seconds. The deviation between repeated measurements, a few dozen milliseconds, was about as large as the difference between Pd-extended and Pd-double. I've verified that double precision numbers are indeed displayed with a maximum of 14 significant digits in Pd-double.
I also checked the time for writing 32 million samples to a table without graphical display. This took ~170 milliseconds for Pd-extended and ~ 220 milliseconds for Pd-double.
Later I'll do another test where transmission over network is more specifically tested.
Katja
On Tue, Apr 10, 2012 at 7:05 PM, Jonathan Wilkes jancsika@yahoo.com wrote:
----- Original Message -----
From: katja katjavetter@gmail.com To: pd-list@iem.at Cc: Sent: Tuesday, April 10, 2012 9:45 AM Subject: Re: [PD] why does PD round numbers? (in tables, in messageboxes, etc)
2012/4/10 IOhannes m zmölnig zmoelnig@iem.at:
On 04/10/12 10:33, katja wrote:
i was talking about pd/pd-gui communication (and keep the number format for both saving and pd/gui communication the same). when displaying/updating a table every single number is converted to text using printf, send over the wire and then converted back to a number for drawing the table. it makes a difference if you have to transmit 44100 *4 bytes or 44100*12 bytes.
Ah I see. It is not uncommon to display complete audio files, much more than 44100 samples.
I've never seen a patch that _displays_ all the data for an array that large. To
transmit 44,100 drawing instructions you need a _canvas_ size of "44100" and
if anyone has actually needed to do that in a patch I'd really like to see it.
So all these samples are converted to text and back to numbers, as they go over the network?
No. See plot_vis inside g_template.c. There is not a 1-to-1 correspondence between # of array elements and rectangles/polygon-coords on the tk canvas (at least for garrays, not sure about data structures). If you create a 44100 element array and make the "size" field in that arrays canvas dialog "2", Pd will only send data to the gui for those 2 pixels, not for the entire array. However, it will loop through the entire array on the c side _every_ time plot_vis is called, in order to figure out what info should be sent to the gui.
For example: running with -d 3, create an array with 10,000,000 elements. Now make the "size" field in its canvas dialog "2". Select the array. Click an array key to move it. Notice the lag, but also notice Pd is only sending two commands to the gui to draw the elements. It's because Pd must loop through 5,000,000 elements before it hits the next pixel where it needs to send another drawing instruction to the gui!
(While in the end, only a couple hundred values are displayed). And every character goes through the loop in binbuf_text() with all it's cases... well that is a bottleneck which should not be further aggravated. At least this performance issue can be quickly tested, using Pd vs Pd-double. I'll make a test patch for that.
Keep in mind that you're using a horribly implemented feature of Pd to do your test-- that is, if you're using garrays. For example, moving an array shouldn't
send _any_ element data to the gui. It doesn't in Pd-l2ork because it just moves
everything by tag, using one line of tk, thus there is no bottleneck in that case.
A practical test I can think of to compare 4byte vs 12byte payload is
something like [metro 100]--[tabwrite~] animation for a visible garray. I'd be
curious to know if there is a significant performance difference there.
-Jonathan
Katja
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Hi all -
It's a wierd thing abut TK that drawing polygons is far more efficient than drawing arrays of points; "polygons" are primitive objects that are apparently optimized internally to TK whereas arrays of points have to be drawn one by one (TK thinks they're each a separate object).
cheers Miller
On Tue, Apr 10, 2012 at 07:24:31PM +0200, katja wrote:
Thanks for your tips, Jonathan. Anyway, while you were writing the mail I was already doing a test patch where update times of a 32 million samples array and table are tested, see attached.
I ran the test with Pd-extended 0.43.1 and Pd-double on OSX. For an array with that length, update takes ~3 seconds. For a table displayed graphically, ~3.1 seconds. Surprisingly, drawing as points (instead of polygon) takes almost 5 seconds. The deviation between repeated measurements, a few dozen milliseconds, was about as large as the difference between Pd-extended and Pd-double. I've verified that double precision numbers are indeed displayed with a maximum of 14 significant digits in Pd-double.
I also checked the time for writing 32 million samples to a table without graphical display. This took ~170 milliseconds for Pd-extended and ~ 220 milliseconds for Pd-double.
Later I'll do another test where transmission over network is more specifically tested.
Katja
On Tue, Apr 10, 2012 at 7:05 PM, Jonathan Wilkes jancsika@yahoo.com wrote:
----- Original Message -----
From: katja katjavetter@gmail.com To: pd-list@iem.at Cc: Sent: Tuesday, April 10, 2012 9:45 AM Subject: Re: [PD] why does PD round numbers? (in tables, in messageboxes, etc)
2012/4/10 IOhannes m zmölnig zmoelnig@iem.at:
 On 04/10/12 10:33, katja wrote:
 i was talking about pd/pd-gui communication (and keep the number format for  both saving and pd/gui communication the same).  when displaying/updating a table every single number is converted to text  using printf, send over the wire and then converted back to a number for  drawing the table.  it makes a difference if you have to transmit 44100  *4 bytes or 44100*12 bytes.
Ah I see. It is not uncommon to display complete audio files, much more than 44100 samples.
I've never seen a patch that _displays_ all the data for an array that large. To
transmit 44,100 drawing instructions you need a _canvas_ size of "44100" and
if anyone has actually needed to do that in a patch I'd really like to see it.
So all these samples are converted to text and back to numbers, as they go over the network?
No. See plot_vis inside g_template.c. There is not a 1-to-1 correspondence between # of array elements and rectangles/polygon-coords on the tk canvas (at least for garrays, not sure about data structures). If you create a 44100 element array and make the "size" field in that arrays canvas dialog "2", Pd will only send data to the gui for those 2 pixels, not for the entire array. However, it will loop through the entire array on the c side _every_ time plot_vis is called, in order to figure out what info should be sent to the gui.
For example: running with -d 3, create an array with 10,000,000 elements. Now make the "size" field in its canvas dialog "2". Select the array. Click an array key to move it. Notice the lag, but also notice Pd is only sending two commands to the gui to draw the elements. It's because Pd must loop through 5,000,000 elements before it hits the next pixel where it needs to send another drawing instruction to the gui!
(While in the end, only a couple hundred values are displayed). And every character goes through the loop in binbuf_text() with all it's cases... well that is a bottleneck which should not be further aggravated. At least this performance issue can be quickly tested, using Pd vs Pd-double. I'll make a test patch for that.
Keep in mind that you're using a horribly implemented feature of Pd to do your test-- that is, if you're using garrays. For example, moving an array shouldn't
send _any_ element data to the gui. It doesn't in Pd-l2ork because it just moves
everything by tag, using one line of tk, thus there is no bottleneck in that case.
A practical test I can think of to compare 4byte vs 12byte payload is
something like [metro 100]--[tabwrite~] animation for a visible garray. I'd be
curious to know if there is a significant performance difference there.
-Jonathan
Katja
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Makes sense to me. Each individual point can have its own coords, fill, color, tags, etc. while a polygon just has one set of all those for the whole thing.
This whole discussion makes me think that arrays should be available to the GUI via shared memory. Then the 'pd' side can freely update things on its own clock, while 'pd-gui' can update things using its own clock (much slower, like 60hz) and also its own resolution. For example, if a 400 million point array is drawn in a 400 pixel wide box, then the GUI can just read every 100,000th value in the array. Or for more accuracy, take the median of those 100,000 points, or mean for perhaps more accuracy. That should drastically speed up array drawing.
.hc
On Apr 10, 2012, at 1:31 PM, Miller Puckette wrote:
Hi all -
It's a wierd thing abut TK that drawing polygons is far more efficient than drawing arrays of points; "polygons" are primitive objects that are apparently optimized internally to TK whereas arrays of points have to be drawn one by one (TK thinks they're each a separate object).
cheers Miller
On Tue, Apr 10, 2012 at 07:24:31PM +0200, katja wrote:
Thanks for your tips, Jonathan. Anyway, while you were writing the mail I was already doing a test patch where update times of a 32 million samples array and table are tested, see attached.
I ran the test with Pd-extended 0.43.1 and Pd-double on OSX. For an array with that length, update takes ~3 seconds. For a table displayed graphically, ~3.1 seconds. Surprisingly, drawing as points (instead of polygon) takes almost 5 seconds. The deviation between repeated measurements, a few dozen milliseconds, was about as large as the difference between Pd-extended and Pd-double. I've verified that double precision numbers are indeed displayed with a maximum of 14 significant digits in Pd-double.
I also checked the time for writing 32 million samples to a table without graphical display. This took ~170 milliseconds for Pd-extended and ~ 220 milliseconds for Pd-double.
Later I'll do another test where transmission over network is more specifically tested.
Katja
On Tue, Apr 10, 2012 at 7:05 PM, Jonathan Wilkes jancsika@yahoo.com wrote:
----- Original Message -----
From: katja katjavetter@gmail.com To: pd-list@iem.at Cc: Sent: Tuesday, April 10, 2012 9:45 AM Subject: Re: [PD] why does PD round numbers? (in tables, in messageboxes, etc)
2012/4/10 IOhannes m zmölnig zmoelnig@iem.at:
On 04/10/12 10:33, katja wrote:
i was talking about pd/pd-gui communication (and keep the number format for both saving and pd/gui communication the same). when displaying/updating a table every single number is converted to text using printf, send over the wire and then converted back to a number for drawing the table. it makes a difference if you have to transmit 44100 *4 bytes or 44100*12 bytes.
Ah I see. It is not uncommon to display complete audio files, much more than 44100 samples.
I've never seen a patch that _displays_ all the data for an array that large. To
transmit 44,100 drawing instructions you need a _canvas_ size of "44100" and
if anyone has actually needed to do that in a patch I'd really like to see it.
So all these samples are converted to text and back to numbers, as they go over the network?
No. See plot_vis inside g_template.c. There is not a 1-to-1 correspondence between # of array elements and rectangles/polygon-coords on the tk canvas (at least for garrays, not sure about data structures). If you create a 44100 element array and make the "size" field in that arrays canvas dialog "2", Pd will only send data to the gui for those 2 pixels, not for the entire array. However, it will loop through the entire array on the c side _every_ time plot_vis is called, in order to figure out what info should be sent to the gui.
For example: running with -d 3, create an array with 10,000,000 elements. Now make the "size" field in its canvas dialog "2". Select the array. Click an array key to move it. Notice the lag, but also notice Pd is only sending two commands to the gui to draw the elements. It's because Pd must loop through 5,000,000 elements before it hits the next pixel where it needs to send another drawing instruction to the gui!
(While in the end, only a couple hundred values are displayed). And every character goes through the loop in binbuf_text() with all it's cases... well that is a bottleneck which should not be further aggravated. At least this performance issue can be quickly tested, using Pd vs Pd-double. I'll make a test patch for that.
Keep in mind that you're using a horribly implemented feature of Pd to do your test-- that is, if you're using garrays. For example, moving an array shouldn't
send _any_ element data to the gui. It doesn't in Pd-l2ork because it just moves
everything by tag, using one line of tk, thus there is no bottleneck in that case.
A practical test I can think of to compare 4byte vs 12byte payload is
something like [metro 100]--[tabwrite~] animation for a visible garray. I'd be
curious to know if there is a significant performance difference there.
-Jonathan
Katja
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Looking at things from a more basic level, you can come up with a more direct solution... It may sound small in theory, but it in practice, it can change entire economies. - Amy Smith
----- Original Message -----
From: Hans-Christoph Steiner hans@at.or.at To: Miller Puckette msp@ucsd.edu Cc: pd-list@iem.at Sent: Tuesday, April 10, 2012 1:38 PM Subject: Re: [PD] why does PD round numbers? (in tables, in messageboxes, etc)
Makes sense to me. Each individual point can have its own coords, fill, color, tags, etc. while a polygon just has one set of all those for the whole thing.
This whole discussion makes me think that arrays should be available to the GUI via shared memory. Then the 'pd' side can freely update things on its own clock, while 'pd-gui' can update things using its own clock (much slower, like 60hz) and also its own resolution. For example, if a 400 million point array is drawn in a 400 pixel wide box, then the GUI can just read every 100,000th value in the array. Or for more accuracy, take the median of those 100,000 points, or mean for perhaps more accuracy. That should drastically speed up array drawing.
How do you calculate the median/mean without iterating through the array?
-Jonathan
.hc
On Apr 10, 2012, at 1:31 PM, Miller Puckette wrote:
Hi all -
It's a wierd thing abut TK that drawing polygons is far more efficient than drawing arrays of points; "polygons" are primitive objects
that
are apparently optimized internally to TK whereas arrays of points have to be drawn one by one (TK thinks they're each a separate object).
cheers Miller
On Tue, Apr 10, 2012 at 07:24:31PM +0200, katja wrote:
Thanks for your tips, Jonathan. Anyway, while you were writing the mail I was already doing a test patch where update times of a 32 million samples array and table are tested, see attached.
I ran the test with Pd-extended 0.43.1 and Pd-double on OSX. For an array with that length, update takes ~3 seconds. For a table displayed graphically, ~3.1 seconds. Surprisingly, drawing as points (instead of polygon) takes almost 5 seconds. The deviation between repeated measurements, a few dozen milliseconds, was about as large as the difference between Pd-extended and Pd-double. I've verified that double precision numbers are indeed displayed with a maximum of 14 significant digits in Pd-double.
I also checked the time for writing 32 million samples to a table without graphical display. This took ~170 milliseconds for Pd-extended and ~ 220 milliseconds for Pd-double.
Later I'll do another test where transmission over network is more specifically tested.
Katja
On Tue, Apr 10, 2012 at 7:05 PM, Jonathan Wilkes
jancsika@yahoo.com wrote:
----- Original Message -----
From: katja katjavetter@gmail.com To: pd-list@iem.at Cc: Sent: Tuesday, April 10, 2012 9:45 AM Subject: Re: [PD] why does PD round numbers? (in tables, in
messageboxes, etc)
2012/4/10 IOhannes m zmölnig zmoelnig@iem.at:
On 04/10/12 10:33, katja wrote:
i was talking about pd/pd-gui communication (and keep the
number format for
both saving and pd/gui communication the same). when displaying/updating a table every single number is
converted to text
using printf, send over the wire and then converted back
to a number for
drawing the table. it makes a difference if you have to transmit 44100 *4 bytes or 44100*12 bytes.
Ah I see. It is not uncommon to display complete audio files,
much
more than 44100 samples.
I've never seen a patch that _displays_ all the data for an
array that large. To
transmit 44,100 drawing instructions you need a _canvas_ size of
"44100" and
if anyone has actually needed to do that in a patch I'd really
like to see it.
So all these samples are converted to text and back to numbers, as they go over the network?
No. See plot_vis inside g_template.c. There is not a 1-to-1
correspondence between
# of array elements and rectangles/polygon-coords on the tk canvas
(at least for
garrays, not sure about data structures). If you create a 44100
element array
and make the "size" field in that arrays canvas dialog
"2", Pd will only send
data to the gui for those 2 pixels, not for the entire array.
However, it will loop
through the entire array on the c side _every_ time plot_vis is
called, in order to
figure out what info should be sent to the gui.
For example: running with -d 3, create an array with 10,000,000
elements.
Now make the "size" field in its canvas dialog
"2".
Select the array. Click an array key to move it. Notice the lag, but also notice Pd is only sending two commands to
the gui to
draw the elements. It's because Pd must loop through 5,000,000 elements before it
hits the next pixel
where it needs to send another drawing instruction to the gui!
(While in the end, only a couple hundred values are displayed). And every
character goes
through the loop in binbuf_text() with all it's cases...
well that is
a bottleneck which should not be further aggravated. At least this performance issue can be quickly tested, using Pd vs Pd-double.
I'll
make a test patch for that.
Keep in mind that you're using a horribly implemented feature
of Pd to do your
test-- that is, if you're using garrays. For example, moving
an array shouldn't
send _any_ element data to the gui. It doesn't in Pd-l2ork
because it just moves
everything by tag, using one line of tk, thus there is no
bottleneck in that case.
A practical test I can think of to compare 4byte vs 12byte payload
is
something like [metro 100]--[tabwrite~] animation for a visible
garray. I'd be
curious to know if there is a significant performance difference
there.
-Jonathan
Katja
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management ->
http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management ->
http://lists.puredata.info/listinfo/pd-list
Looking at things from a more basic level, you can come up with a more direct solution... It may sound small in theory, but it in practice, it can change entire economies. - Amy Smith
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Yes, you'd have to iterate. I'm sure there are algorithms for this kind of thing that are better, like in programs like Audacity. Those would be the ones to use.
.hc
(by the way, bottom-posting on a top-posted thread is bad form, IMHO. Whether you hate either top or bottom posting, I think its clear that its bad to mix).
On Apr 10, 2012, at 2:40 PM, Jonathan Wilkes wrote:
----- Original Message -----
How do you calculate the median/mean without iterating through the array?
-Jonathan
From: Hans-Christoph Steiner hans@at.or.at To: Miller Puckette msp@ucsd.edu Cc: pd-list@iem.at Sent: Tuesday, April 10, 2012 1:38 PM Subject: Re: [PD] why does PD round numbers? (in tables, in messageboxes, etc)
Makes sense to me. Each individual point can have its own coords, fill, color, tags, etc. while a polygon just has one set of all those for the whole thing.
This whole discussion makes me think that arrays should be available to the GUI via shared memory. Then the 'pd' side can freely update things on its own clock, while 'pd-gui' can update things using its own clock (much slower, like 60hz) and also its own resolution. For example, if a 400 million point array is drawn in a 400 pixel wide box, then the GUI can just read every 100,000th value in the array. Or for more accuracy, take the median of those 100,000 points, or mean for perhaps more accuracy. That should drastically speed up array drawing.
.hc
On Apr 10, 2012, at 1:31 PM, Miller Puckette wrote:
Hi all -
It's a wierd thing abut TK that drawing polygons is far more efficient than drawing arrays of points; "polygons" are primitive objects
that
are apparently optimized internally to TK whereas arrays of points have to be drawn one by one (TK thinks they're each a separate object).
cheers Miller
On Tue, Apr 10, 2012 at 07:24:31PM +0200, katja wrote:
Thanks for your tips, Jonathan. Anyway, while you were writing the mail I was already doing a test patch where update times of a 32 million samples array and table are tested, see attached.
I ran the test with Pd-extended 0.43.1 and Pd-double on OSX. For an array with that length, update takes ~3 seconds. For a table displayed graphically, ~3.1 seconds. Surprisingly, drawing as points (instead of polygon) takes almost 5 seconds. The deviation between repeated measurements, a few dozen milliseconds, was about as large as the difference between Pd-extended and Pd-double. I've verified that double precision numbers are indeed displayed with a maximum of 14 significant digits in Pd-double.
I also checked the time for writing 32 million samples to a table without graphical display. This took ~170 milliseconds for Pd-extended and ~ 220 milliseconds for Pd-double.
Later I'll do another test where transmission over network is more specifically tested.
Katja
On Tue, Apr 10, 2012 at 7:05 PM, Jonathan Wilkes
jancsika@yahoo.com wrote:
----- Original Message -----
From: katja katjavetter@gmail.com To: pd-list@iem.at Cc: Sent: Tuesday, April 10, 2012 9:45 AM Subject: Re: [PD] why does PD round numbers? (in tables, in
messageboxes, etc)
2012/4/10 IOhannes m zmölnig zmoelnig@iem.at: > On 04/10/12 10:33, katja wrote:
> i was talking about pd/pd-gui communication (and keep the
number format for
> both saving and pd/gui communication the same). > when displaying/updating a table every single number is
converted to text
> using printf, send over the wire and then converted back
to a number for
> drawing the table. > it makes a difference if you have to transmit 44100 > *4 bytes or 44100*12 bytes.
Ah I see. It is not uncommon to display complete audio files,
much
more than 44100 samples.
I've never seen a patch that _displays_ all the data for an
array that large. To
transmit 44,100 drawing instructions you need a _canvas_ size of
"44100" and
if anyone has actually needed to do that in a patch I'd really
like to see it.
So all these samples are converted to text and back to numbers, as they go over the network?
No. See plot_vis inside g_template.c. There is not a 1-to-1
correspondence between
# of array elements and rectangles/polygon-coords on the tk canvas
(at least for
garrays, not sure about data structures). If you create a 44100
element array
and make the "size" field in that arrays canvas dialog
"2", Pd will only send
data to the gui for those 2 pixels, not for the entire array.
However, it will loop
through the entire array on the c side _every_ time plot_vis is
called, in order to
figure out what info should be sent to the gui.
For example: running with -d 3, create an array with 10,000,000
elements.
Now make the "size" field in its canvas dialog
"2".
Select the array. Click an array key to move it. Notice the lag, but also notice Pd is only sending two commands to
the gui to
draw the elements. It's because Pd must loop through 5,000,000 elements before it
hits the next pixel
where it needs to send another drawing instruction to the gui!
(While in the end, only a couple hundred values are displayed). And every
character goes
through the loop in binbuf_text() with all it's cases...
well that is
a bottleneck which should not be further aggravated. At least this performance issue can be quickly tested, using Pd vs Pd-double.
I'll
make a test patch for that.
Keep in mind that you're using a horribly implemented feature
of Pd to do your
test-- that is, if you're using garrays. For example, moving
an array shouldn't
send _any_ element data to the gui. It doesn't in Pd-l2ork
because it just moves
everything by tag, using one line of tk, thus there is no
bottleneck in that case.
A practical test I can think of to compare 4byte vs 12byte payload
is
something like [metro 100]--[tabwrite~] animation for a visible
garray. I'd be
curious to know if there is a significant performance difference
there.
-Jonathan
Katja
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management ->
http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management ->
http://lists.puredata.info/listinfo/pd-list
Looking at things from a more basic level, you can come up with a more direct solution... It may sound small in theory, but it in practice, it can change entire economies. - Amy Smith
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
I have the audacity to believe that peoples everywhere can have three meals a day for their bodies, education and culture for their minds, and dignity, equality and freedom for their spirits. - Martin Luther King, Jr.
----- Original Message -----
From: Hans-Christoph Steiner hans@at.or.at To: Jonathan Wilkes jancsika@yahoo.com Cc: Miller Puckette msp@ucsd.edu; "pd-list@iem.at" pd-list@iem.at Sent: Tuesday, April 10, 2012 2:43 PM Subject: Re: [PD] why does PD round numbers? (in tables, in messageboxes, etc)
Yes, you'd have to iterate. I'm sure there are algorithms for this kind of thing that are better, like in programs like Audacity. Those would be the ones to use.
Pd already does iterate through the array. As I wrote, that's the source of the sluggishness users experience when they move one that has more than a couple hundred elements in anything other than pd-l2ork.
It might be worthwhile to look at Audacity, but then again I don't think those devs would be optimizing the code for the kind of realtime user interaction we deal with in Pd. I mean, one isn't going to record a track while at the same time drawing a waveform on another area of the screen, are they?
Garrays really cover a lot of user interface ground, which is a problem when you want to optimize any of their behavior. On the one hand, maybe it would be nice to accumulate element changes on the tcl side and only send them when the mouse button comes up (kind of like Max's [mousefilter]), but that would be to the detriment of the realtime response when that array is used in a signal object (as garrays are in many of the audio tutorials). Similarly, it would be a _vast_ improvement if array elements could jump-on-click, as so many people use them as a kind of poor man's Max-style multi-slider. But then graphs can hold multiple arrays...
In a way a garray is the most Max-like object in Pd, in that it is a monolithic multi-featured object that tries to cover as many use cases as it can at the expense of not being very extensible (I think of [coll], that lcd object, multislider, etc.). I guess the opposite would be something like the set of data structure objects, each of which have clear/simple functions which can be combined to do very powerful things with data and visualizations of it. But I suppose a measure of their power is whether or not they can be used to both replace and extend garrays; they cannot, and I haven't been able to think of any future additions to them that would make this possible.
-Jonathan
.hc
(by the way, bottom-posting on a top-posted thread is bad form, IMHO. Whether you hate either top or bottom posting, I think its clear that its bad to mix).
I was responding to the point you made directly before what I wrote. That's called "inlinereplying", and it is suggested in this FAQ: http://www.netmeister.org/news/learn2quote2.html#ss2.3
That is linked from the Pure Data Netiquette guide at: http://puredata.info/community/lists/Netiquette
-Jonathan
----- Original Message -----
From: IOhannes m zmölnig zmoelnig@iem.at To: pd-list@iem.at Cc: Sent: Tuesday, April 10, 2012 8:32 AM Subject: Re: [PD] why does PD round numbers? (in tables, in messageboxes, etc)
On 04/10/12 10:33, katja wrote:
I mean to say that switching to any format other than decimal ASCII would make it impossible for Pd to interpret patch files using the current format.
why?
Large tables are mostly stored in an audio format, rather than text.
i was talking about pd/pd-gui communication (and keep the number format for both saving and pd/gui communication the same). when displaying/updating a table every single number is converted to text using printf, send over the wire and then converted back to a number for drawing the table. it makes a difference if you have to transmit 44100 *4 bytes or 44100*12 bytes.
Wow! How wide is your screen?
-Jonathan
fgmasdr IOhannes
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On Tue, Apr 10, 2012 at 7:32 AM, IOhannes m zmölnig zmoelnig@iem.at wrote:
On 04/10/12 10:33, katja wrote:
I mean to say that switching to any format other than decimal ASCII would make it impossible for Pd to interpret patch files using the current format.
why?
I think that using any non-decimal ascii format would be bad for backwards compatibility. You'd need to have a function that would be able to distinguish the decimal numbers from old patches from the newer format. For example, 9000 is both a decimal and hexadecimal number--so then, you'd need identifiers to know which method to use.
It's not a very clear-cut tradeoff, to adopt anything other than longer decimal ascii numbers. Things could get very complicated trying to keep multiple formats alive--or they could be simple and just use a lot more characters.
The main reason why this is still like this is because no one has written better code, then done thorough testing in order to prove that the new code doesn't break anything. People have written better code for this before, no one has done the thorough testing part...
.hc
On Apr 7, 2012, at 10:58 PM, Martin Peach wrote:
It's because Pd saves the value by printing it as text into the patch file using a reduced precision format specifier (%g instead of %f, or %0.6f) so that the numbers look good on screen, with no extra zeros for example. I don't like it either.
Martin
On 2012-04-07 22:40, Angakok Thoth wrote:
that's not what i mean.
i mean, that when i write 12345678 (must be 100% accurate within 32bit float with 24bit mantissa) into an array and read it from there, it's still 12345678. but when i save that patch, close it and reload it, and i read from the array, i get 12345700.
nothing to do with limitations of floating point number.
On Sun, Apr 8, 2012 at 4:23 AM, i go bananas <hard.off@gmail.com mailto:hard.off@gmail.com> wrote:
http://en.wikipedia.org/wiki/Floating_point
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
"A cellphone to me is just an opportunity to be irritated wherever you are." - Linus Torvalds
On Sun, Apr 8, 2012 at 8:43 PM, Hans-Christoph Steiner hans@at.or.at wrote:
The main reason why this is still like this is because no one has written better code, then done thorough testing in order to prove that the new code doesn't break anything. People have written better code for this before, no one has done the thorough testing part...
Hans, can you point to such code, is there still something available somewhere? I may be able to test it using the testtools templates. Such code could be a very useful precision improvement, even though it's not the same as double precision.
Katja
Here's a patch I submitted: http://sourceforge.net/tracker/?func=detail&aid=2952880&group_id=557...
Martin
On 2012-04-08 15:17, katja wrote:
On Sun, Apr 8, 2012 at 8:43 PM, Hans-Christoph Steinerhans@at.or.at wrote:
The main reason why this is still like this is because no one has written better code, then done thorough testing in order to prove that the new code doesn't break anything. People have written better code for this before, no one has done the thorough testing part...
Hans, can you point to such code, is there still something available somewhere? I may be able to test it using the testtools templates. Such code could be a very useful precision improvement, even though it's not the same as double precision.
Katja
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On Apr 8, 2012, at 3:17 PM, katja wrote:
On Sun, Apr 8, 2012 at 8:43 PM, Hans-Christoph Steiner hans@at.or.at wrote:
The main reason why this is still like this is because no one has written better code, then done thorough testing in order to prove that the new code doesn't break anything. People have written better code for this before, no one has done the thorough testing part...
Hans, can you point to such code, is there still something available somewhere? I may be able to test it using the testtools templates. Such code could be a very useful precision improvement, even though it's not the same as double precision.
Hmm, can't think of any off hand, but its not too hard. My thought would be to sprintf("%.06f"), then strip off trailing zeros and decimal points. The rounding stuff is harder, but I am not sure that the current sprintf("%g") handles the rounding any differently:
(from man 3 printf) gG The double argument is converted in style f or e (or F or E for G conversions). The precision specifies the number of significant digits. If the precision is missing, 6 digits are given; if the precision is zero, it is treated as 1. Style e is used if the expo- nent from its conversion is less than -4 or greater than or equal to the precision. Trailing zeros are removed from the fractional part of the result; a decimal point appears only if it is followed by at least one digit.
.hc
I have the audacity to believe that peoples everywhere can have three meals a day for their bodies, education and culture for their minds, and dignity, equality and freedom for their spirits. - Martin Luther King, Jr.
On 2012-04-08 20:45, Hans-Christoph Steiner wrote:
On Apr 8, 2012, at 3:17 PM, katja wrote:
On Sun, Apr 8, 2012 at 8:43 PM, Hans-Christoph Steinerhans@at.or.at wrote:
The main reason why this is still like this is because no one has written better code, then done thorough testing in order to prove that the new code doesn't break anything. People have written better code for this before, no one has done the thorough testing part...
Hans, can you point to such code, is there still something available somewhere? I may be able to test it using the testtools templates. Such code could be a very useful precision improvement, even though it's not the same as double precision.
Hmm, can't think of any off hand, but its not too hard. My thought would be to sprintf("%.06f"), then strip off trailing zeros and decimal points. The rounding stuff is harder, but I am not sure that the current sprintf("%g") handles the rounding any differently:
(from man 3 printf) gG The double argument is converted in style f or e (or F or E for G conversions). The precision specifies the number of significant digits. If the precision is missing, 6 digits are given; if the precision is zero, it is treated as 1. Style e is used if the expo- nent from its conversion is less than -4 or greater than or equal to the precision. Trailing zeros are removed from the fractional part of the result; a decimal point appears only if it is followed by at least one digit.
I get this on WinXP:
float pi as %g: 3.14159 as %f: 3.141593 as %0.24f: 3.141592741012573200000000 double pi as %g: 3.14159 as %f: 3.141593 as %0.24f: 3.141592653589793100000000 Pi to 24 places: 3.141592653589793238462643.
using this code:
#include <stdio.h> #include<stdlib.h> #include <math.h>
int main (void) { float fpi = 4.0*atan(1.0); double dpi = 4.0*atan(1.0);
fprintf(stdout, "float pi as %%g: %g as %%f: %f as %%0.24f:
%0.24f\n", fpi, fpi, fpi); fprintf(stdout, "double pi as %%g: %g as %%f: %f as %%0.24f: %0.24f\n", dpi, dpi, dpi); fprintf(stdout, "Pi to 24 places: 3.141592653589793238462643.\n"); exit (EXIT_SUCCESS); }
Martin
-
Angakok Thoth -
Charles Henry -
Hans-Christoph Steiner -
i go bananas -
IOhannes m zmölnig -
Jonathan Wilkes -
katja -
Krzysztof Czaja -
Martin Peach -
Matteo Sisti Sette -
Miller Puckette -
Pedro Lopes