Hi list. I have a question on allocating data and reading it through an external. As I have seen so far, the externals that reads arrays, as tabread, allocate the data in their own memory space. As far as I can understand, this result in a "double allocation" of the data. One in the array itself and the other one in the internal memory space of the instance of the tabread external. The point is that I would like to use an array to hold in memory a considerable amount of data and to make it available to an arbitrary "reader" objects without the need of allocating the data again in each one of the readers. Is it possible to write an external that that reads data from an array "on the fly" without the need of allocating the data in its own memory space? Any help will be most appreciate.
Oscar Pablo Di Liscia
I'm pretty sure that the [tab*] objects just point into existing arrays rather than copying them. Each time [tabread] receives a float, for instance, it checks to see if the array it has been set to read from exists, and if so, updates a pointer to the indicated index of the array, and grabs the float stored there.
On Sun, Sep 20, 2015 at 6:22 PM, oscar pablo di liscia odiliscia@gmail.com wrote:
Hi list. I have a question on allocating data and reading it through an external. As I have seen so far, the externals that reads arrays, as tabread, allocate the data in their own memory space. As far as I can understand, this result in a "double allocation" of the data. One in the array itself and the other one in the internal memory space of the instance of the tabread external. The point is that I would like to use an array to hold in memory a considerable amount of data and to make it available to an arbitrary "reader" objects without the need of allocating the data again in each one of the readers. Is it possible to write an external that that reads data from an array "on the fly" without the need of allocating the data in its own memory space? Any help will be most appreciate.
Oscar Pablo Di Liscia
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On 2015-09-21 00:22, oscar pablo di liscia wrote:
Hi list. I have a question on allocating data and reading it through an external. As I have seen so far, the externals that reads arrays, as tabread, allocate the data in their own memory space. As far as I can understand, this result in a "double allocation" of the data. One in the array itself and the other one in the internal memory space of the instance of the tabread external. The point is that I would like to use an array to hold in memory a considerable amount of data and to make it available to an arbitrary "reader" objects without the need of allocating the data again in each one of the readers. Is it possible to write an external that that reads data from an array "on the fly" without the need of allocating the data in its own memory space? Any help will be most appreciate.
virtually all¹ objects accessing table data reference the data "directly" using the "garray_getfloatwords()" function to get a pointer where the array stores its data. this does not involve any double allocation at all. it is also the only sane way to always use up-to-date data: if the user modifies the data via [tabwrite] or whatever, there is no way to signal the table consumers that the table data has changed (so that they can copy the relevant new data into their local copy), so the consumer must use the changed data directly.
which objects have you found that proxy the data?
gfasd,rt IOhannes
¹ i seem to remember that [partconv~] does copy the data during DSP-graph recompilation, mainly because it does some heavy processing on the data (FFTs and what not), which it doesn't want to do for each signal block. i might be mistaken though.
Many thanks. I am trying to load in a table spectral data to make these data available to an arbitrary number of "readers". I will see how I can deal with this using the tabread and tabwrite externals. Also, it is not clear to me the meaning and the use of the function: "garray_usedindsp" and what is the effect of it. Of course I know that the code is available, but if if somebody can give me a brief idea, I will save a lot of time. Best
Oscar Pablo Di Liscia
2015-09-21 4:27 GMT-03:00 IOhannes m zmoelnig zmoelnig@iem.at:
On 2015-09-21 00:22, oscar pablo di liscia wrote:
Hi list. I have a question on allocating data and reading it through an external. As I have seen so far, the externals that reads arrays, as tabread,
allocate
the data in their own memory space. As far as I can understand, this
result
in a "double allocation" of the data. One in the array itself and the
other
one in the internal memory space of the instance of the tabread external. The point is that I would like to use an array to hold in memory a considerable amount of data and to make it available to an arbitrary "reader" objects without the need of allocating the data again in each one of the readers. Is it possible to write an external that that reads data from an array
"on
the fly" without the need of allocating the data in its own memory space? Any help will be most appreciate.
virtually all¹ objects accessing table data reference the data "directly" using the "garray_getfloatwords()" function to get a pointer where the array stores its data. this does not involve any double allocation at all. it is also the only sane way to always use up-to-date data: if the user modifies the data via [tabwrite] or whatever, there is no way to signal the table consumers that the table data has changed (so that they can copy the relevant new data into their local copy), so the consumer must use the changed data directly.
which objects have you found that proxy the data?
gfasd,rt IOhannes
¹ i seem to remember that [partconv~] does copy the data during DSP-graph recompilation, mainly because it does some heavy processing on the data (FFTs and what not), which it doesn't want to do for each signal block. i might be mistaken though.
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On 2015-09-21 16:04, oscar pablo di liscia wrote:
Many thanks. I am trying to load in a table spectral data to make these data available to an arbitrary number of "readers". I will see how I can deal with this using the tabread and tabwrite externals. Also, it is not clear to me the meaning and the use of the function: "garray_usedindsp" and what is the effect of it. Of course I know that the code is available, but if if somebody can give me a brief idea, I will save a lot of time.
incidentally this has been discussed a few hours ago on this mailinglist [1].
in short: garray_usedindsp() marks the table as being used in DSP processing, so any change to the table that involves potential invalidating of the memory returned to by garray_getfloatwords() will re-trigger a recompilation of the DSP graph.
fasdrm IOhannes
[1] http://lists.puredata.info/pipermail/pd-list/2015-09/111520.html
Many thanks. So that, if a table is not used in a dsp process, and we don't use the garray_usedindsp() function, it is safe to change its contents providing the fact that this change is properly done using the access methods available?
Oscar Pablo Di Liscia
2015-09-21 11:15 GMT-03:00 IOhannes m zmoelnig zmoelnig@iem.at:
On 2015-09-21 16:04, oscar pablo di liscia wrote:
Many thanks. I am trying to load in a table spectral data to make these data available to an arbitrary number of "readers". I will see how I can deal with this using the tabread and tabwrite externals. Also, it is not clear to me the meaning and the use of the function: "garray_usedindsp" and what is the effect of it. Of course I know that the code is available, but if if somebody can give me a brief idea, I will save a lot of time.
incidentally this has been discussed a few hours ago on this mailinglist [1].
in short: garray_usedindsp() marks the table as being used in DSP processing, so any change to the table that involves potential invalidating of the memory returned to by garray_getfloatwords() will re-trigger a recompilation of the DSP graph.
fasdrm IOhannes
[1] http://lists.puredata.info/pipermail/pd-list/2015-09/111520.html
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On 2015-09-21 16:24, oscar pablo di liscia wrote:
Many thanks. So that, if a table is not used in a dsp process, and we don't use the garray_usedindsp() function, it is safe to change its contents providing the fact that this change is properly done using the access methods available?
really there is only a single access method (to the data) available: garray_getfloatwords() returns a pointer to an array of words which you can dereference to read or write the actual values.
it's always safe to *immediately* use the data returned by garray_getfloatwords().
however, if you want - for performance reasons - to cache the returned pointer, then you need some wy of invalidating it (e.g. because the table the pointer referred to has moved to some other place). that's the use case of garray_usedindsp().
it builds on the assumption that only DSP objects would use this kind of optimization. as a result, you *can* only use this optimization in DSP perform routines. all other table access has to be done through garray_getfloatwords() immediately before dereferencing the pointer.
if you must change the size of a table, use garray_resize_long(); after that you must call garray_usedindsp() again to get the new data pointer.
hopefully this makes it a bit clearer.
fgmasdr IOhannes
Very clear, many thanks.
Oscar Pablo Di Liscia
2015-09-21 11:41 GMT-03:00 IOhannes m zmoelnig zmoelnig@iem.at:
On 2015-09-21 16:24, oscar pablo di liscia wrote:
Many thanks. So that, if a table is not used in a dsp process, and we don't use the garray_usedindsp() function, it is safe to change its contents providing the fact that this change is properly done using the access methods available?
really there is only a single access method (to the data) available: garray_getfloatwords() returns a pointer to an array of words which you can dereference to read or write the actual values.
it's always safe to *immediately* use the data returned by garray_getfloatwords().
however, if you want - for performance reasons - to cache the returned pointer, then you need some wy of invalidating it (e.g. because the table the pointer referred to has moved to some other place). that's the use case of garray_usedindsp().
it builds on the assumption that only DSP objects would use this kind of optimization. as a result, you *can* only use this optimization in DSP perform routines. all other table access has to be done through garray_getfloatwords() immediately before dereferencing the pointer.
if you must change the size of a table, use garray_resize_long(); after that you must call garray_usedindsp() again to get the new data pointer.
hopefully this makes it a bit clearer.
fgmasdr IOhannes
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
-
IOhannes m zmoelnig -
Matt Barber -
oscar pablo di liscia