Hello list,
I'm trying to teach myself how to use the data structure feature in 0.38-3 - it seems this feature is still very much under development - is there a way to dynamically delete structures, to "un-append" them from a list? I want data structures to appear alongside sound-events, then expire as the events pass on.
BTW, I find that the inconsistent terminology used in the documentation to describe data structures makes it quite baffling to the neophyte, as terms like "list" (a list of scalars) confuse with a normal pd list of atoms (floats, symbols etc.) and they're treated quite differently. Also, the documentation hasn't been updated to reflect changes in the features (templates, obsolete by 0.38, are still referred to in the docs, for instance) - I would be more than willing to work with whomever has taken on such things to see to it that the documentation gets proofread and checked for consistency with each release - I've had no problem learning most of the rest of PD but these data structures are something different altogether. Or does anyone else out there know of any better/revised documentation? Do many people find the data structures useful? I think the idea has a lot of potential but hasn't yet been worked out in all its details. I'm interested in seeing this happen.
Thanks, David
Hallo, david golightly hat gesagt: // david golightly wrote:
I'm trying to teach myself how to use the data structure feature in 0.38-3
Stop right here! You lag behind an imporant version: pd 0.39 has much improved data structures and extended documentation.
- it seems this feature is still very much under development - is there a
way to dynamically delete structures, to "un-append" them from a list? I want data structures to appear alongside sound-events, then expire as the events pass on.
... however even 0.39 can not yet delete structure instances by messages. I guess this is a bit tricky to implement, as it leads to invalid pointers, however it would be really useful.
I guess you're already beyond that, but the pd~convention docs contain a data structure tutorial by me, available on puredata.info in the pd~convention Documentation folder, which is a bit easier to follow than Miller's tutorial IMO, if you're really just starting out with data structures. I need to update it to 0.39 though.
Ciao
Frank Barknecht _ ______footils.org_ __goto10.org__
On Sun, 2 Oct 2005, david golightly wrote:
I'm trying to teach myself how to use the data structure feature in 0.38-3 - it seems this feature is still very much under development - is there a way to dynamically delete structures, to "un-append" them from a list? I want data structures to appear alongside sound-events, then expire as the events pass on.
Well, that would require not just un-append but also delete from anywhere
in the linked-list. There's possibly also a difference between deleting an
object and just removing it from the list. If it's possible to just remove
the object and keep it alive, or if it's possible to keep pointers to any
object, then it's necessary to use a deallocation mechanism, unless the
plan is to never deallocate, or to make Pd just as low-level as C/C++.
Other than C/C++, which languages make it normal to put the
responsibility/burden of deallocating on the programmer? How many Pd users
want to do that?
BTW, I find that the inconsistent terminology used in the documentation to describe data structures makes it quite baffling to the neophyte, as terms like "list" (a list of scalars) confuse with a normal pd list of atoms (floats, symbols etc.) and they're treated quite differently.
Yes they are. From language to language, terminology varies a lot, but Pd is the only one I know which has always used twice the word "list" to mean two completely different things, and this at the core of the language.
On one side there are list-messages (and anything-messages) which are stack-allocated indexed-lists of atoms, which you can't explicitly get a pointer to.
On the other side you have heap-allocated linked-lists of atoms, which you can (and have) to get a pointer to.
That's two completely different worlds.
Also, the name "data structures" is very unfortunate because it already means more than that and less than that too. Everywhere else, data structures are considered to not be tied to any appearance at all, and they are normally a lot more flexible than they are in Pd (especially before 0.39, but even with 0.39)
Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju Freelance Digital Arts Engineer, Montréal QC Canada
Well, I ended up abandoning attempts to delete the data structures and have
just assigned one data structure to each voice in a polyphonic texture, so
that the DS's get moved around in response to note-events within that voice
rather than created & deleted. Seems to work fine for now, but nonetheless:
it seems to me that creating an effective DS system would involve the use
of doubly-linked lists, each atom having (in C++ terminology) a pointer to
the next object and to the previous object, which are then updated each time
anything gets added or deleted to the list, making it less of a list and
more a 2-way ring (like the <deque> template in C++'s STL), as following
successive pointers in any direction gets you back where you started. That
way, deleting an atom from the list causes the list to close up around the
gap, and the memory address of the defunct atom is returned to the heap and
rendered irrelevant while all remaining pointer references remain valid.
Stuff like this is old hat in C++, and doesn't necessarily involve direct
user allocation/deallocation of memory - a simplified PD equivalent of the
Standard Template Library would be a good way to approach this problem. I
think it can be done automatically without introducing end-user memory
management, though I'm going to have to take a look at the source myself
before signing this statement in my own blood.
On a more germane note, how about integrating the graphical representation of DS's with the rest of PD as useful tools and not just pretty shapes? It would be fabulous if you could somehow "read" the contours of a DS bezier curve like you would an array (ack! another term that gets used twice!! - I mean, a table), without having to literally input hundreds of points - you could just have breakpoints in the DS, but enable indexing of the intermediate points on the curve... which would necessarily be dependent on the DS points - does this make sense? That way you could for instance just use the lines or curves of a DS as a table to control envelopes, any parameter you want, really, without having to do the interpolating yourself. Or am I taking crazy pills here? Just some thoughts.
-David
From: Mathieu Bouchard matju@artengine.ca To: david golightly davigoli@hotmail.com CC: pd-list@iem.at Subject: Re: [PD] deleting data structures Date: Tue, 4 Oct 2005 01:18:57 -0400 (EDT)
On Sun, 2 Oct 2005, david golightly wrote:
I'm trying to teach myself how to use the data structure feature in 0.38-3 - it seems this feature is still very much under development - is there a way to dynamically delete structures, to "un-append" them from a list? I want data structures to appear alongside sound-events, then expire as the events pass on.
Well, that would require not just un-append but also delete from anywhere in the linked-list. There's possibly also a difference between deleting an object and just removing it from the list. If it's possible to just remove the object and keep it alive, or if it's possible to keep pointers to any object, then it's necessary to use a deallocation mechanism, unless the plan is to never deallocate, or to make Pd just as low-level as C/C++. Other than C/C++, which languages make it normal to put the responsibility/burden of deallocating on the programmer? How many Pd users want to do that?
BTW, I find that the inconsistent terminology used in the documentation to describe data structures makes it quite baffling to the neophyte, as terms like "list" (a list of scalars) confuse with a normal pd list of atoms (floats, symbols etc.) and they're treated quite differently.
Yes they are. From language to language, terminology varies a lot, but Pd is the only one I know which has always used twice the word "list" to mean two completely different things, and this at the core of the language.
On one side there are list-messages (and anything-messages) which are stack-allocated indexed-lists of atoms, which you can't explicitly get a pointer to.
On the other side you have heap-allocated linked-lists of atoms, which you can (and have) to get a pointer to.
That's two completely different worlds.
Also, the name "data structures" is very unfortunate because it already means more than that and less than that too. Everywhere else, data structures are considered to not be tied to any appearance at all, and they are normally a lot more flexible than they are in Pd (especially before 0.39, but even with 0.39)
Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju Freelance Digital Arts Engineer, Montréal QC Canada
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Hallo, david golightly hat gesagt: // david golightly wrote:
Well, I ended up abandoning attempts to delete the data structures and have just assigned one data structure to each voice in a polyphonic texture, so that the DS's get moved around in response to note-events within that voice rather than created & deleted. Seems to work fine for now, but nonetheless: it seems to me that creating an effective DS system would involve the use of doubly-linked lists, each atom having (in C++ terminology) a pointer to the next object and to the previous object, which are then updated each time anything gets added or deleted to the list, making it less of a list and more a 2-way ring (like the <deque> template in C++'s STL), as following successive pointers in any direction gets you back where you started. That way, deleting an atom from the list causes the list to close up around the gap, and the memory address of the defunct atom is returned to the heap and rendered irrelevant while all remaining pointer references remain valid.
This sounds like a sensible approach.
To add a third name, that is used in different contexts to mean different things: The problem with deleting a data structure is not only one of memory (de)allocation (that can be solved using C language features), but the fact, that there is a [pointer] Pd type, which can be passed around just like a float or a symbol can. A float can not become invalid, however a [pointer] can:
If you work a bit with DS, you will encounter messages like
error: ptrobj_bang: empty pointer error: ptrobj_next: stale pointer error: set: empty pointer
a lot, if you delete a DS, whose [pointer] is still in use in your patch. This doesn't crash Pd (most of the time) but it will do other strange things. For example in the Midi-Loop-Sequencer I posted (also in my PWD~ ("public working directory") at [1]) the sequencing will stop as soon as you delete a note in your pattern, probably because not only the deleted [pointer] became invalid, but also all other [pointer]s in the same list (i.e. subwindow).
So to allow clean deletion of DS-objects, some care has to be taken with dangling [pointer]s in a Pd patch's logic, too.
However as deletion of DS is possible by hand (and also by Pd's internal messages for mouse movement and editing) adding a "remove"-method to [pointer] would not create a fundamentaly new problem, as far as I can see.
[1] http://royalrabbit.goto10.org/svn/goto10/pd-patches/fbar/pipeseq/
Ciao
Frank Barknecht _ ______footils.org_ __goto10.org__
Hm, some good insights Frank. I'm not too familiar with how PD handles its
own [pointer] objs internally but perhaps it might be a good approach for
devel to remodel after the C/C++ technique I described. As you say, I don't
see how deleting by messages would be fundametally different from deleting
by mouse click, in terms of internal relations between pointers in a list.
Again, in C++ terminology (and I don't have to tell this to any developer!),
most instances in a linked list will contain a pointer to the next obj & a
pointer to the previous obj. Upon deletion, the object will call a
"deconstructor" routine that looks at the prev. obj and makes its "next"
pointer point to the current obj's "next", and vice versa, like a circle of
people holding hands, and you step out of the circle but join the hands of
the two people next to you to keep the circle from breaking. Therefore, no
pointer references become "invalid" because no pointers have moved! If this
is how things work in C/C++ (and most other mid-level languages), I don't
see why this couldn't be applied to the internal pointer logic in PD as
well...
I don't think my C chops are up to the task of taking on a re-engineering of the internal [pointer] logic the DS, though again I'll have to take a look at it and see. I could be waaaay off base here, and there are doubtless other technicalities I'm not seeing. Any one in Devel who has any thoughts?
Thanks, David
From: Frank Barknecht fbar@footils.org To: pd-list@iem.at Subject: Re: [PD] deleting data structures Date: Tue, 4 Oct 2005 21:26:21 +0200
Hallo, david golightly hat gesagt: // david golightly wrote:
Well, I ended up abandoning attempts to delete the data structures and
have
just assigned one data structure to each voice in a polyphonic texture,
so
that the DS's get moved around in response to note-events within that
voice
rather than created & deleted. Seems to work fine for now, but nonetheless: it seems to me that creating an effective DS system would involve the use of doubly-linked lists, each atom having (in C++ terminology) a pointer to the next object and to the previous object,
which
are then updated each time anything gets added or deleted to the list, making it less of a list and more a 2-way ring (like the <deque>
template
in C++'s STL), as following successive pointers in any direction gets
you
back where you started. That way, deleting an atom from the list causes the list to close up around the gap, and the memory address of the
defunct
atom is returned to the heap and rendered irrelevant while all remaining pointer references remain valid.
This sounds like a sensible approach.
To add a third name, that is used in different contexts to mean different things: The problem with deleting a data structure is not only one of memory (de)allocation (that can be solved using C language features), but the fact, that there is a [pointer] Pd type, which can be passed around just like a float or a symbol can. A float can not become invalid, however a [pointer] can:
If you work a bit with DS, you will encounter messages like
error: ptrobj_bang: empty pointer error: ptrobj_next: stale pointer error: set: empty pointer
a lot, if you delete a DS, whose [pointer] is still in use in your patch. This doesn't crash Pd (most of the time) but it will do other strange things. For example in the Midi-Loop-Sequencer I posted (also in my PWD~ ("public working directory") at [1]) the sequencing will stop as soon as you delete a note in your pattern, probably because not only the deleted [pointer] became invalid, but also all other [pointer]s in the same list (i.e. subwindow).
So to allow clean deletion of DS-objects, some care has to be taken with dangling [pointer]s in a Pd patch's logic, too.
However as deletion of DS is possible by hand (and also by Pd's internal messages for mouse movement and editing) adding a "remove"-method to [pointer] would not create a fundamentaly new problem, as far as I can see.
[1] http://royalrabbit.goto10.org/svn/goto10/pd-patches/fbar/pipeseq/
Ciao
Frank Barknecht _ ______footils.org_ __goto10.org__
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
...meaning, ultimately, that deleting a pointer simply shifts the previous reference to the next one. You still ultimately have the "null pointer" to deal with if you delete all the elements in the list. But this way you don't have to see it as much. It's having your pointer cake & eating it too.
Just downloaded 0.39 - the DS tutorial is brilliant - much clearer now, thanks Frank. Highly recommended for anyone scared away from DS in the past.
-David
From: "david golightly" davigoli@hotmail.com To: fbar@footils.org, pd-list@iem.at Subject: Re: [PD] deleting data structures Date: Tue, 04 Oct 2005 15:57:55 -0400
Hm, some good insights Frank. I'm not too familiar with how PD handles its own [pointer] objs internally but perhaps it might be a good approach for devel to remodel after the C/C++ technique I described. As you say, I don't see how deleting by messages would be fundametally different from deleting by mouse click, in terms of internal relations between pointers in a list. Again, in C++ terminology (and I don't have to tell this to any developer!), most instances in a linked list will contain a pointer to the next obj & a pointer to the previous obj. Upon deletion, the object will call a "deconstructor" routine that looks at the prev. obj and makes its "next" pointer point to the current obj's "next", and vice versa, like a circle of people holding hands, and you step out of the circle but join the hands of the two people next to you to keep the circle from breaking.
Therefore, no pointer references become "invalid" because no pointers have moved! If this is how things work in C/C++ (and most other mid-level languages), I don't see why this couldn't be applied to the internal pointer logic in PD as well...I don't think my C chops are up to the task of taking on a re-engineering of the internal [pointer] logic the DS, though again I'll have to take a look at it and see. I could be waaaay off base here, and there are doubtless other technicalities I'm not seeing. Any one in Devel who has any thoughts?
Thanks, David
From: Frank Barknecht fbar@footils.org To: pd-list@iem.at Subject: Re: [PD] deleting data structures Date: Tue, 4 Oct 2005 21:26:21 +0200
Hallo, david golightly hat gesagt: // david golightly wrote:
Well, I ended up abandoning attempts to delete the data structures and
have
just assigned one data structure to each voice in a polyphonic texture,
so
that the DS's get moved around in response to note-events within that
voice
rather than created & deleted. Seems to work fine for now, but nonetheless: it seems to me that creating an effective DS system would involve the use of doubly-linked lists, each atom having (in C++ terminology) a pointer to the next object and to the previous object,
which
are then updated each time anything gets added or deleted to the list, making it less of a list and more a 2-way ring (like the <deque>
template
in C++'s STL), as following successive pointers in any direction gets
you
back where you started. That way, deleting an atom from the list
causes
the list to close up around the gap, and the memory address of the
defunct
atom is returned to the heap and rendered irrelevant while all
remaining
pointer references remain valid.
This sounds like a sensible approach.
To add a third name, that is used in different contexts to mean different things: The problem with deleting a data structure is not only one of memory (de)allocation (that can be solved using C language features), but the fact, that there is a [pointer] Pd type, which can be passed around just like a float or a symbol can. A float can not become invalid, however a [pointer] can:
If you work a bit with DS, you will encounter messages like
error: ptrobj_bang: empty pointer error: ptrobj_next: stale pointer error: set: empty pointer
a lot, if you delete a DS, whose [pointer] is still in use in your patch. This doesn't crash Pd (most of the time) but it will do other strange things. For example in the Midi-Loop-Sequencer I posted (also in my PWD~ ("public working directory") at [1]) the sequencing will stop as soon as you delete a note in your pattern, probably because not only the deleted [pointer] became invalid, but also all other [pointer]s in the same list (i.e. subwindow).
So to allow clean deletion of DS-objects, some care has to be taken with dangling [pointer]s in a Pd patch's logic, too.
However as deletion of DS is possible by hand (and also by Pd's internal messages for mouse movement and editing) adding a "remove"-method to [pointer] would not create a fundamentaly new problem, as far as I can see.
[1] http://royalrabbit.goto10.org/svn/goto10/pd-patches/fbar/pipeseq/
Ciao
Frank Barknecht _ ______footils.org_ __goto10.org__
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
I posted this composition that I did using the data structures to this
list a while back. I definitely enjoyed the process and am going to
develop my sequencer program further once I find the time...
.hc
On Oct 2, 2005, at 6:05 PM, david golightly wrote:
Hello list,
I'm trying to teach myself how to use the data structure feature in
0.38-3 - it seems this feature is still very much under development -
is there a way to dynamically delete structures, to "un-append" them
from a list? I want data structures to appear alongside sound-events,
then expire as the events pass on.BTW, I find that the inconsistent terminology used in the
documentation to describe data structures makes it quite baffling to
the neophyte, as terms like "list" (a list of scalars) confuse with a
normal pd list of atoms (floats, symbols etc.) and they're treated
quite differently. Also, the documentation hasn't been updated to
reflect changes in the features (templates, obsolete by 0.38, are
still referred to in the docs, for instance) - I would be more than
willing to work with whomever has taken on such things to see to it
that the documentation gets proofread and checked for consistency with
each release - I've had no problem learning most of the rest of PD but
these data structures are something different altogether. Or does
anyone else out there know of any better/revised documentation? Do
many people find the data structures useful? I think the idea has a
lot of potential but hasn't yet been worked out in all its details.
I'm interested in seeing this happen.Thanks, David
PD-list@iem.at mailing list UNSUBSCRIBE and account-management ->
http://lists.puredata.info/listinfo/pd-list
Man has survived hitherto because he was too ignorant to know how to
realize his wishes.
Now that he can realize them, he must either change them, or perish.
-William Carlos
Williams
-
david golightly -
Frank Barknecht -
Hans-Christoph Steiner -
Mathieu Bouchard