Hi,
I'm trying to use integers in pd externals, but found out, there is no such type. Therefor it seems to be impossible to deal with big integer numbers (ca. 2 ^ 20 and above). Since I'm using them for masking purposes, the numbers have to be transmitted correctly between objects and I would like to use at least 32-bit precision
My question:
Is it possible to have integer inlets in externals?
How could a message preserve the precise value of the integer (I found out, typing a big integer in a message box and closing/reopening the patch results in a cast to a real number with 7 in radix 10 notation)?
If all this fails: How exact can a float represent an integer number (up to which value would the following function print "equal")?
void myclass_float (t_myclass *x, t_floatarg f) { int i;
i = f;
if (i == f) post("equal"); else post("not equal"); }
Thanx in advance,
Orm
Looks like the maxuimum you can put in a number box without it reverting to exponential notation is 999999. That's enough for 18 bits of masking (using the decimal digits for three bits each so that you can actually read the masks.) For more than 18 bits worth, it's probably best to use lists.
cheers Miller
On Wed, Jan 16, 2002 at 04:33:01PM +0100, Orm Finnendahl wrote:
hi,
how about passing symbols, possibly written as somehow prefixed octal, hex or binary numbers, and then atoi()/strtol()/sscanf()/whatever() them to integers?
But I wonder: if there was a huge amount of different masks generated during a Pd session, how this would affect the overall performance? What I mean is that the penalty of symbol hash table slot chaining could get high.
If this is true, then I wonder: is the above a viable reason for giving users a way (like sending a message to pd) to clear symbol table of currently unused (not bound, etc.) elements?
Krzysztof
Orm Finnendahl wrote: ...
hi,
thanx to miller and krzystof for the ideas!
Am Donnerstag, den 17. Januar 2002 um 12:03:24 Uhr (+0100) schrieb Krzysztof Czaja:
That's a very nice idea, but I think, I will end up using Millers suggestion of passing lists of smaller integers instead, since that's the way I used the object before anyway (see below).
Yes, that would be definitely necessary, as the amount of generated symbols would be immense.
The object, I'm needing it for is the markov object, I mentioned some time ago. It stores elements in two data structures: (1.) A table, implemented as linked list, which defines the succession of elements and (2.) a correlated multidimensional search tree, which stores the statistics of occurence of elements of the table in respect to markov properties of any order. These statistics can for example be used to generate new elements with probabilities which match the current state of the table. The table can be extended by supplying elements to the object from the outside world, or by the elements generated by the object itself, it can be shrunk, manipulated etc...
The elements to be stored need to have the following properties:
They have to be comparable not only for equality, but also, in special cases, whether they lie in a certain range (for example, if you code delta times of performers in real time performances, or if some properties of a complex event should not matter in respect of markov properties, but need to be stored within the element nevertheless)
It should be possible to store an element with more than one dimension, for example if you want to store a chord progression including the velocities or delta times.
The idea, I came up with when I first wrote the object, was, to use one 32-bit integer value internally for each element and define it as up to 8 bitfields of variable (and userdefinable) width. In practice that worked fine even with 32 bits, although sometimes problems arose (like if you want to code 5 voice counterpoints including interval structure, metrical weight, delta time and tie in information). It would probably be nicest, to use pointers as reference to elements of arbitrary type instead of integers since the 32 bit limitations could be avoided. But then the user would have to provide a function which tests for equality, and I have no idea, how this could be done within the framework of pd.
I will probably end up using integers again internally (maybe 64 bits instead of 32) and will send a list for storing each element which is range checked, length checked and then converted into an integer. But if someone has another idea, how to implement this, it would be greatly appreciated. As I put in some work, I would really like, if the object is considered useful by a lot of people and this is the chance to have the object tailored to your own needs.
grüezi Orm
hi,
Am Donnerstag, den 17. Januar 2002 um 12:03:24 Uhr (+0100) schrieb Krzysztof Czaja:
That's a very nice idea, but I think, I will end up using Millers suggestion of passing lists of smaller integers instead, since that's the way I used the object before anyway (see below).
Yes, that would be definitely necessary, as the amount of generated symbols would be immense.
The object, I'm needing it for is the markov object, I mentioned some time ago. It stores elements in two data structures: (1.) A table, implemented as linked list, which defines the succession of elements and (2.) a correlated multidimensional search tree, which stores the statistics of occurence of elements of the table in respect to markov properties of any order. These statistics can for example be used to generate new elements with probabilities which match the current state of the table. The table can be extended by supplying elements to the object from the outside world, or by the elements generated by the object itself, it can be shrunk, manipulated etc...
The elements to be stored need to have the following properties:
They have to be comparable not only for equality, but also, in special cases, whether they lie in a certain range (for example, if you code delta times of performers in real time performances, or if some properties of a complex event should not matter in respect of markov properties, but need to be stored within the element nevertheless)
It should be possible to store an element with more than one dimension, for example if you want to store a chord progression including the velocities or delta times.
The idea, I came up with when I first wrote the object, was, to use one 32-bit integer value internally for each element and define it as up to 8 bitfields of variable (and userdefinable) width. In practice that worked fine even with 32 bits, although sometimes problems arose (like if you want to code 5 voice counterpoints including interval structure, metrical weight, delta time and tie in information). It would probably be nicest, to use pointers as reference to elements of arbitrary type instead of integers since the 32 bit limitations could be avoided. But then the user would have to provide a function which tests for equality, and I have no idea, how this could be done within the framework of pd.
I will probably end up using integers again internally (maybe 64 bits instead of 32) and will send a list for storing each element which is range checked, length checked and then converted into an integer. But if someone has another idea, how to implement this, it would be greatly appreciated. As I put in some work, I would really like, if the object is considered useful by a lot of people and this is the chance to have the object tailored to your own needs.
grüezi Orm
hi Orm,
much of what follows is my guesswork, hope it is not completely off target...
there is no real need for any coding of the elements, except to speed up the comparisons. If this is not a bottleneck, then just store the elements as arrays of floats packed into a standard struct _elem { struct _elem *next; float data[1]; }. The markov class would accept list messages, and would be instantiated like [markov <ndimensions>]. The main struct of the class would have ``ndimen'' field (and probably also elemsize field added for speed and convenience).
real-time performance input without quantizing (possibly vector quantizing) could be useful -- with too broad a sample space one needs huge amount of ``training'' data to start with. And if there is to be quantizing, then I would rather do it in separate externals or abstractions tuned for any particular domain, not in a generic markov external. It may occur, that in most cases such a generic object works best on a small one-dimensional sample space (obtained after vector quantizing), that will fit into undistorted floats...
Krzysztof
Orm Finnendahl wrote: ...
Hi Krzysztof,
Am Samstag, den 19. Januar 2002 um 17:25:44 Uhr (+0100) schrieb Krzysztof Czaja:
That was the major reason. When I wrote the object, cputime was expensive (my computer had 8MHz clocking) and I was obsessed with efficiency.
You are absolutely right: Since it is a tree structure, you only need up to 20 comparisons to find any element in a tree with 1000000 elements and therefore it makes perfect sense to use float arrays altogether. But why did you suggest the linked list structure? I guess, a normal array of floats works well, costs less memory space and still can be initialized in the setup function to an arbitrary size.
Not really. I attached a markov object to the live performance of an improvising pianist, capturing pitch, velocity and timing and the object could play along in realtime after only a couple of seconds, continuing the stuff of the pianist in much the same manner and style, as he did.
That was the way I used it. Quantization was of course done outside the object.
To store only one dimension is normally not sufficient to capture musical events. It gives a completely different result if you seperate different musical dimensions in different markov objects than combining them in one object. And even if you only use one or two dimensions for comparison (to increase the redundancy), the other information has to stay attached to the element/event. This combining could be done outside the object, but then you need to transfer reliable BIGNUMS to the object, which was the reason for my posting in the first place (see, we are already getting recursive...)
Nevertheless the basic idea of the object is -as you suggest- to have a small but very efficient engine for storing numbers (now: arrays of floats) and retrieving them according to their markov properties of any order, and not so much to have an "all in one" object. For flexibility reasons there has to be some deal of functionality built in which accounts for a lot of messages which can be sent to it. But my goal was to design this object to be quite "pure" in respect to being small, modular and specific in its functionality (in a way like linux and pd ;-) ) as this is how I like things best.
much of what follows is my guesswork, hope it is not completely off target...
Not at all. You obviously got quite a good picture of it and I'm really grateful for the comments as it helps me to view things from a different perspective. In a seperate email I will send the manual of the object to your private email address in case you are interested to get an even better idea and want to help with further suggestions. Anyone who is interested can drop me a note and I will send the manual around.
For all the others: Sorry for spamming this list with lengthy emails.
Gruezi, Orm
hi Orm,
Orm Finnendahl wrote: ...
altogether. But why did you suggest the linked list structure? I
You did (in a previous post):
time ago. It stores elements in two data structures: (1.) A table, implemented as linked list, which defines the succession of elements
The example of struct _event { struct _event *next; float data[1]; } meant using a continuous array (`data' field) as a multi-dimensional event value, while linking successive events. One would call (in Pd) getbytes(sizeof(struct _event) + (<ndimen> - 1) * sizeof(float)) to allocate an event. Of course the event sequence as a whole may also be stored in a large (possibly growable) array.
...
What I meant was this: say, after range checking and quantizing
we have 80 pitches times 80 velocity levels times 80 durations (or
deltas?) which gives 512000-element sample space. With an average
density of around 10 events per second, transition matrix would
still be nearly empty after few seconds. In such case there would
be no branches, and plain markov process would keep generating
exact imitation of live performance sequence for much longer then
few seconds -- unless previous knowledge was applied. But now
I see that your external is much more flexible than that (with
margin',
mutation', and various `modes'). Btw. such effect of
gradually deviating imitation may be interesting in its own right.
...
To store only one dimension is normally not sufficient to capture musical events. It gives a completely different result if you seperate
Yes, I was not precise -- I meant vector quantizing (representing similar events as equal), and _not_ projecting into one-dimensional space. Since deciding what is `similar' is not easy, and usually involves costly clustering techniques, this hint was misleading.
Krzysztof