Hello everyone,

Thanks Miller for the suggestions. I tried loading the parent patch with an empty abstraction in place of the one that is called 2197 times (as mentioned, this repeatedly called abstraction is very small, but has two inlets, two outlets, and four receive objects) and the parent patch loaded fine. I noticed while trying to load the original parent patch (with the repeated abstraction in place) with the -verbose flag, that the parent pd window scrolls as if it's loading many of the instances being called (trying and failing to find the .dll, finding the .pd) for about fifteen seconds before pd dies. I also noticed that after I rebuilt the parent patch, it dies at a different point (presumably because I put the objects on the screen in a different order than I did when I initially built the patch). To me, this suggests a memory problem, but, as I mentioned, my system resource meter doesn't show any problems. Could the resource meter just be reacting too slowly to show the dip in memory? Would pd run out of memory before my system actually does (I'm ignorant about how operating systems allocate memory to programs)? Any suggestions?

Thanks again, Greg NYU

Miller Puckette wrote:

Hi all,

Well, this is every software developer's nightmare...

To start with, I'd suggest trying to load the patch with dummies in place of the abstractions, and with the "-noloadbang" flag. There could be some limit I don't know about in Pd's ability to handle patches with huge numbers of subwindows (or any of a host of other possible problems..)

I do know that Pd is ungraceful when it runs out of memory; it's sometimes hard to find out when that happens...

cheers Miller

On Tue, Aug 21, 2001 at 02:35:58AM -0400, Greg Rippin wrote:

...

Hello all,

I have been working for some time on a patch that does a realtime harmonic analysis of an incoming MIDI stream and constructs probability tables for creating second-order markov chains based on the input. The patch is fairly large and complex; the probability calculating abstraction (fairly small in itself) is called 13x13x13 times.

Recently, pd has occasionally died when I try to open this patch, but I could get it to open after a few failed attempts. Now, quite suddenly, it won't open at all; pd generates an invalid page fault whenever I try to load this patch. It opens all other patches without trouble. Most puzzling to me is that I haven't actually made any changes to the patch; I added some comments to the abstractions that are called by the parent patch, but made no functional changes immediately before this started to happen. I thought that perhaps the file was corrupt; I rebuilt the patch from it's constituent parts, it worked perfectly. I saved it under a different filename, closed it, tried to reopen it, and got the same error message. I even checked my resource meters, and nothing (memory, etc.) seems to dip when trying to open this patch. I'm stumped.

What could I have done wrong to cause behavior like this? I'm running pd .33 patch 1 with the iemlib extensions on win 98. The error message I get is:

PD caused an invalid page fault in module PD.DLL at 015f:10046e98. Registers: EAX=00960008 CS=015f EIP=10046e98 EFLGS=00210293 EBX=00000150 SS=0167 ESP=0063da90 EBP=0063dab0 ECX=00962838 DS=0167 ESI=00000013 FS=0fbf EDX=0309f00c ES=0167 EDI=00000014 GS=0000 Bytes at CS:EIP: 8b 49 04 89 4a 04 8b 4d f0 8d 0c f9 89 4a 08 89 Stack dump: 03100df0 00000200 0309f050 00000000 00962798 00000150 0309f00c 00000040 00960138 10042d20 00960138 00000003 03100df0 0309f050 00000110 00000200

Any help is greatly appreciated, Greg

-- Greg Rippin, Graduate Assistant Music Technology Program New York University 35 West 4th Street, Suite 777 New York, NY 10012

phone 212-998-5422 fax 212-995-4043 musictechgrad.info@nyu.edu www.nyu.edu

-- Greg Rippin, Graduate Assistant Music Technology Program New York University 35 West 4th Street, Suite 777 New York, NY 10012

phone 212-998-5422 fax 212-995-4043 musictechgrad.info@nyu.edu www.nyu.edu

Hmm, since memory is getting allocated, but almost never freed, as a patch gets loaded, I don't think fragmentation is possible. Perhaps w98 has some per-process memory limit (like the old Mac OS?) I'm afraid even to guess. But I'll at least put an appropriate doomsday message for the next Pd release so it at least prints something out...

cheers Miller

On Wed, Aug 22, 2001 at 02:49:40PM +0200, Krzysztof Czaja wrote:

hi,

not really shure what you mean by _calling_ an abstraction. Reusing abstraction definition only (a .pd file), which is more like macro expansion (let me talk in a procedural way), or reusing program resources, as if calling a routine? The later would mean passing a message to the _same_ place in a patch. But the same inlet or receive as created in an abstraction definition is _not_ the same place if used in two different instances of this abstraction.

The -verbose printout suggests all 2197 copies are created (either dynamically with 'obj' message, or statically while reading objects from a parent patchfile). In Pd you always get a separate copy of each abstraction instance, thus the full object creation is repeated 2197 times -- all steps, including loading from a file and allocating memory.

Each copy of an empty abstraction takes a little over 600 bytes, and 600*13**3 == 1.3Mb. Putting a simple object, like inlet or receive, into a patch increases the per-instance amount by about 100 bytes (connections take much less). Assuming your abstraction has 8 i/o objects and ~10 simple objects doing the real job, this abstraction uses ~4 times the empty abstraction memory size. But still 5Mb is nothing nowadays, so the only possible reason I know of is memory _fragmentation_.

Krzysztof

(ok, since I was only guessing, the above is not very helpful; perhaps you could attach this small abstraction -- not the multi-thousand-line main patch of course).

Greg Rippin wrote: ...

...

Thanks Miller for the suggestions. I tried loading the parent patch with an empty abstraction in place of the one that is called 2197 times (as mentioned, this repeatedly called abstraction is very small, but has two inlets, two outlets, and four receive objects) and the parent patch loaded fine. I noticed while trying to load the original parent patch (with the repeated abstraction in place) with the -verbose flag, that the parent pd window scrolls as if it's loading many of the instances being called (trying and failing to find the .dll, finding the .pd) for about fifteen seconds before pd dies.

...

Krzysztof and others,

Please excuse my confusion of the terminology. I'm a musician first and a programmer second; both my training and my experience with more complex computer matters are both lacking (which, I am sure, is not helping me solve this problem at all).

The abstract definition is reused, called repeatedly with different initialization arguments each time. I supposed the correct term would therefore be "instances". As I said, each instance has two inlets, two outlets, and two [r ] objects. One of the [r ] objects is the same in each instance (which receives a global reset command). The other [r ] is the same in every group of 13 instances, as defined by the initialization arguments.

I've attached a copy of the abstraction in question. I'm only (only!?) running 256 Mb, and I'm starting to think that this may be the problem. Incidentally, the resource monitor I've been using is the one that came with Windows 98. Can anyone suggest a better utility for this?

Here's the detail that's been puzzling me throughout the day: if I rebuild the patch, it works just fine. It's only when I save it, close it, and try to reopen it that the problems occur.

Thanks all for your help! Greg

Krzysztof Czaja wrote:

hi,

not really shure what you mean by _calling_ an abstraction. Reusing abstraction definition only (a .pd file), which is more like macro expansion (let me talk in a procedural way), or reusing program resources, as if calling a routine? The later would mean passing a message to the _same_ place in a patch. But the same inlet or receive as created in an abstraction definition is _not_ the same place if used in two different instances of this abstraction.

The -verbose printout suggests all 2197 copies are created (either dynamically with 'obj' message, or statically while reading objects from a parent patchfile). In Pd you always get a separate copy of each abstraction instance, thus the full object creation is repeated 2197 times -- all steps, including loading from a file and allocating memory.

Each copy of an empty abstraction takes a little over 600 bytes, and 600*13**3 == 1.3Mb. Putting a simple object, like inlet or receive, into a patch increases the per-instance amount by about 100 bytes (connections take much less). Assuming your abstraction has 8 i/o objects and ~10 simple objects doing the real job, this abstraction uses ~4 times the empty abstraction memory size. But still 5Mb is nothing nowadays, so the only possible reason I know of is memory _fragmentation_.

Krzysztof

(ok, since I was only guessing, the above is not very helpful; perhaps you could attach this small abstraction -- not the multi-thousand-line main patch of course).

Greg Rippin wrote: ...

...

Thanks Miller for the suggestions. I tried loading the parent patch with an empty abstraction in place of the one that is called 2197 times (as mentioned, this repeatedly called abstraction is very small, but has two inlets, two outlets, and four receive objects) and the parent patch loaded fine. I noticed while trying to load the original parent patch (with the repeated abstraction in place) with the -verbose flag, that the parent pd window scrolls as if it's loading many of the instances being called (trying and failing to find the .dll, finding the .pd) for about fifteen seconds before pd dies.

...

-- Greg Rippin, Graduate Assistant Music Technology Program New York University 35 West 4th Street, Suite 777 New York, NY 10012

phone 212-998-5422 fax 212-995-4043 musictechgrad.info@nyu.edu www.nyu.edu

#N canvas -46 14 782 534 12; #X obj 258 370 /; #X obj -112 271 r reset; #X obj 32 173 inlet; #X obj 13 504 outlet; #X obj 127 504 outlet; #X obj 56 405 f 0; #X msg 262 443 bang; #X obj 269 329 r $1total; #X obj 166 451 * 100; #X obj 121 406 f 0; #X obj -112 301 delay 50; #X obj 177 284 + 1; #X obj 115 284 float 0; #X obj -111 330 f -1; #X obj 371 175 inlet; #X obj 373 227 pipe 50; #X obj 373 203 - 1; #X text -69 -126 inputs a bang representing a categorized event (specific harmonic sequence); #X text -73 -88 outputs total number of events that match that category ; #X text -71 -75 outputs probability of that category of event occuring ; #X text -72 -57 e.g. if the analysis algorithm sees a two-five-one progression , a bang is sent to the "one" probcalc module inside the "fiveone" harmanalmod. This probcalc keeps a running total of bangs it recieves. It also inputs the total number of two-five-xxx sequences that have occured. It divides the total number by the local count , yielding the probability that , when the sequence two-five occurs , the next interval will be a one.; #X text 75 -151 probcalc abstraction; #X text -161 234 reset function; #X text -189 247 resets counter to zero; #X text 10 147 bang input; #X text 126 259 counter; #X text 327 122 load function; #X text 249 137 sets counter to number from table; #X text 251 149 stored in text file; #X text 231 302 total input from parent harmonanalmod; #X text -25 525 number counted; #X text 113 524 probability; #X connect 0 0 8 0; #X connect 1 0 10 0; #X connect 2 0 12 0; #X connect 5 0 3 0; #X connect 6 0 9 0; #X connect 7 0 0 1; #X connect 7 0 6 0; #X connect 8 0 4 0; #X connect 9 0 0 0; #X connect 10 0 13 0; #X connect 11 0 5 0; #X connect 11 0 9 0; #X connect 11 0 12 1; #X connect 12 0 11 0; #X connect 13 0 12 0; #X connect 14 0 16 0; #X connect 15 0 12 0; #X connect 16 0 15 0;

Here's the detail that's been puzzling me throughout the day: if I rebuild

the

patch, it works just fine. It's only when I save it, close it, and try to

reopen

it that the problems occur.

This is similar behaviour to the segmentation fault error I reported to the list, that happens to me when reopening patches that contain arrays with the 'save contents' flag checked in the same pd session. They load correctly again when restarting pd though.

Hello all,

Thank you all very much for all of your help over the last few days. I finally managed to get my patch to load without dying. . .without your help, I wouldn't have had any idea of where to start looking for a solution. Thanks!

I checked all of my arrays, thinking that I might have left the "save contents" flag checked on some or all of them. Not so.

In the end, I went back and removed most of the comments from all of the subpatches, and thing loaded like magic. In hindsight, I guess I should have done this earlier, since adding comments was one of the last things I did before this started happening. I suppose that I was thinking of comments as being pretty benign, irrelevant portions of a patch, and that made me overlook them as possible suspects. What could I have put in a comment to make it kill the entire patch? I was pretty careful to avoid placing $'s in them. . .

I suppose I should reexamine and try to streamline my patches a bit more (thanks all for the helpful example patches). I'll also try to track down any uncontrolled loops that I may have missed.

Thanks again, everyone. (By the way, the name harmonanal.pd stands for harmonic analysis, despite whatever else it suggests...)

Greg NYU