Hello everyone,
Would someone be able to explain to me why d_soundfile.c has the following buffer size allocation?
#define MAXBUFSIZE 16777216 /* arbitrary; just don't want to hang malloc */
I'm not sure about the 'hanging malloc' thing...does it mean that enlarging that buffer size could screw with memory allocation because it we might 'run out'?
AFAICT, 16777216 is about 6.3 minutes. If I need more like 26460000 (10m) allocation, could I just alter MAXBUFSIZE and recompile? Or is there a saner way of doing it, so that I don't end up with a 'custom' pd?
d
-- David Plans Casal Researcher, UEA Studios d.casal at uea dot ac dot uk http://www.davidcasal.com
On Tue, 28 Feb 2006, David Plans Casal wrote:
Maybe a better word than hanging would be flooding, as in taking too much RAM. But there is another reason not stated, so it's not that arbitrary.
16777216, which is 2^24, is the last contiguous integer representable in the float32 format. 16777217 (and following odd integers) get rounded to the closest even integer. starting at 2^25 you only get multiples of 4 (which is 2^2) and so on. note that 2^25/2^2 = 2^23 which is the degree of precision of float32.
This means that [tabread~] doesn't do what you want starting at 16777216; it also means that [tabread4~] does exactly the same as [tabread~] starting at 2^23=8388608, which is not what you want, and that appears gradually, because between 2^22 and 2^23 you only have steps of 0.5, between 2^21 and 2^22 you only have steps of 0.25, etc.
There are still some objects that can handle big arrays, so the limit should be configurable at run time... but it's not.
does it mean that enlarging that buffer size could screw with memory allocation because it we might 'run out'?
You're allowed to malloc something close to 2^30 bytes = 2^28 float32's in one chunk if you have enough RAM. In 64-bit mode you would be able to malloc more, but pd arrays don't work in 64-bit mode.
If Pd used float64 you wouldn't have problems with tabread~ because float64 contains all integers up to 2^53. However Pd doesn't support float64 yet, and once it does, you can't use float32 anymore, so all your arrays will take double the RAM. To support multiple number formats in signals would be quite a overhaul of Pd. To support them in arrays only, would be easier but still quite a bit of work. (But then you would be able to store your values in int16 format, which is half of the RAM of float32!)
AFAICT, 16777216 is about 6.3 minutes. If I need more like 26460000 (10m) allocation, could I just alter MAXBUFSIZE and recompile?
yes.
Or is there a saner way of doing it, so that I don't end up with a 'custom' pd?
no... well, do you have an hex editor? ;-)
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
On 28 Feb 2006, at 13:21, Georg Holzmann wrote:
Do you mean [soundfiler] ? I thought it could only read wav, aiff and nexstep?
I'm already using soundfiler, but the problems are PD's buffers themselves, or am I wrong...
d
-- David Plans Casal Researcher, UEA Studios d.casal at uea dot ac dot uk http://www.davidcasal.com
On 28 Feb 2006, at 13:51, Georg Holzmann wrote:
OH! _THere_ it is! :-)
Thanks Georg...
d
-- David Plans Casal Researcher, UEA Studios d.casal at uea dot ac dot uk http://www.davidcasal.com
On some OSes (I needn't name names:) the system can hang if one asks to malloc too much space. This was a major cause of headaches for people trying to use Pd to read large soundfiles. Anyway, as you already found, this doesn't affect the "soundfiler" object which has a settable maximum.
cheers Miller
On Tue, Feb 28, 2006 at 11:54:39AM +0000, David Plans Casal wrote:
On 28 Feb 2006, at 18:29, Miller Puckette wrote:
Thank you for that clarification. I understand the original limitation now.
However, I am wondering whether it wouldn't be better to pre-empt large soundfile loading, as opposed to making people who need to use large soundfile arrays use custom compilation?
Just a thought, I realise there's history at play here.
d
-- David Plans Casal Researcher, UEA Studios d.casal at uea dot ac dot uk http://www.davidcasal.com