Hi, when compiling ELSE for camomille in windows, me and Esteban are getting some errors. Offending pieces of code are when trying to do things like
t_atom at[ac];
and
t_float temp[n];
from stackflow https://stackoverflow.com/questions/9881777/why-do-i-get-cannot-allocate-an-array-of-constant-size-0 we see "*You cannot allocate an array of unknown size with automatic storage duration in C++. If you want a variable sized array then you need to dynamically allocate it (or, better yet; just use a vector).*"
Thing is we don't know how to proceed here and are looking for hints, maybe showing how other objects in Pd deal with this. BTW, there's an issue https://github.com/porres/pd-else/issues/881 on ELSE's repository.
Thanks Alex
On Thu, 3 Dec 2020 at 23:15, Alexandre Torres Porres porres@gmail.com wrote:
Hi, when compiling ELSE for camomille in windows, me and Esteban are getting some errors. Offending pieces of code are when trying to do things like
t_atom at[ac];
If you want to maintain straight C compiler compatibility
t_atom* at = (t_atom*)malloc(ac * sizeof(t_atom));
but you have to remember to free(at), &cet. You can avoid the free() if you HAVE_ALLOCA with
t_atom* at = (t_atom*)alloca(ac * sizeof(t_atom));
if you want to do it the C++ way without a std::vector<t_atom>
t_atom* at = new t_atom[ac];
but again you will have to
delete at;
For my own externals, I write them all in C++ and use STL. Making the change from the C-world allocation of PD to the C++ world is not so hard, but it does involve a tiny bit of trickery which I only justify through expediency.
- d
I'm using getbytes and freebytes for now, any disadvantages over alloca?
thanks!
Em qui., 3 de dez. de 2020 às 20:59, David Rush kumoyuki@gmail.com escreveu:
On Thu, 3 Dec 2020 at 23:15, Alexandre Torres Porres porres@gmail.com wrote:
Hi, when compiling ELSE for camomille in windows, me and Esteban are getting some errors. Offending pieces of code are when trying to do things like
t_atom at[ac];
If you want to maintain straight C compiler compatibility
t_atom* at = (t_atom*)malloc(ac * sizeof(t_atom));but you have to remember to free(at), &cet. You can avoid the free() if you HAVE_ALLOCA with
t_atom* at = (t_atom*)alloca(ac * sizeof(t_atom));if you want to do it the C++ way without a std::vector<t_atom>
t_atom* at = new t_atom[ac];but again you will have to
delete at;For my own externals, I write them all in C++ and use STL. Making the change from the C-world allocation of PD to the C++ world is not so hard, but it does involve a tiny bit of trickery which I only justify through expediency.
- d
alloca() "allocates" memory on the stack. This is done by simply incrementing the stack pointer. So it's extremely fast and - more importantly - equally fast for all sizes.
malloc(), on the other hand, actually uses the system memory allocator which can take arbitrarily long and might even block!
Generally, you should avoid using any malloc() in real-time code paths. Instead, pre-allocate temporary buffers (e.g. in the "dsp" method) or allocate on the stack (but note the caveats mentioned in the other mails).
Christof
On 04.12.2020 03:28, Alexandre Torres Porres wrote:
I'm using getbytes and freebytes for now, any disadvantages over alloca?
thanks!
Em qui., 3 de dez. de 2020 às 20:59, David Rush <kumoyuki@gmail.com mailto:kumoyuki@gmail.com> escreveu:
On Thu, 3 Dec 2020 at 23:15, Alexandre Torres Porres <porres@gmail.com <mailto:porres@gmail.com>> wrote: Hi, when compiling ELSE for camomille in windows, me and Esteban are getting some errors. Offending pieces of code are when trying to do things like t_atom at[ac]; If you want to maintain straight C compiler compatibility t_atom* at = (t_atom*)malloc(ac * sizeof(t_atom)); but you have to remember to free(at), &cet. You can avoid the free() if you HAVE_ALLOCA with t_atom* at = (t_atom*)alloca(ac * sizeof(t_atom)); if you want to do it the C++ way without a std::vector<t_atom> t_atom* at = new t_atom[ac]; but again you will have to delete at; For my own externals, I write them all in C++ and use STL. Making the change from the C-world allocation of PD to the C++ world is not so hard, but it does involve a tiny bit of trickery which I only justify through expediency. - d
Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
well, as for the atom size in the GUI objects I'm using getbytes/freebytes and I guess that's the way to go.
as for the other issue, it's for a median~ external that takes medians and we're now using malloc/realloc for a temp t_float variable that gets the block size value inside the "dsp" function. see => https://github.com/porres/pd-else/blob/master/Classes/Source/median~.c#L78
I use this to get the median of bits of a signal block from FFT amplitudes, so, yeah, block sizes can be big but not too big.
anyway, seems to be working fine but if you have suggestions I'm all ears.
thanks a lot
Em sex., 4 de dez. de 2020 às 11:43, Christof Ressi info@christofressi.com escreveu:
alloca() "allocates" memory on the stack. This is done by simply incrementing the stack pointer. So it's extremely fast and - more importantly - equally fast for all sizes.
malloc(), on the other hand, actually uses the system memory allocator which can take arbitrarily long and might even block!
Generally, you should avoid using any malloc() in real-time code paths. Instead, pre-allocate temporary buffers (e.g. in the "dsp" method) or allocate on the stack (but note the caveats mentioned in the other mails).
Christof On 04.12.2020 03:28, Alexandre Torres Porres wrote:
I'm using getbytes and freebytes for now, any disadvantages over alloca?
thanks!
Em qui., 3 de dez. de 2020 às 20:59, David Rush kumoyuki@gmail.com escreveu:
On Thu, 3 Dec 2020 at 23:15, Alexandre Torres Porres porres@gmail.com wrote:
Hi, when compiling ELSE for camomille in windows, me and Esteban are getting some errors. Offending pieces of code are when trying to do things like
t_atom at[ac];
If you want to maintain straight C compiler compatibility
t_atom* at = (t_atom*)malloc(ac * sizeof(t_atom));but you have to remember to free(at), &cet. You can avoid the free() if you HAVE_ALLOCA with
t_atom* at = (t_atom*)alloca(ac * sizeof(t_atom));if you want to do it the C++ way without a std::vector<t_atom>
t_atom* at = new t_atom[ac];but again you will have to
delete at;For my own externals, I write them all in C++ and use STL. Making the change from the C-world allocation of PD to the C++ world is not so hard, but it does involve a tiny bit of trickery which I only justify through expediency.
- d
Pd-dev mailing listPd-dev@lists.iem.athttps://lists.puredata.info/listinfo/pd-dev
Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
On Friday, December 4, 2020, 9:43:20 AM EST, Christof Ressi info@christofressi.com wrote: alloca() "allocates" memory on the stack. This is done by simply incrementing the stack pointer. So it's extremely fast and - more importantly - equally fast for all sizes.
It also requires you, the programmer, to add up the total number of possible allocations in a recursive call to the object, for supported platforms with small default stack sizes, and doing the math in your head to ensure your algorithm will never go over that stack limit, for all possible cases. Since that stack limit is many orders of magnitude smaller than the RAM on the most popular Pd platform, you're way more likely to accidentally cause crashes for your users by relying on alloca.
Again, the ATOMS_ALLOCA macro in x_list.c is the careful, thoughtful reference for use of alloca. And even in that case there are almost certainly multiple ways to cause a crasher from it. In other words, it's nearly impossible to use alloca safely. Don't use alloca *unless* you have made worst case measurements on every other algorithm you can think of, and none of them are satisfactory. Even then, *measure* alloca to be worst-case safe and write regression tests for the recursive edge cases that could blow the stack. Chances are when you consider doing that extra work, you'll quickly think up a different algorithm that doesn't rely on alloca.
malloc(), on the other hand, actually uses the system memory allocator which can take arbitrarily long and might even block!
Generally, you should avoid using any malloc() in real-time code paths. Instead, pre-allocate temporary buffers (e.g. in the "dsp" method) or allocate on the stack (but note the caveats mentioned in the other mails).
Pre-allocate. If you can't, ask on the list how to use alloca without blowing the stack. Once you crowdsource a truly safe algorithm, write tests so you catch the crashers that the crowd missed the first time around.
To be clear-- I basically grepped for "alloca" in the current codebase, opened up x_list.c, and *assumed* because alloca is tricky that there is somehow a crasher bug. It took about 5 minutes to come up with a case that should crash on Windows. I also see it in m_binbuf.c, and I'd make the same bet it can blow the Windows stack if someone spends five minutes looking at the code. For a common building block of realtime safe algos, I shouldn't be able to make claims like these for any use of alloca I happen to find. I keep harping on this because the default description of alloca makes it sound like the quintessential building block of realtime algos. Please weigh that alluring set of seeming realtime safe benefits against the history of realtime unsafe crashers of which will likely include your use alloca.
Best, Jonathan
Christof
On 04.12.2020 03:28, Alexandre Torres Porres wrote:
I'm using getbytes and freebytes for now, any disadvantages over alloca? thanks! Em qui., 3 de dez. de 2020 às 20:59, David Rush kumoyuki@gmail.com escreveu:
On Thu, 3 Dec 2020 at 23:15, Alexandre Torres Porres porres@gmail.com wrote:
Hi, when compiling ELSE for camomille in windows, me and Esteban are getting some errors. Offending pieces of code are when trying to do things like t_atom at[ac];
If you want to maintain straight C compiler compatibility t_atom* at = (t_atom*)malloc(ac * sizeof(t_atom)); but you have to remember to free(at), &cet. You can avoid the free() if you HAVE_ALLOCA with t_atom* at = (t_atom*)alloca(ac * sizeof(t_atom)); if you want to do it the C++ way without a std::vector<t_atom> t_atom* at = new t_atom[ac]; but again you will have to delete at; For my own externals, I write them all in C++ and use STL. Making the change from the C-world allocation of PD to the C++ world is not so hard, but it does involve a tiny bit of trickery which I only justify through expediency. - d
_______________________________________________ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev _______________________________________________ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
Your concerns are certainly warranted.
But if the function is
a) never called recursively,
b) is never inlined(!)* and
c) the buffer size is guaranteed not to exceed some reasonable size for stack allocation (say a few hundred bytes),
then alloca() is IMO the best tool for the job. It is not only simpler but also faster than pre-allocation, because the stack memory is much more likely to be in cache.
Again, I wouldn't recommend using alloca() for general purpose programming, but when used cautiously it can be a great tool for real-time code.
---
BTW, here's an example for how alloca() might be used in a production-grade library: https://github.com/gcp/opus/blob/master/celt/stack_alloc.h
It uses alloca() resp. VLAs, but can fallback to a pseudo stack for platforms without alloca()/VLA support or small stack sizes (e.g. embedded devices).
Christof
*) IIRC, GCC refuses to inline functions if they contain alloca() statements. What can go wrong with inlined functions containing alloca(): https://stackoverflow.com/a/3410689/6063908
On 04.12.2020 16:57, Jonathan Wilkes wrote:
On Friday, December 4, 2020, 9:43:20 AM EST, Christof Ressi
info@christofressi.com wrote:
alloca() "allocates" memory on the stack. This is done by simply
incrementing the stack pointer. So it's extremely fast and - more importantly - equally fast for all sizes.
It also requires you, the programmer, to add up the total number of possible allocations in a recursive call to the object, for supported platforms with small default stack sizes, and doing the math in your head to ensure your algorithm will never go over that stack limit, for all possible cases. Since that stack limit is many orders of magnitude smaller than the RAM on the most popular Pd platform, you're way more likely to accidentally cause crashes for your users by relying on alloca.
Again, the ATOMS_ALLOCA macro in x_list.c is the careful, thoughtful reference for use of alloca. And even in that case there are almost certainly multiple ways to cause a crasher from it. In other words, it's nearly impossible to use alloca safely.
Don't use alloca *unless* you have made worst case measurements on every other algorithm you can think of, and none of them are satisfactory. Even then, *measure* alloca to be worst-case safe and write regression tests for the recursive edge cases that could blow the stack. Chances are when you consider doing that extra work, you'll quickly think up a different algorithm that doesn't rely on alloca.
malloc(), on the other hand, actually uses the system memory
allocator which can take arbitrarily long and might even block!
Generally, you should avoid using any malloc() in real-time code
paths. Instead, pre-allocate temporary buffers (e.g. in the "dsp" method) or allocate on the stack (but note the caveats mentioned in the other mails).
Pre-allocate.
If you can't, ask on the list how to use alloca without blowing the stack. Once you crowdsource a truly safe algorithm, write tests so you catch the crashers that the crowd missed the first time around.
To be clear-- I basically grepped for "alloca" in the current codebase, opened up x_list.c, and *assumed* because alloca is tricky that there is somehow a crasher bug. It took about 5 minutes to come up with a case that should crash on Windows.
I also see it in m_binbuf.c, and I'd make the same bet it can blow the Windows stack if someone spends five minutes looking at the code. For a common building block of realtime safe algos, I shouldn't be able to make claims like these for any use of alloca I happen to find.
I keep harping on this because the default description of alloca makes it sound like the quintessential building block of realtime algos. Please weigh that alluring set of seeming realtime safe benefits against the history of realtime unsafe crashers of which will likely include your use alloca.
Best, Jonathan
Christof
On 04.12.2020 03:28, Alexandre Torres Porres wrote: I'm using getbytes and freebytes for now, any disadvantages over alloca?
thanks!
Em qui., 3 de dez. de 2020 às 20:59, David Rush <kumoyuki@gmail.com mailto:kumoyuki@gmail.com> escreveu:
On Thu, 3 Dec 2020 at 23:15, Alexandre Torres Porres <porres@gmail.com <mailto:porres@gmail.com>> wrote: Hi, when compiling ELSE for camomille in windows, me and Esteban are getting some errors. Offending pieces of code are when trying to do things like t_atom at[ac]; If you want to maintain straight C compiler compatibility t_atom* at = (t_atom*)malloc(ac * sizeof(t_atom)); but you have to remember to free(at), &cet. You can avoid the free() if you HAVE_ALLOCA with t_atom* at = (t_atom*)alloca(ac * sizeof(t_atom)); if you want to do it the C++ way without a std::vector<t_atom> t_atom* at = new t_atom[ac]; but again you will have to delete at; For my own externals, I write them all in C++ and use STL. Making the change from the C-world allocation of PD to the C++ world is not so hard, but it does involve a tiny bit of trickery which I only justify through expediency. - d
Pd-dev mailing list Pd-dev@lists.iem.at mailto:Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev https://lists.puredata.info/listinfo/pd-dev _______________________________________________ Pd-dev mailing list Pd-dev@lists.iem.at mailto:Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev https://lists.puredata.info/listinfo/pd-dev
I forget to mention another important caveat, which is related to b):
d) don't ever call alloca() *inside* a loop
Alloca() is *not* block scoped! The stack is only restored on the next function return, so by calling alloca() in a loop you can easily create a stack overflow by accident.
VLAs, on the other hand, are block scoped, so they can be safely used in a loop. Unfortunately, they are not part of the most recent C or C++ standard. C99 made VLAs part of the language, but C11 made it optional again. C++ never included VLAs, but the feature has existed in the form of compiler extensions.
---
Maybe I can sum it up with:
Don't use alloca() unless you have a good reason, know how it works internally and are aware of all its problems :-).
Christof
On 05.12.2020 12:54, Christof Ressi wrote:
Your concerns are certainly warranted.
But if the function is
a) never called recursively,
b) is never inlined(!)* and
c) the buffer size is guaranteed not to exceed some reasonable size for stack allocation (say a few hundred bytes),
then alloca() is IMO the best tool for the job. It is not only simpler but also faster than pre-allocation, because the stack memory is much more likely to be in cache.
Again, I wouldn't recommend using alloca() for general purpose programming, but when used cautiously it can be a great tool for real-time code.
BTW, here's an example for how alloca() might be used in a production-grade library: https://github.com/gcp/opus/blob/master/celt/stack_alloc.h
It uses alloca() resp. VLAs, but can fallback to a pseudo stack for platforms without alloca()/VLA support or small stack sizes (e.g. embedded devices).
Christof
*) IIRC, GCC refuses to inline functions if they contain alloca() statements. What can go wrong with inlined functions containing alloca(): https://stackoverflow.com/a/3410689/6063908
On 04.12.2020 16:57, Jonathan Wilkes wrote:
On Friday, December 4, 2020, 9:43:20 AM EST, Christof Ressi
info@christofressi.com wrote:
alloca() "allocates" memory on the stack. This is done by simply
incrementing the stack pointer. So it's extremely fast and - more importantly - equally fast for all sizes.
It also requires you, the programmer, to add up the total number of possible allocations in a recursive call to the object, for supported platforms with small default stack sizes, and doing the math in your head to ensure your algorithm will never go over that stack limit, for all possible cases. Since that stack limit is many orders of magnitude smaller than the RAM on the most popular Pd platform, you're way more likely to accidentally cause crashes for your users by relying on alloca.
Again, the ATOMS_ALLOCA macro in x_list.c is the careful, thoughtful reference for use of alloca. And even in that case there are almost certainly multiple ways to cause a crasher from it. In other words, it's nearly impossible to use alloca safely.
Don't use alloca *unless* you have made worst case measurements on every other algorithm you can think of, and none of them are satisfactory. Even then, *measure* alloca to be worst-case safe and write regression tests for the recursive edge cases that could blow the stack. Chances are when you consider doing that extra work, you'll quickly think up a different algorithm that doesn't rely on alloca.
malloc(), on the other hand, actually uses the system memory
allocator which can take arbitrarily long and might even block!
Generally, you should avoid using any malloc() in real-time code
paths. Instead, pre-allocate temporary buffers (e.g. in the "dsp" method) or allocate on the stack (but note the caveats mentioned in the other mails).
Pre-allocate.
If you can't, ask on the list how to use alloca without blowing the stack. Once you crowdsource a truly safe algorithm, write tests so you catch the crashers that the crowd missed the first time around.
To be clear-- I basically grepped for "alloca" in the current codebase, opened up x_list.c, and *assumed* because alloca is tricky that there is somehow a crasher bug. It took about 5 minutes to come up with a case that should crash on Windows.
I also see it in m_binbuf.c, and I'd make the same bet it can blow the Windows stack if someone spends five minutes looking at the code. For a common building block of realtime safe algos, I shouldn't be able to make claims like these for any use of alloca I happen to find.
I keep harping on this because the default description of alloca makes it sound like the quintessential building block of realtime algos. Please weigh that alluring set of seeming realtime safe benefits against the history of realtime unsafe crashers of which will likely include your use alloca.
Best, Jonathan
Christof
On 04.12.2020 03:28, Alexandre Torres Porres wrote: I'm using getbytes and freebytes for now, any disadvantages over alloca?
thanks!
Em qui., 3 de dez. de 2020 às 20:59, David Rush <kumoyuki@gmail.com mailto:kumoyuki@gmail.com> escreveu:
On Thu, 3 Dec 2020 at 23:15, Alexandre Torres Porres <porres@gmail.com <mailto:porres@gmail.com>> wrote: Hi, when compiling ELSE for camomille in windows, me and Esteban are getting some errors. Offending pieces of code are when trying to do things like t_atom at[ac]; If you want to maintain straight C compiler compatibility t_atom* at = (t_atom*)malloc(ac * sizeof(t_atom)); but you have to remember to free(at), &cet. You can avoid the free() if you HAVE_ALLOCA with t_atom* at = (t_atom*)alloca(ac * sizeof(t_atom)); if you want to do it the C++ way without a std::vector<t_atom> t_atom* at = new t_atom[ac]; but again you will have to delete at; For my own externals, I write them all in C++ and use STL. Making the change from the C-world allocation of PD to the C++ world is not so hard, but it does involve a tiny bit of trickery which I only justify through expediency. - d
Pd-dev mailing list Pd-dev@lists.iem.at mailto:Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev https://lists.puredata.info/listinfo/pd-dev _______________________________________________ Pd-dev mailing list Pd-dev@lists.iem.at mailto:Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev https://lists.puredata.info/listinfo/pd-dev
Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
Don't use alloca() unless you have a good reason, know how it works internally and are aware of all its problems :-).
I was thinking of a good Pd inside joke: never optimize anything: "When interacting with your patch, Pd is designed to walk linked lists as much as possible. The spikes that result train the patch author to design the program with maximum audio processing efficiency, squeezing every last bit of performance out of the little CPU time that remains for the user." :)
Best, Jonathan
Christof
On 05.12.2020 12:54, Christof Ressi wrote:
Your concerns are certainly warranted.
But if the function is
a) never called recursively,
b) is never inlined(!)* and
c) the buffer size is guaranteed not to exceed some reasonable size for stack allocation (say a few hundred bytes),
then alloca() is IMO the best tool for the job. It is not only simpler but also faster than pre-allocation, because the stack memory is much more likely to be in cache.
Again, I wouldn't recommend using alloca() for general purpose programming, but when used cautiously it can be a great tool for real-time code.
---
BTW, here's an example for how alloca() might be used in a production-grade library: https://github.com/gcp/opus/blob/master/celt/stack_alloc.h
It uses alloca() resp. VLAs, but can fallback to a pseudo stack for platforms without alloca()/VLA support or small stack sizes (e.g. embedded devices).
Christof
*) IIRC, GCC refuses to inline functions if they contain alloca() statements. What can go wrong with inlined functions containing alloca(): https://stackoverflow.com/a/3410689/6063908
On 04.12.2020 16:57, Jonathan Wilkes wrote:
On Friday, December 4, 2020, 9:43:20 AM EST, Christof Ressi info@christofressi.com wrote: alloca() "allocates" memory on the stack. This is done by simply incrementing the stack pointer. So it's extremely fast and - more importantly - equally fast for all sizes.
It also requires you, the programmer, to add up the total number of possible allocations in a recursive call to the object, for supported platforms with small default stack sizes, and doing the math in your head to ensure your algorithm will never go over that stack limit, for all possible cases. Since that stack limit is many orders of magnitude smaller than the RAM on the most popular Pd platform, you're way more likely to accidentally cause crashes for your users by relying on alloca.
Again, the ATOMS_ALLOCA macro in x_list.c is the careful, thoughtful reference for use of alloca. And even in that case there are almost certainly multiple ways to cause a crasher from it. In other words, it's nearly impossible to use alloca safely. Don't use alloca *unless* you have made worst case measurements on every other algorithm you can think of, and none of them are satisfactory. Even then, *measure* alloca to be worst-case safe and write regression tests for the recursive edge cases that could blow the stack. Chances are when you consider doing that extra work, you'll quickly think up a different algorithm that doesn't rely on alloca.
malloc(), on the other hand, actually uses the system memory allocator which can take arbitrarily long and might even block!
Generally, you should avoid using any malloc() in real-time code paths. Instead, pre-allocate temporary buffers (e.g. in the "dsp" method) or allocate on the stack (but note the caveats mentioned in the other mails).
Pre-allocate. If you can't, ask on the list how to use alloca without blowing the stack. Once you crowdsource a truly safe algorithm, write tests so you catch the crashers that the crowd missed the first time around.
To be clear-- I basically grepped for "alloca" in the current codebase, opened up x_list.c, and *assumed* because alloca is tricky that there is somehow a crasher bug. It took about 5 minutes to come up with a case that should crash on Windows. I also see it in m_binbuf.c, and I'd make the same bet it can blow the Windows stack if someone spends five minutes looking at the code. For a common building block of realtime safe algos, I shouldn't be able to make claims like these for any use of alloca I happen to find. I keep harping on this because the default description of alloca makes it sound like the quintessential building block of realtime algos. Please weigh that alluring set of seeming realtime safe benefits against the history of realtime unsafe crashers of which will likely include your use alloca.
Best, Jonathan
Christof
On 04.12.2020 03:28, Alexandre Torres Porres wrote:
I'm using getbytes and freebytes for now, any disadvantages over alloca? thanks! Em qui., 3 de dez. de 2020 às 20:59, David Rush kumoyuki@gmail.com escreveu:
On Thu, 3 Dec 2020 at 23:15, Alexandre Torres Porres porres@gmail.com wrote:
Hi, when compiling ELSE for camomille in windows, me and Esteban are getting some errors. Offending pieces of code are when trying to do things like t_atom at[ac];
If you want to maintain straight C compiler compatibility t_atom* at = (t_atom*)malloc(ac * sizeof(t_atom)); but you have to remember to free(at), &cet. You can avoid the free() if you HAVE_ALLOCA with t_atom* at = (t_atom*)alloca(ac * sizeof(t_atom)); if you want to do it the C++ way without a std::vector<t_atom> t_atom* at = new t_atom[ac]; but again you will have to delete at; For my own externals, I write them all in C++ and use STL. Making the change from the C-world allocation of PD to the C++ world is not so hard, but it does involve a tiny bit of trickery which I only justify through expediency. - d
_______________________________________________ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev _______________________________________________ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
_______________________________________________ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev _______________________________________________ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
then alloca() is IMO the best tool for the job. It is not only simpler but also faster than pre-allocation, because the stack memory is much more likely to be in cache.
I'm not sure that's quite constrained enough yet. Not only do we want to avoid an arbitrary user-controlled upper bound, we don't want future devs adding the "feature" of an arbitrary user-controlled upper bound, or even a new upper bound that isn't user-controlled but is still too high given what you and I know about alloca.
So unless those constraints are documented in the code, this is still asking for trouble. And history of trouble is scattered throughout the usage history of alloca in Pd and elsewhere. I'm willing to concede that "Don't use alloca," is a bit like, "eval is evil." But given the proliferation of wrong uses of alloca I believe it's warranted. At least here we've gone from avoiding a call to free to a list of very niche constraints.
Best, Jonathan
You can avoid the free() if you HAVE_ALLOCA with t_atom* at = (t_atom*)alloca(ac * sizeof(t_atom));
If the user can control ac, then you have likely introduced a potential crasher just to avoid a call to free. Systemd had a cve from using alloca awhile back. Pd recently fixed a crasher due to alloca. In neither case was alloca necessary. I'd strongly advise to avoid it in general.
In fact, the next time I get on a Windows machine I bet I can create a trivial Pd patch to blow the stack using only `[list prepend]` and `[list fromsymbol]`. If so it will be 100% due to alloca, and an ostensibly defensive use of it at that. (Have a look at the code for the ATOMS_ALLOCA macro if someone wants to beat me to it.)
Best, Jonathan
I'd strongly advise to avoid it in general.
For general purpose code I would agree, but for realtime safe code we want to avoid dynamic memory allocation and careful use of alloca() is perfectly fine. But there are caveats.
In fact, we had a crasher in FFT objects which used temporary buffers on the stack. Users tried to do a 100 000 point FFT and it would crash on Windows. There are at least threee ways to solve this problem:
a) conditionally switch to malloc() beyond a certain size (don't forget to free the buffer :-)
b) use a pre-allocated buffer in the object
c) use thread local storage
In fact, the next time I get on a Windows machine I bet I can create a trivial Pd patch to blow the stack using only `[list prepend]` and `[list fromsymbol]`.
To elaborate: The limit for ATOM_ALLOCA is 100 and the stack limit is 1000. On a 64-bit system, t_atom takes 16 bytes (including padding). This means we could have up to 1,600,000 bytes on the stack before Pd's stack protection kicks in. The default stack size on Windows is 1 MB, so it would blow up.
Christof
On 04.12.2020 07:30, Jonathan Wilkes via Pd-dev wrote:
You can avoid the free() if you HAVE_ALLOCA with
t_atom* at = (t_atom*)alloca(ac * sizeof(t_atom));
If the user can control ac, then you have likely introduced a potential crasher just to avoid a call to free.
Systemd had a cve from using alloca awhile back. Pd recently fixed a crasher due to alloca. In neither case was alloca necessary.
I'd strongly advise to avoid it in general.
In fact, the next time I get on a Windows machine I bet I can create a trivial Pd patch to blow the stack using only `[list prepend]` and `[list fromsymbol]`. If so it will be 100% due to alloca, and an ostensibly defensive use of it at that. (Have a look at the code for the ATOMS_ALLOCA macro if someone wants to beat me to it.)
Best, Jonathan
Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
In fact, we had a crasher in FFT objects which used temporary buffers on the stack. Users tried to do a 100 000 point FFT and it would crash on Windows.
That's interesting. Is this a case where the algorithm requires allocating for 100,000 points *inside* the perform routine? -Jonathan
Hi,
On 03/12/2020 23:58, David Rush wrote:
if you want to do it the C++ way without a std::vector<t_atom>
t_atom* at = new t_atom[ac];
but again you will have to
delete at;
This should be delete[] at;
Claude
Off topic.
Since C++11, using raw "new" is a code smell, because it easily causes problems as the one below. Instead use smart pointers, which will automatically free the memory once they go out of scope:
auto at = std::unique_ptr<t_atom[]>(new t_atom[c]);
Or even better, but requires C++14:
auto at = std::make_unique<t_atom[]>(ac);
In both cases, note the square brackets!
Christof
On 04.12.2020 12:40, Claude Heiland-Allen wrote:
Hi,
On 03/12/2020 23:58, David Rush wrote:
if you want to do it the C++ way without a std::vector<t_atom>
t_atom* at = new t_atom[ac];
but again you will have to
delete at;
This should be delete[] at;
Claude
-
Alexandre Torres Porres -
Christof Ressi -
Claude Heiland-Allen -
David Rush -
Jonathan Wilkes