OK, I've just spent several hours on this, testing upwards of 200,000 parallel abstractions and crashing my computer almost as many times. But I managed to figure it out.

(Most of) you will be relieved to hear that the problem does not originate with the PD core, but with externals, as IOhannes suggested. Alex will not be relieved to hear that the culprit is [else/dir]. It seems that the 1019th instance of this object crashes PD! How strange.

I'm filing a bug report on git and moving on with my life.

Liam


From: Pd-list <pd-list-bounces@lists.iem.at> on behalf of Liam Goodacre <liamg_uw@hotmail.com>
Sent: 22 May 2018 05:36
To: pd-list@lists.iem.at
Subject: Re: [PD] "too many open files" error in 0.48.1
 
Thank you for the input. It will be a few days before I can do any serious testing, but I can answer some of your questions now.

IOhannes: yes, I upgraded PD, my externals, and my OS, all at the same time. I'll try it out with various different versions of the same and let you know what happens. But just as a heads-up, it's quite possible that I am trying to load more than 10,000 abstractions at once. Quite difficult to count them though!

If neither the externals nor the OS turn out to be the problem, I'll try to come up with a simple test case and get back to you.
From: Pd-list <pd-list-bounces@lists.iem.at> on behalf of IOhannes m zmölnig <zmoelnig@iem.at>
Sent: 21 May 2018 20:50
To: pd-list@lists.iem.at
Subject: Re: [PD] "too many open files" error in 0.48.1
 
On 05/21/2018 09:37 PM, ub@xdv wrote:
> On 21.05.2018 20:54, Claude Heiland-Allen wrote:
>>
>>
>> On 21/05/18 19:35, ub@xdv wrote:
>>> hello,
>>>
>>> On 20.05.2018 06:50, Liam Goodacre wrote:
>>>> In 0.48.1 on Ubuntu, I'm getting a horrible scenario where PD refuses to
>>>> open patches or create any more abstractions for me. I get an error
>>>> message saying "too many open files". Granted I have a lot open, but
>>>> this is a serious problem as it means I can't access all of my old
>>>> performances. They worked fine in 0.47.
>>>>
>>>> Any ideas?
>>> this is not really a problem with pd
>>
>> I disagree. The most common cause of "too many open files" is a bug in
>> closing files properly,

intuitively i would agree.

> possible. never occured to me. is it in the issue tracker?

no (afaik). it hasn't been reported before.

>
>> because 1024 simultaneously-open files should be
>> enough for most use cases.definitely, but when someone says "Granted I have a lot open", that
> could be an understatement. or a regression of some sort.

so i did a test run, with some heavily embedded abstraction, that is
loaded a total of 10000 times.
$ ulimit -Sn
1024
$ valgrind --track-fds=yes pd -noprefs -oss -nosound -nomidi -nrt
-stderr -open test.pd
[...]
==29030== FILE DESCRIPTORS: 3 open at exit.
==29030== Open file descriptor 2: /dev/pts/12
==29030==    <inherited from parent>
==29030==
==29030== Open file descriptor 1: /dev/pts/12
==29030==    <inherited from parent>
==29030==
==29030== Open file descriptor 0: /dev/pts/12
==29030==    <inherited from parent>
==29030==
[...]
$

so there seems to be neither a temporary file-handle leak (because my
10000 abstractions load just fine) nor an absolute file-handle leak
(since the only handles that are left open by the program are stdin,
stdout, stderr)


so i guess there must be something wrong with the patch.

@liam are there any externals in use? did you upgrade the entire system
or just Pd? when using an older version of Pd, does the problem persist?

gfmtdsar
IOhannes


>
>
>>
>>
>> Claude
>>
>>> , but with your shell- or system
>>> configuration limiting the number of open file descriptors.
>>>
>>> check your current shell-limit with
>>> ulimit -Sn
>>> -S is for soft limit (you can lower, but not raise, the hard limit).
>>> raise the limit in your shell with
>>> ulimit -n 65536
>>> start pd from that shell and see if the situation improves.
>>>
>>> if that doesn't help, you can check the kernel limits with
>>> cat /proc/sys/fs/file-nr
>>> which returns 3 numbers, the first of which is the number of open files,
>>> the last of which is the limit.
>>>
>>> increase the value of "nofile" in  /etc/security/limits.conf
>>> do
>>> sudo sysctl -p
>>>
>>> additional steps are required to make these settings permanent, the
>>> ulimit -n 65536 would have to go into your .bashrc so it's executed at
>>> system startup. if you normally start pd from your GUI you'd have to
>>> restart your system (actually just xorg) so your master shell knows
>>> about the new value. there's other ways, one of which would be to start
>>> pd from a wrapper script, or probably gnome provides that sort of
>>> environmental setup for it's program shortcuts.
>>>
>>> there's more information
>>> -
>>> https://askubuntu.com/questions/162229/how-do-i-increase-the-open-files-limit-for-a-non-root-user
>>>
>>> -
>>> https://unix.stackexchange.com/questions/29577/ulimit-difference-between-hard-and-soft-limits
>>>
>>>
>>> hope that helps ... cheers,
>>> ub
>>>
>>>> Liam
>>>>
>>>>
>>>> _______________________________________________
>>>> Pd-list@lists.iem.at mailing list
>>>> UNSUBSCRIBE and account-management ->
>>>> https://lists.puredata.info/listinfo/pd-list
>>>>
>>>
>>> _______________________________________________
>>> Pd-list@lists.iem.at mailing list
>>> UNSUBSCRIBE and account-management ->
>>> https://lists.puredata.info/listinfo/pd-list
>
>
> _______________________________________________
> Pd-list@lists.iem.at mailing list
> UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
>