I think the solution is to fix the bug. I most likely introduced it but I can't really look at this right now, so it would be *great* if more people helped.
Hello, I'm trying to find a solution for my project where large amount of midi data should be send.
Dan Wilcox @danomatika http://twitter.com/danomatika danomatika.com http://danomatika.com/ robotcowboy.com http://robotcowboy.com/
hey, can you point me to your changes?
Gesendet: Freitag, 03. Mai 2019 um 23:08 Uhr Von: "Dan Wilcox" danomatika@gmail.com An: "cyrille henry" ch@chnry.net Cc: Pd-List pd-list@lists.iem.at Betreff: Re: [PD] pd-lork and midi on osX
I think the solution is to fix the bug. I most likely introduced it but I can't really look at this right now, so it would be *great* if more people helped. Hello, I'm trying to find a solution for my project where large amount of midi data should be send.
Dan Wilcox @danomatika[http://twitter.com/danomatika] danomatika.com[http://danomatika.com] robotcowboy.com[http://robotcowboy.com] _______________________________________________ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list%5Bhttps://lists.puredata.info/l...]
https://github.com/pure-data/pure-data/pull/214 https://github.com/pure-data/pure-data/pull/214
On May 3, 2019, at 11:29 PM, Christof Ressi christof.ressi@gmx.at wrote:
hey, can you point me to your changes?
Dan Wilcox @danomatika http://twitter.com/danomatika danomatika.com http://danomatika.com/ robotcowboy.com http://robotcowboy.com/
thanks! FWIW, I also get the blocking behavior on Pd 0.46 - 0.49 (Windows) and also on Pd extended, so at least on Windows this has always been a problem. Cyrill only said that it worked on Pd extended but he didn't say anything about older Pd vanilla releases. You've done some macOS specific changes in portmidi/portmidi/pm_mac/pmmacosxcm.c but afaict it's only about *receiving* MIDI messages. So actually I don't think that you've introduced a regression here.
note that when I turn on Cyrill's test patch, audio output is completely blocked and the GUI lags. from what I understand portmidi uses a lockfree ringbuffer so sending a lot of MIDI messages shouldn't block Pd. the patch itself doesn't show a significant CPU increase, it's just blocking unnecessarily.
since it seems to affect both Windows and macOS but not Linux (as Cyrill has noted, I didn't check), we should probably look at the portmidi part.
Christof
Gesendet: Freitag, 03. Mai 2019 um 23:31 Uhr Von: "Dan Wilcox" danomatika@gmail.com An: "Christof Ressi" christof.ressi@gmx.at Cc: Pd-List pd-list@lists.iem.at Betreff: Re: [PD] pd-lork and midi on osX
https://github.com/pure-data/pure-data/pull/214
On May 3, 2019, at 11:29 PM, Christof Ressi <christof.ressi@gmx.at[mailto:christof.ressi@gmx.at]> wrote:
hey, can you point me to your changes?
Dan Wilcox @danomatika[http://twitter.com/danomatika] danomatika.com[http://danomatika.com] robotcowboy.com[http://robotcowboy.com]
from what I understand portmidi uses a lockfree ringbuffer so sending a lot of MIDI messages shouldn't block Pd.
I was wrong, the lockfree ringbuffer is only used for *receiving* MIDI messages. looking at pmwinmm.c, sending MIDI messages certainly involves blocking operations. I think the midi_outqueue should be read by a seperate worker thread.
Christof
Gesendet: Samstag, 04. Mai 2019 um 00:23 Uhr Von: "Christof Ressi" christof.ressi@gmx.at An: "Dan Wilcox" danomatika@gmail.com Cc: Pd-List pd-list@lists.iem.at Betreff: Re: [PD] pd-lork and midi on osX
thanks! FWIW, I also get the blocking behavior on Pd 0.46 - 0.49 (Windows) and also on Pd extended, so at least on Windows this has always been a problem. Cyrill only said that it worked on Pd extended but he didn't say anything about older Pd vanilla releases. You've done some macOS specific changes in portmidi/portmidi/pm_mac/pmmacosxcm.c but afaict it's only about *receiving* MIDI messages. So actually I don't think that you've introduced a regression here.
note that when I turn on Cyrill's test patch, audio output is completely blocked and the GUI lags. from what I understand portmidi uses a lockfree ringbuffer so sending a lot of MIDI messages shouldn't block Pd. the patch itself doesn't show a significant CPU increase, it's just blocking unnecessarily.
since it seems to affect both Windows and macOS but not Linux (as Cyrill has noted, I didn't check), we should probably look at the portmidi part.
Christof
Gesendet: Freitag, 03. Mai 2019 um 23:31 Uhr Von: "Dan Wilcox" danomatika@gmail.com An: "Christof Ressi" christof.ressi@gmx.at Cc: Pd-List pd-list@lists.iem.at Betreff: Re: [PD] pd-lork and midi on osX
https://github.com/pure-data/pure-data/pull/214
On May 3, 2019, at 11:29 PM, Christof Ressi <christof.ressi@gmx.at[mailto:christof.ressi@gmx.at]> wrote:
hey, can you point me to your changes?
Dan Wilcox @danomatika[http://twitter.com/danomatika] danomatika.com[http://danomatika.com] robotcowboy.com[http://robotcowboy.com]
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
That sounds about right, yeah. That matches other issues people have brought up that highlight sending MIDI is affected by the block size.
On May 4, 2019, at 2:31 AM, Christof Ressi christof.ressi@gmx.at wrote:
from what I understand portmidi uses a lockfree ringbuffer so sending a lot of MIDI messages shouldn't block Pd.
I was wrong, the lockfree ringbuffer is only used for *receiving* MIDI messages. looking at pmwinmm.c, sending MIDI messages certainly involves blocking operations. I think the midi_outqueue should be read by a seperate worker thread.
Christof
Gesendet: Samstag, 04. Mai 2019 um 00:23 Uhr Von: "Christof Ressi" christof.ressi@gmx.at An: "Dan Wilcox" danomatika@gmail.com Cc: Pd-List pd-list@lists.iem.at Betreff: Re: [PD] pd-lork and midi on osX
thanks! FWIW, I also get the blocking behavior on Pd 0.46 - 0.49 (Windows) and also on Pd extended, so at least on Windows this has always been a problem. Cyrill only said that it worked on Pd extended but he didn't say anything about older Pd vanilla releases. You've done some macOS specific changes in portmidi/portmidi/pm_mac/pmmacosxcm.c but afaict it's only about *receiving* MIDI messages. So actually I don't think that you've introduced a regression here.
note that when I turn on Cyrill's test patch, audio output is completely blocked and the GUI lags. from what I understand portmidi uses a lockfree ringbuffer so sending a lot of MIDI messages shouldn't block Pd. the patch itself doesn't show a significant CPU increase, it's just blocking unnecessarily.
since it seems to affect both Windows and macOS but not Linux (as Cyrill has noted, I didn't check), we should probably look at the portmidi part.
Christof
Gesendet: Freitag, 03. Mai 2019 um 23:31 Uhr Von: "Dan Wilcox" danomatika@gmail.com An: "Christof Ressi" christof.ressi@gmx.at Cc: Pd-List pd-list@lists.iem.at Betreff: Re: [PD] pd-lork and midi on osX
https://github.com/pure-data/pure-data/pull/214
On May 3, 2019, at 11:29 PM, Christof Ressi <christof.ressi@gmx.at[mailto:christof.ressi@gmx.at]> wrote:
hey, can you point me to your changes?
Dan Wilcox @danomatika[http://twitter.com/danomatika] danomatika.com[http://danomatika.com] robotcowboy.com[http://robotcowboy.com]
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Dan Wilcox @danomatika http://twitter.com/danomatika danomatika.com http://danomatika.com/ robotcowboy.com http://robotcowboy.com/
FWIW, this winter I had to dig into the SuperCollider internals and the server has a nice way to ensure realtime safetey: it maintains a simple single-producer-single-consumer lockfree queue; the so-called realtime thread (RT) puts all non-realtime-safe commands on the queue (e.g. loading soundfiles, sending OSC packets) and the so-called non-realtime thread (NRT) consumes those commands and executes them. I'm thinking of doing something like that for sending MIDI, but it could also be worthwile for the net objects because they are not really realtime safe either (e.g. "netsend/netreceive blocked %d msec"). also, the same approach could help for adding asynchronous versions of the read/write operations for objects like [text] and [soundfiler] - but first we need to make 'canvas_open' and 'open_via_path' threadsafe.
later, the Pd API could even expose a method for externals to queue tasks similar to SC's 'DoAsynchronousCommand', 'SendMsgToRT' and 'SendMsgFromRT'. The idea is that you pass a pointer to user data and some callback functions which are to be called in the NRT or RT. In SC, 'SendMsgToRT' and 'SendMsgFromRT' are simple one-way commands to/from the NRT, whereas 'DoAsynchronousCommand' offers 4 stages: NRT, RT, NRT, RT (cleanup).
Just some food for thought :-)
Christof
Gesendet: Samstag, 04. Mai 2019 um 02:33 Uhr Von: "Dan Wilcox" danomatika@gmail.com An: "Christof Ressi" christof.ressi@gmx.at Cc: Pd-List pd-list@lists.iem.at Betreff: Re: [PD] pd-lork and midi on osX
That sounds about right, yeah. That matches other issues people have brought up that highlight sending MIDI is affected by the block size. On May 4, 2019, at 2:31 AM, Christof Ressi <christof.ressi@gmx.at[mailto:christof.ressi@gmx.at]> wrote:
from what I understand portmidi uses a lockfree ringbuffer so sending a lot of MIDI messages shouldn't block Pd. I was wrong, the lockfree ringbuffer is only used for *receiving* MIDI messages. looking at pmwinmm.c, sending MIDI messages certainly involves blocking operations. I think the midi_outqueue should be read by a seperate worker thread.
Christof Gesendet: Samstag, 04. Mai 2019 um 00:23 Uhr Von: "Christof Ressi" <christof.ressi@gmx.at[mailto:christof.ressi@gmx.at]> An: "Dan Wilcox" <danomatika@gmail.com[mailto:danomatika@gmail.com]> Cc: Pd-List <pd-list@lists.iem.at[mailto:pd-list@lists.iem.at]> Betreff: Re: [PD] pd-lork and midi on osX
thanks! FWIW, I also get the blocking behavior on Pd 0.46 - 0.49 (Windows) and also on Pd extended, so at least on Windows this has always been a problem. Cyrill only said that it worked on Pd extended but he didn't say anything about older Pd vanilla releases. You've done some macOS specific changes in portmidi/portmidi/pm_mac/pmmacosxcm.c but afaict it's only about *receiving* MIDI messages. So actually I don't think that you've introduced a regression here.
note that when I turn on Cyrill's test patch, audio output is completely blocked and the GUI lags. from what I understand portmidi uses a lockfree ringbuffer so sending a lot of MIDI messages shouldn't block Pd. the patch itself doesn't show a significant CPU increase, it's just blocking unnecessarily.
since it seems to affect both Windows and macOS but not Linux (as Cyrill has noted, I didn't check), we should probably look at the portmidi part.
Christof
Gesendet: Freitag, 03. Mai 2019 um 23:31 Uhr Von: "Dan Wilcox" <danomatika@gmail.com[mailto:danomatika@gmail.com]> An: "Christof Ressi" <christof.ressi@gmx.at[mailto:christof.ressi@gmx.at]> Cc: Pd-List <pd-list@lists.iem.at[mailto:pd-list@lists.iem.at]> Betreff: Re: [PD] pd-lork and midi on osX
https://github.com/pure-data/pure-data/pull/214%5Bhttps://github.com/pure-da...]
On May 3, 2019, at 11:29 PM, Christof Ressi <christof.ressi@gmx.at[mailto:christof.ressi@gmx.at]> wrote:
hey, can you point me to your changes?
Dan Wilcox @danomatika[http://twitter.com/danomatika] danomatika.com[http://danomatika.com] robotcowboy.com[http://robotcowboy.com]
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Dan Wilcox @danomatika[http://twitter.com/danomatika] danomatika.com[http://danomatika.com] robotcowboy.com[http://robotcowboy.com]
-
Christof Ressi -
Dan Wilcox