salut patrick!
so -rt is just necessary if precise timing (latency) really matters (in audio), while for video it's better without -rt but it does not mean that my pd will be completely out of time ;) right?
max
P.S. i'll be back in montréal next tuesday!
Am 05.07.2004 um 16:57 schrieb patrick:
hi max!
patrick from montreal (11h11.com).
a quick note about pure data in realtime mode :
first of all, if you're doing audio, you should chmod +s on /usr/bin/pd and then add -rt to your .pdrc if you're doing video, for me it's not working when i'm in RT mode: watchdog...
then, i would say that the REALTIME mode is for audio : with jack and low-latency kernel. patrick
----- Original Message ----- From: "Max Neupert" abonnements@revolwear.com To: pd-list@iem.at Sent: Saturday, July 03, 2004 1:35 PM Subject: [PD] -rt for whom?
who should use -rt? who not? what does it do?
Am 03.07.2004 um 19:05 schrieb chris clepper:
On Jul 3, 2004, at 6:35 AM, aym3ric wrote:
Bugs/Issues: everything is fine, have some weird issues in RT though, not GEM related but really annoying (GEM 0.90 + RT = freeze/crash/watchdog, PDP 0.12.3 + RT + GLX window = freeze/crash/watchdog).
I think a similar thing happens on OSX using -rt as well. The GUI locks up when the CPU loads gets high although GEM continues to render to the window. I've always thought it had to do with GEM monopolizing the video hardware and not giving other processes a chance to draw, and do other UI events. The problem seemed to stem from OSX's GUI being an OpenGL process itself, but looks like that might not be the case.
Thanks cgc
+! aym3ric.
hi max,
with my setup (linux, pd 0.37, gem cvs) i can't run pd in -rt if i'm doing video (-lib Gem) because of watchdog... so i don't know if performance is better in RT mode?
P.S. i'll be back in montrĂƒÂ©al next tuesday!
there's a lot of pure data things happening in montreal right now. welcome back. patrick
----- Original Message ----- From: "Max Neupert" abonnements@revolwear.com To: pd-list@iem.at Sent: Monday, July 05, 2004 11:48 AM Subject: Re: [PD] -rt for whom?
salut patrick!
so -rt is just necessary if precise timing (latency) really matters (in audio), while for video it's better without -rt but it does not mean that my pd will be completely out of time ;) right?
max
P.S. i'll be back in montrĂƒÂ©al next tuesday!
A few notes about -rt mode:
First, -rt is different on OSX compared to Linux. On Linux, the flag sets the priority of the pd process just like nice or renice, but on OSX -rt set priority using a pthread scheduling call. The difference is due to how each OS handles scheduling of tasks - Linux by PID and OSX by threads. The proper way to schedule a 'real-time' thread on OSX is to use the kernel threading API which guarantees time slices for a given process. Pd would have to undergo a but of reworking to use this though (the DSP would have to go in it's own thread separate from the GUI and main routines).
Second, the issues with -rt and video/GL appear to be different for each platform. On OSX, the problem is that the pthread scheduling works too well. GEM simply locks out all other UI processes if CPU load gets too high. This may appear like a freeze or lockup, but the system is actually chugging along as if everything is fine. GEM performance with the higher priority enabled is phenomenal - no dropped frames, immediate movie loading, and low latency. It's just far too easy to have GEM lock you out of all other UI though, which is not a good tradeoff.
Following Tom's idea about turning RT on and off by routine, I should look into something like setting the pthread priority right before and immediately after the render callbacks. This might give the render chain enough time to work while providing a breather for the task manager to allocate time to other lower priority processes. I have no idea what the overhead would be for calling these many times a second.
cgc
On Monday 05 July 2004 13:57, chris clepper wrote:
A few notes about -rt mode:
First, -rt is different on OSX compared to Linux. On Linux, the flag sets the priority of the pd process just like nice or renice
Sorry if I somehow have a basic misunderstanding about how POSIX real-time scheduling works on Linux, but I think you're very wrong (or else the behavior of "nice" has changed a bit since the last time I researched it).
At least on Linux, "pd -rt" schedules the process with POSIX SCHED_FIFO priority (in addition to locking up all allocated memory to avoid page faults), unless something has changed for the worse in the last couple of years. The last time I checked, this is not at all the same as just "nicing" the application; SCHED_FIFO processes get priority over all other procecesses, and assuming the kernel is patches for low latency, the highest priority SCHED_FIFO process will get the processor as soon as it is ready to run, minus what ever latency is still inherent in the Linux kernel.
My understanding was always that this is not the same as simply nicing a process to high heaven. But maybe things have changed? Let's turn this in to a question then: Assuming no sched-fifo procecess are active in a Linux system with the kernel suitably patched for low latency, will the most agressively "niced" process get the CPU as soon as it becomes ready to run, or is it still subject to a time-slicing algorithm? For example, might it only be scheduled on the next clock tick? Or with modern Linux kernels, does the least "nice" application in the system behave identically to a process that has POSIX real-time priority?
Larry Troxler
On Jul 5, 2004, at 3:24 PM, Larry Troxler wrote:
On Monday 05 July 2004 13:57, chris clepper wrote:
A few notes about -rt mode:
First, -rt is different on OSX compared to Linux. On Linux, the flag sets the priority of the pd process just like nice or renice
Sorry if I somehow have a basic misunderstanding about how POSIX
real-time scheduling works on Linux, but I think you're very wrong (or else the behavior of "nice" has changed a bit since the last time I researched
it).
I think you completely missed the point I was making, which concerns
the difference between the way processes are represented in the two
systems. You may indeed be right that (re)nice won't do exactly the
same thing as calling the POSIX scheduling API, but the point is that
treating scheduling on OSX just like Linux is worthless because the
kernel looks at tasks and threads differently. More to the point, Pd
has no actual real-time scheduling enabled on OSX, so the -rt flag is a
misnomer to begin with. Real-time scheduling would be done using the
Mach kernel thread API as outlined in this document:
http://developer.apple.com/documentation/Darwin/Conceptual/ KernelProgramming/scheduler/chapter_8_section_4.html
The section under 'Using the Mach Thread API to Influence Scheduling'
contains the relevant info. Setting priority to FIFO doesn't appear to
do anything at all related to 'real-time' scheduling. By 'real-time' I
mean that exact amounts of time in cycles are set aside for a given
process per second and having the kernel agree to only do that task in
those cycles. Just setting a high priority for a thread won't do that,
and also has some nasty side-effects as evidenced by GEM. Again, this
is apparently completely and totally different from Linux 'real-time',
and implementations should reflect that.
Until recently (0.37 I think) Pd just tried to renice itself on OSX, but currently it uses pthread_setschedparam() to set the SCHED_RR policy. I'm not sure how this relates to SCHED_FIFO and I sure should find out :).
Pd effectively tries to make its main thread the "real-time" one. If you stick hard-core computations in the main thread, as GEM does, this can have a bad effect on your machine. THis is why "-rt" and GEM disagree with each other on OSX - and I suppose on linux as well although I don't know that.
cheers Miller
On Mon, Jul 05, 2004 at 04:57:47PM -0500, chris clepper wrote:
On Jul 5, 2004, at 3:24 PM, Larry Troxler wrote:
On Monday 05 July 2004 13:57, chris clepper wrote:
A few notes about -rt mode:
First, -rt is different on OSX compared to Linux. On Linux, the flag sets the priority of the pd process just like nice or renice
Sorry if I somehow have a basic misunderstanding about how POSIX
real-time scheduling works on Linux, but I think you're very wrong (or else the behavior of "nice" has changed a bit since the last time I researched
it).I think you completely missed the point I was making, which concerns
the difference between the way processes are represented in the two
systems. You may indeed be right that (re)nice won't do exactly the
same thing as calling the POSIX scheduling API, but the point is that
treating scheduling on OSX just like Linux is worthless because the
kernel looks at tasks and threads differently. More to the point, Pd
has no actual real-time scheduling enabled on OSX, so the -rt flag is a
misnomer to begin with. Real-time scheduling would be done using the
Mach kernel thread API as outlined in this document:http://developer.apple.com/documentation/Darwin/Conceptual/ KernelProgramming/scheduler/chapter_8_section_4.html
The section under 'Using the Mach Thread API to Influence Scheduling'
contains the relevant info. Setting priority to FIFO doesn't appear to
do anything at all related to 'real-time' scheduling. By 'real-time' I
mean that exact amounts of time in cycles are set aside for a given
process per second and having the kernel agree to only do that task in
those cycles. Just setting a high priority for a thread won't do that,
and also has some nasty side-effects as evidenced by GEM. Again, this
is apparently completely and totally different from Linux 'real-time',
and implementations should reflect that.
PD-list mailing list PD-list@iem.at to manage your subscription (including un-subscription) see http://iem.at/cgi-bin/mailman/listinfo/pd-list
On Monday 05 July 2004 18:07, Miller Puckette wrote:
Until recently (0.37 I think) Pd just tried to renice itself on OSX, but currently it uses pthread_setschedparam() to set the SCHED_RR policy. I'm not sure how this relates to SCHED_FIFO and I sure should find out :).
SCHED_RR and SCHED_FIFO are close. I think the only difference is if the tasks have the same priority, then maybe SCHED_RR actually starts time-slicing.
But don't take my work for it, look it up :-)
Larry
SCHED_RR and SCHED_FIFO are close. I think the only difference is if the tasks have the same priority, then maybe SCHED_RR actually starts time-slicing. But don't take my work for it, look it up :-)
that's correct ... if there are two threads (a,b) with the same priority, SCHED_FIFO will cause that the first of these thread will run until it's finished or sleeping, then the second thread can start (aaaaaabbbbbb). SCHED_RR will cause to run both threads for small timeslices (ababababab)... but afaik, it only has an effect if both threads are using the same priority, so with pd, i don't know if there is an effect when using -rt. if there are two threads, one having a high priority (reniced by -rt) and one low priority there would be no difference ... with a threaded pd (e.g. the devel_0_37) there would be effects. iirc, the soundfiler thread and the gui thread are working in the same priority...
another feature of the -rt mode is the memory locking ... it's enabled on machines, where _POSIX_MEMLOCK is defined (at least on linux it is). this will force the memory to be represented in the physical memory. (one will not be able to allocate more memory).
cheers ... tim
another feature of the -rt mode is the memory locking ... it's enabled on machines, where _POSIX_MEMLOCK is defined (at least on linux it is). this will force the memory to be represented in the physical memory. (one will not be able to allocate more memory).
actually i think they should be two separate features :
the real-time priority ( -rt )
and the memory locking ( -ml ? )
cause you might want a high priority without the memory limitation.
cheers, sevy
On Tuesday 06 July 2004 06:42, Tim Blechmann wrote:
SCHED_RR and SCHED_FIFO are close. I think the only difference is if the tasks have the same priority, then maybe SCHED_RR actually starts time-slicing. But don't take my work for it, look it up :-)
that's correct ... if there are two threads (a,b) with the same priority, SCHED_FIFO will cause that the first of these thread will run until it's finished or sleeping, then the second thread can start (aaaaaabbbbbb). SCHED_RR will cause to run both threads for small timeslices (ababababab)... but afaik, it only has an effect if both threads are using the same priority, so with pd, i don't know if there is an effect when using -rt.
I suppose it could matter if there is an other application running one of its threads SCHED_FIFO with the same priority.
if there are two threads, one having a high priority (reniced by -rt) and one low priority there would be no difference ...
Please don't say that "-rt" "renices" the thread. Unless I'm mistaken, the "nice" setting only applies to non-realtime threads. It is a different scheduling policy entirely. Try looking at the "nice" value of PD threads in "ps" or "top" for example. It won't make any sense, because its value is not used by the POSIX soft-realtime scheduler.
another feature of the -rt mode is the memory locking ... it's enabled on machines, where _POSIX_MEMLOCK is defined (at least on linux it is). this will force the memory to be represented in the physical memory. (one will not be able to allocate more memory).
I thought that the memory locking was done by a seperate system call, and that setting the scheduling policy to SCHED_FIFO or SCHED_RR by itself did not lock memory. Also, you can still allocate memory after making this call. I had assumed that if you use memlockall (or however it's spelled), that any memory allocated after the call will also be locked (is this correct?)
Larry
I thought that the memory locking was done by a seperate system call, and that setting the scheduling policy to SCHED_FIFO or SCHED_RR by itself did not lock memory. Also, you can still allocate memory after making this call. I had assumed that if you use memlockall (or however it's spelled), that any memory allocated after the call will also be locked (is this correct?)
yes it is independant from the scheduling policy ... the memory locking _could_, as yves proposed, be used independantly from the realtime scheduling...
cheers... tim
On Mon, 5 Jul 2004, Miller Puckette wrote:
Pd effectively tries to make its main thread the "real-time" one. If you stick hard-core computations in the main thread, as GEM does, this can have a bad effect on your machine. THis is why "-rt" and GEM disagree with each other on OSX - and I suppose on linux as well although I don't know that.
My understanding was that currently it is simply not possible to do non-trivial audio and video processing with pd at the same time. This is because the video extensions do not have their own thread. I think pdp has sort of an option to run in its own thread, but I don't know how well this works, or how it is implemented.
The solution in general is to have two pd's running at the same time. One doing the video using the -nosound flag and communicating with the other one, with -rt which is doing the audio.
In theory, with Thomas Grills simple lock mechanism it should be possible to change gem and pdp and put them into a separate thread.
Guenter
cheers Miller
On Mon, Jul 05, 2004 at 04:57:47PM -0500, chris clepper wrote:
On Jul 5, 2004, at 3:24 PM, Larry Troxler wrote:
On Monday 05 July 2004 13:57, chris clepper wrote:
A few notes about -rt mode:
First, -rt is different on OSX compared to Linux. On Linux, the flag sets the priority of the pd process just like nice or renice
Sorry if I somehow have a basic misunderstanding about how POSIX real-time scheduling works on Linux, but I think you're very wrong (or else the behavior of "nice" has changed a bit since the last time I researched it).
I think you completely missed the point I was making, which concerns the difference between the way processes are represented in the two systems. You may indeed be right that (re)nice won't do exactly the same thing as calling the POSIX scheduling API, but the point is that treating scheduling on OSX just like Linux is worthless because the kernel looks at tasks and threads differently. More to the point, Pd has no actual real-time scheduling enabled on OSX, so the -rt flag is a misnomer to begin with. Real-time scheduling would be done using the Mach kernel thread API as outlined in this document:
http://developer.apple.com/documentation/Darwin/Conceptual/ KernelProgramming/scheduler/chapter_8_section_4.html
The section under 'Using the Mach Thread API to Influence Scheduling' contains the relevant info. Setting priority to FIFO doesn't appear to do anything at all related to 'real-time' scheduling. By 'real-time' I mean that exact amounts of time in cycles are set aside for a given process per second and having the kernel agree to only do that task in those cycles. Just setting a high priority for a thread won't do that, and also has some nasty side-effects as evidenced by GEM. Again, this is apparently completely and totally different from Linux 'real-time', and implementations should reflect that.
PD-list mailing list PD-list@iem.at to manage your subscription (including un-subscription) see http://iem.at/cgi-bin/mailman/listinfo/pd-list
PD-list mailing list PD-list@iem.at to manage your subscription (including un-subscription) see http://iem.at/cgi-bin/mailman/listinfo/pd-list
On Monday 05 July 2004 17:57, chris clepper wrote:
On Jul 5, 2004, at 3:24 PM, Larry Troxler wrote:
On Monday 05 July 2004 13:57, chris clepper wrote:
A few notes about -rt mode:
First, -rt is different on OSX compared to Linux. On Linux, the flag sets the priority of the pd process just like nice or renice
Sorry if I somehow have a basic misunderstanding about how POSIX real-time scheduling works on Linux, but I think you're very wrong (or else the behavior of "nice" has changed a bit since the last time I researched it).
I think you completely missed the point I was making, which concerns the difference between the way processes are represented in the two systems.
Hi Chris. I do understand that you were talking primarily about OSX and not about Linux, and I'm not at all disupting the primary point you are making. But even though it may have nothing do do with your argument, you made a statement about how "pd -rt" works on Linux (which I do have some experience with), and I believe that what you said is in error. So, I felt that I needed to reply. The fact is, that on Linux, using POSIX real-time scheduling as enabled by the "-rt" option, _is_ different then simply "nice'ing" a process to a not so nice level. I just wanted to clarify that. That's really all.
Larry Troxler
On Jul 5, 2004, at 6:39 PM, Larry Troxler wrote:
Hi Chris. I do understand that you were talking primarily about OSX and not about Linux, and I'm not at all disupting the primary point you are making. But even though it may have nothing do do with your argument, you made a statement about how "pd -rt" works on Linux (which I do have some experience with), and I believe that what you said is in error. So, I felt that I needed to reply. The fact is, that on Linux, using POSIX real-time scheduling as enabled by the "-rt" option, _is_ different then simply "nice'ing" a process to a not so nice level. I just wanted to clarify that. That's really all.
Yep, you are correct. I was looking at my local source for s_inter.c
and it has a renice call for OSX, which I thought was also for Linux.
This file is out of date however and no longer applies to any current
version of Pd.
cgc