hi miller,

Well, forking the source wasn't my idea... I knew it would cause trouble.

well, for me it seems to be the only way to get a working performance system for me, that will do what i want without waiting for another century. basically the current devel branch has been extended so far, that it's not easy to merge it with Head any more ... "diff -u Head/src/ devel_0_39/src/ | wc -l" are more than 12500 lines (Head itself has about 66000 lines)

i stopped posting patches to the patch tracker, since the patches would be way to big or depend on each other ...

i'm currently thinking to fork off from pure data and release the current devel code as pure devil because it's not the same software any more and i'm loosing too much time with porting / merging ... (if i would have worked on a day job during the time i read diffs, merged files or rewrote things, i could easily afford a fancy powerbook and a few licenses of max/msp) i'd really prefer to spend my time improving pd / making music than merging branches or rewrite stuff...

In general, the question of scheduling graphics updates is hard, especially now that (arrays can contain (lists that contain (arrays of scalars))), etc. Having "garrays" separate from the "real" data structure mechanism was always a temporary measure, and now I'm glad it's gone. But what this means is that now we need a more general way of updating changes to data. If I can think of a good one I'll get it into 0.39 (it's prety high on the dolist at the moment...)

i've already ported most of your changes from 0.38 to 0.39 to devel (not the other way around, because that would be pretty impossible due the amount of code), so i would probably fork off with pure devil when both your changes to Head become stable and i consider my changes to devel as stable. i'm trying to keep (binary) compatibility as long as possible, but if i have to break compatibility (e.g. to declare getfloatarray as deprecated) i would definitely do that ...

pd has the potential to become a professional software, but i don't want to wait for another 10 or 20 years for pd to become usable / threadsafe / low-latency optimized / professional. with all the troubles of forking the advantage would be that i could break Head and you could break devel without having to think too much about the effects on the other branch ... (well, we don't care about the effects at the moment, either :-(, which results in hours of merging from Head to devel at the moment)

the best thing of course, would be to combine the efforts of Head and devel/devil having several maintainers for several parts of pd (data structures, scheduler, dsp kernel, messaging, midi, simd, gui, build system, documentation, win32, osx, linux, irix, pda, cell phone, microcontroller, whatever) ...

cheers ... tim

If you want to fork, then do it, that's fine. The people who have been working devel have been going that way for a while. I fully support the goal of addressing usability issues now, that's inevitably going to break things. But you can't expect people who aren't using devel at all to track your changes, especially if you are unwilling to submit working patches. Like you said there aren't

well, i don't expect that people track my changes, since i track miller's changes .... my local copy of devel_0_39 is an exact copy of devel_0_38 with miller's changes from 0.38 to 0.39 ... (i'll check it in the cvs, when it compiles fine on osx) beside that, it would be easier for miller to track my changes (not suggesting, that he should do that), than for me to track his changes, since he wrote pd and thus knows about 100% of the code and thanks to the pd-cvs mailing lists, it's even very easy to track changes...

Sure, submitting patches is a slow process, but I think its starting to work. Miller has started accepting a number of patches, mostly

is it? the patches that miller accepted where usually only patches of a few lines ... and he also said, he's not taking the simd code written by thomas grill and ported to linux by me, since he can't / don't want to maintain it ... (well, both thomas and i could easily maintain this) ... nor did he accept günter's tool tips (which are possibly the most useful addition to pd)

he's also very critical about the use of threads, thus i guess, he's not willing to maintain my callback-based scheduler / idle-callbacks, nor the lockfree fifo implementation... (miller, please correct me if i'm wrong :) and the improved portaudio code (allowing acceptable latencies) and the current jack implementation rely on both the callback-based scheduler and the idle-callbacks ... as you stated, submitting patches is a slow process ... a very slow process ... patches work fine if the don't really depend on each other, but if they do, managing patches becomes pretty difficult ... it would be easier to have a development team (e.g. consisting of the people who commited changes to devel during the last two years) working on the same sources on the cvs. this would definitely improve the speed of development a lot ...

bugfixes, but its a start. I am personally disappointed that there is a fork, each branch has its own unique strengths. I still think that

a fork is about the worst thing that can happen to the development of a program, you are totally right ... i really don't _want_ to fork, i'd prefer to work on a community pure data, but if this is not working, i'll try it with a community pure devil ...

if the extra effort is taken to break up many/most of devel's changes into digestable patches, it will get included.

well, i'd guess, it would take quite some time to break down devel's changes to single patches, during this time, i could possibly write a native core-audio interface or the feature to load pd as a ladspa plugin ... or record a solo cd ... beside that, if miller rejects my improved scheduler or the idle callbacks, most other patches will fail and i did the work for nothing ... it would basically be much easier to go through the diffs between stable and devel and talk about the changes ... most of them are even well commented.

i really think, parts of pd are really great and i highly appreciate millers work, but i also see some problems and think that a community-driven pure devil would solve these problems much faster ... so that we can think sooner of making music than of making music software ...

cheers ... tim

Hi all, i'm basically of the same opinion as Günther, but i also think we are stuck in a non-satisfying situation, where i'm fully with Tim. Right now, the critical parts in PD for me and my GUI project vibrez are arrays und DSP. I'm afraid that arrays won't be really usable for some time, because there's no click-free loading of large amounts of data, no synchronisation between array-using objects and no feedback about dirty regions. Currently, i am thinking of implementing my own array objects, because there's not much lost (in my view) with abandoning the current PD array functionality and associated internals. Concerning DSP, i probably make a new attempt in the future to split the DSP chain in separate parts for each root-patcher in the first place, and then maybe even try that for each patcher/abstraction, as already discussed here some time ago. This is the most straighforward way to enable a click-free delayed loading of patchers, as to be implemented in dyn~. Still, since this is a lot of work, it would be good to hear Miller's opinion on this, whether it collides with his ideas or not. It's very hard to contribute things without having the greater picture of future developments. As for the overall view, i would very much support a more modular concept of PD, with separation into several then easier to maintain parts, above all concerning kernel and GUI, but also data processing (like SIMD), data structures, internals, DSP etc.

best greetings, Thomas

� schrieb:

Hi Tim,

On Sat, 9 Jul 2005, Tim Blechmann wrote:

...

beside that, it would be easier for miller to track my changes (not suggesting, that he should do that), than for me to track his changes, since he wrote pd and thus knows about 100% of the code and thanks to the pd-cvs mailing lists, it's even very easy to track changes...

I think that it might be just as hard, if not harder for Miller to track your changes, also because you probably have a different view of what Pd is supposed to be and where it should be going too. So I fear you have to step back, as you are not the author of Pd.

...

...

Sure, submitting patches is a slow process, but I think its starting to work. Miller has started accepting a number of patches, mostly

is it? the patches that miller accepted where usually only patches of a few lines ... and he also said, he's not taking the simd code written by thomas grill and ported to linux by me, since he can't / don't want to maintain it ... (well, both thomas and i could easily maintain this) ... nor did he accept g�nter's tool tips (which are possibly the most useful addition to pd)

I think that Miller has the absolute right to not accept patches, either because they touch parts of Pd that should be left alone or because they are just too big. Incorporating patches is a task that takes a lot of care, I had to realize that when I included the alsa-midi patch into the Debian package.

...

he's also very critical about the use of threads, thus i guess, he's not willing to maintain my callback-based scheduler / idle-callbacks, nor the lockfree fifo implementation... (miller, please correct me if i'm wrong :)

Changing the scheduler is not a minimal change. I also think that the SSE code can be maintained apart from the main pd. For PDa I have just rewritten the signal objects as externals, the changes to make this work in Pd are just a few lines of code. This way Miller could be sure that no bugs are introduced in the standart Pd version, and using SSE would be a call to "pd -nomsp" and the "MSP" part gets loaded dynamically. I have done something similar with the scheduler.

Anyhow, what I want to say is that structuring the changes might help the transition. And even if your additions don't go into the main Pd distribution, this scheme also helps to port them to a new release of Miller, because your changes in PD itself are just some "hooks" to your code. (this was my main concern when chosing this structure).

I admit that this probably won't work for the soundfiler and array locks, but it would mean that this patch would be a few lines of code and maybe gets accepted by miller.

Cheers, Guenter

...

and the improved portaudio code (allowing acceptable latencies) and the current jack implementation rely on both the callback-based scheduler and the idle-callbacks ... as you stated, submitting patches is a slow process ... a very slow process ... patches work fine if the don't really depend on each other, but if they do, managing patches becomes pretty difficult ... it would be easier to have a development team (e.g. consisting of the people who commited changes to devel during the last two years) working on the same sources on the cvs. this would definitely improve the speed of development a lot ...

...

bugfixes, but its a start. I am personally disappointed that there is a fork, each branch has its own unique strengths. I still think that

a fork is about the worst thing that can happen to the development of a program, you are totally right ... i really don't _want_ to fork, i'd prefer to work on a community pure data, but if this is not working, i'll try it with a community pure devil ...

...

if the extra effort is taken to break up many/most of devel's changes into digestable patches, it will get included.

well, i'd guess, it would take quite some time to break down devel's changes to single patches, during this time, i could possibly write a native core-audio interface or the feature to load pd as a ladspa plugin ... or record a solo cd ... beside that, if miller rejects my improved scheduler or the idle callbacks, most other patches will fail and i did the work for nothing ... it would basically be much easier to go through the diffs between stable and devel and talk about the changes ... most of them are even well commented.

i really think, parts of pd are really great and i highly appreciate millers work, but i also see some problems and think that a community-driven pure devil would solve these problems much faster ... so that we can think sooner of making music than of making music software ...

cheers ... tim

-- mailto:TimBlechmann@gmx.de ICQ: 96771783 http://www.mokabar.tk

latest mp3: kMW.mp3 http://mattin.org/mp3.html

latest cd: Goh Lee Kwang & Tim Blechmann: Drone http://www.geocities.com/gohleekwangtimblechmannduo/

After one look at this planet any visitor from outer space would say "I want to see the manager." William S. Burroughs

---

PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev

---

PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev

I think that Miller has the absolute right to not accept patches, either because they touch parts of Pd that should be left alone or because they are just too big. Incorporating patches is a task that takes a lot of care, I had to realize that when I included the alsa-midi patch into the Debian package.

sure, it's miller's right to reject patches ... like it's my right to fork from pure data to release and maintain my flavor of pd ;-) none of these solutions it optimal, though ...

Changing the scheduler is not a minimal change. I also think that the SSE code can be maintained apart from the main pd. For PDa I have just

well, i never said that the changes are minimal ... but changing the scheduler was necessary to achieve better latencies ... the sse code can't easily be separated from the pd kernel, since functions are used for both dsp objects and the dsp tree ... beside that, no one has to use the simd code ... the inclusion of the simd code is a compile time option ...

Anyhow, what I want to say is that structuring the changes might help the transition. And even if your additions don't go into the main Pd distribution, this scheme also helps to port them to a new release of Miller, because your changes in PD itself are just some "hooks" to your code. (this was my main concern when chosing this structure).

you can probably use hooks for a few tasks, but it's getting difficult, if you are adding features, that you want to use / export for external developers (e.g. tooltips or idle callbacks)

I admit that this probably won't work for the soundfiler and array locks, but it would mean that this patch would be a few lines of code and maybe gets accepted by miller.

one example ... a few days ago i improved the data structures for the clock callbacks (from a singly linked list to nested lists, still O(n), but faster than the old implementation) ... since i can't run the clock callbacks in the dsp callback (threading issue), i introduced a run_clock_callback function that's run in both the traditional and my scheduler for callback-based dsp ticks ... now i can write a patch to port the improvement back to stable, but this would conflict with another patch i would write to submit my scheduler changes ... during this time, i could probably find out, how to implement the tooltips for outlets ...

cheers ... tim

hi miller,

In general, the question of scheduling graphics updates is hard, especially now that (arrays can contain (lists that contain (arrays of scalars))), etc. Having "garrays" separate from the "real" data structure mechanism was always a temporary measure, and now I'm glad it's gone. But what this means is that now we need a more general way of updating changes to data. If I can think of a good one I'll get it into 0.39 (it's prety high on the dolist at the moment...)

before i'm starting to work on an implementation on my own ... have you been thinking about this?

t

On Fri, 5 Aug 2005, Tim Blechmann wrote:

...

In general, the question of scheduling graphics updates is hard, especially now that (arrays can contain (lists that contain (arrays of scalars))), etc. Having "garrays" separate from the "real" data structure mechanism was always a temporary measure, and now I'm glad it's gone. But what this means is that now we need a more general way of updating changes to data. If I can think of a good one I'll get it into 0.39 (it's prety high on the dolist at the moment...)

before i'm starting to work on an implementation on my own ... have you been thinking about this?

I propose something like this:

typedef void (*t_garray_observer)(t_garray*,int start,int length); void garray_add_observer(t_garray*,t_garray_observer); void garray_del_observer(t_garray*,t_garray_observer); void garray_notify(t_garray*self,int start,int length) { self->observer(self,start,length); }

such that the add and del functions maintain a list of observers to notify when a change is being applied.

the garray objects call garray_notify method just _after_ every change they make to the data, to indicate the span of data that has been modified: that is, the start index, and the number of entries that have been modified.

any piece of code that is external to a garray but modifies its data directly instead of through the object's method, must now call garray_notify as if it were the garray object itself. that's the difficult part because it means incompatibility with the objects that may have modified the data silently.

does that make sense?

I'd like a more general mechanism than that, which may apply to all pd objects, such as:

typedef void (*t_pd_observer)(t_pd*); void pd_add_observer(t_pd*,t_pd_observer); void pd_del_observer(t_pd*,t_pd_observer); void pd_notify(t_pd*,...);

This is something being suggested by the IMPD paper presented last year at PdConvention04:

http://artengine.ca/matju/graz/ImpureData_PdConvention04.pdf

In section 1.1; it's just that the one above is more flexible, as it can apply to parts of objects instead of just wholes.

However I'll have to think some more about how the "..." part would work, because varargs in C/C++ are a bit clumsy to use...

____________________________________________________________________ Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju Freelance Digital Arts Engineer, Montréal QC Canada