Thanks for the quick reply!

2 use-cases.

1- we can generate or retrieve a dataset's content to use natively in the creative coding environment (Max Pd Sc) so that it integrates in other workflows of data mangling and drawing. We already have in our dataset object file support and single point entry and query but this allow batch dump and load. The structure is:

{ "cols" : 3, "data" : { "entry-0" : [ -0.06755, 0.44185, -0.33835 ], "entry-1" : [ -0.12305, -0.24085, 0.31295 ], "entry-2" : [ -0.0595, -0.2881, 0.0597 ] }

}

2- we can retrieve or set the state of a complex objects. Our object will spit out its internal status ( for instance, a neural net) and we can use the values of its parameters, like below. More interestingly, we can also query its structure and use that information too.

{ "layers" : [ { "activation" : 3, "biases" : [ -3.076234734727154, 0.772760846709679 ], "cols" : 2, "rows" : 1, "weights" : [ [ 6.015551733036155, -1.826803841455323 ] ] } , { "activation" : 3, "biases" : [ -0.490600074475542 ], "cols" : 1, "rows" : 2, "weights" : [ [ -3.115116035462417 ], [ -3.969281643687132 ] ] } ] }

The key-value nesting is quite powerful for this type of open structure...

On 7 Sep 2021, at 15:51, Christof Ressi info@christofressi.com wrote:

Can you give an example of how the data is structured?

In which ways are users supposed to interact with the data?

Christof

On 07.09.2021 16:37, Pierre Alexandre Tremblay wrote:

...

Dear all

I am trying to find the most Pd-vanilla-way to interface with our Dataset object in the FluCoMa project. In Max and SuperCollider we use Dictionaries, which are nestable and queryable in powerful programmatic ways, working essentially like interfaces to JSON-like data structures.

I’ve looked at [struct] but the [set] object does not allow to do symbols and (list of) floats, and [appends] seem to have the same limitations. In all cases, I’m not certain it is the best approach in any cases to create such a list in Pd...

I wonder how people do it and if I am missing an obvious workflow, especially with nested structures.

Any pointer (pun intended) welcome

p

---

Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list

---

Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list

Does the max documentation of flucoma has concrete examples of what you're looking for?

There is the purest_json library (which isn't vanilla), but maybe with some hacking it might be possible to read files. Not sure about writing, but my hacking isn't up to date with the current pd state.

Thanks for the quick reply!

2 use-cases.

1- we can generate or retrieve a dataset's content to use natively in the creative coding environment (Max Pd Sc) so that it integrates in other workflows of data mangling and drawing. We already have in our dataset object file support and single point entry and query but this allow batch dump and load. The structure is:

{ "cols" : 3, "data" : { "entry-0" : [ -0.06755, 0.44185, -0.33835 ], "entry-1" : [ -0.12305, -0.24085, 0.31295 ], "entry-2" : [ -0.0595, -0.2881, 0.0597 ] }

}

2- we can retrieve or set the state of a complex objects. Our object will spit out its internal status ( for instance, a neural net) and we can use the values of its parameters, like below. More interestingly, we can also query its structure and use that information too.

{ "layers" : [ { "activation" : 3, "biases" : [ -3.076234734727154, 0.772760846709679 ], "cols" : 2, "rows" : 1, "weights" : [ [ 6.015551733036155, -1.826803841455323 ] ] } , { "activation" : 3, "biases" : [ -0.490600074475542 ], "cols" : 1, "rows" : 2, "weights" : [ [ -3.115116035462417 ], [ -3.969281643687132 ] ] } ] }

The key-value nesting is quite powerful for this type of open structure...

...

On 7 Sep 2021, at 15:51, Christof Ressi info@christofressi.com wrote:

Can you give an example of how the data is structured?

In which ways are users supposed to interact with the data?

Christof

On 07.09.2021 16:37, Pierre Alexandre Tremblay wrote:

...

Dear all

I am trying to find the most Pd-vanilla-way to interface with our Dataset object in the FluCoMa project. In Max and SuperCollider we use Dictionaries, which are nestable and queryable in powerful programmatic ways, working essentially like interfaces to JSON-like data structures.

I’ve looked at [struct] but the [set] object does not allow to do symbols and (list of) floats, and [appends] seem to have the same limitations. In all cases, I’m not certain it is the best approach in any cases to create such a list in Pd...

I wonder how people do it and if I am missing an obvious workflow, especially with nested structures.

Any pointer (pun intended) welcome

p

The least-horrible way to learn about it is the "4.data.structures" tutorial sequence.

But the whole idea of hierarchical data structures maps horribly to a patch language - there needs to be a better way to access 'data' in Pd.

cheers M

On Tue, Sep 07, 2021 at 08:23:58PM +0100, Pierre Alexandre Tremblay wrote:

There are plenty of examples indeed :)

I’ll give struct with text and arrays a fair fight, I find the doc especially quite hard to follow for non-graphical usage but it might just be me.

...

On 7 Sep 2021, at 19:23, João Pais jmmmpais@gmail.com wrote:

Does the max documentation of flucoma has concrete examples of what you're looking for?

There is the purest_json library (which isn't vanilla), but maybe with some hacking it might be possible to read files. Not sure about writing, but my hacking isn't up to date with the current pd state.

...

Thanks for the quick reply!

2 use-cases.

1- we can generate or retrieve a dataset's content to use natively in the creative coding environment (Max Pd Sc) so that it integrates in other workflows of data mangling and drawing. We already have in our dataset object file support and single point entry and query but this allow batch dump and load. The structure is:

{ "cols" : 3, "data" : { "entry-0" : [ -0.06755, 0.44185, -0.33835 ], "entry-1" : [ -0.12305, -0.24085, 0.31295 ], "entry-2" : [ -0.0595, -0.2881, 0.0597 ] }

}

2- we can retrieve or set the state of a complex objects. Our object will spit out its internal status ( for instance, a neural net) and we can use the values of its parameters, like below. More interestingly, we can also query its structure and use that information too.

{ "layers" : [ { "activation" : 3, "biases" : [ -3.076234734727154, 0.772760846709679 ], "cols" : 2, "rows" : 1, "weights" : [ [ 6.015551733036155, -1.826803841455323 ] ] } , { "activation" : 3, "biases" : [ -0.490600074475542 ], "cols" : 1, "rows" : 2, "weights" : [ [ -3.115116035462417 ], [ -3.969281643687132 ] ] } ] }

The key-value nesting is quite powerful for this type of open structure...

...

On 7 Sep 2021, at 15:51, Christof Ressi info@christofressi.com wrote:

Can you give an example of how the data is structured?

In which ways are users supposed to interact with the data?

Christof

On 07.09.2021 16:37, Pierre Alexandre Tremblay wrote:

...

Dear all

I am trying to find the most Pd-vanilla-way to interface with our Dataset object in the FluCoMa project. In Max and SuperCollider we use Dictionaries, which are nestable and queryable in powerful programmatic ways, working essentially like interfaces to JSON-like data structures.

I’ve looked at [struct] but the [set] object does not allow to do symbols and (list of) floats, and [appends] seem to have the same limitations. In all cases, I’m not certain it is the best approach in any cases to create such a list in Pd...

I wonder how people do it and if I am missing an obvious workflow, especially with nested structures.

Any pointer (pun intended) welcome

p

---

Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.puredata.info_lis...

you still have my data structures tutorials? that might be easier than the documentation.

I’ll give tutorial 4 a go, with the hints of everyone to see how far I can go.

...

On 7 Sep 2021, at 20:34, Miller Puckette msp@ucsd.edu wrote:

The least-horrible way to learn about it is the "4.data.structures" tutorial sequence.

But the whole idea of hierarchical data structures maps horribly to a patch language - there needs to be a better way to access 'data' in Pd.

cheers M

On Tue, Sep 07, 2021 at 08:23:58PM +0100, Pierre Alexandre Tremblay wrote:

...

There are plenty of examples indeed :)

I’ll give struct with text and arrays a fair fight, I find the doc especially quite hard to follow for non-graphical usage but it might just be me.

...

On 7 Sep 2021, at 19:23, João Pais jmmmpais@gmail.com wrote:

Does the max documentation of flucoma has concrete examples of what you're looking for?

There is the purest_json library (which isn't vanilla), but maybe with some hacking it might be possible to read files. Not sure about writing, but my hacking isn't up to date with the current pd state.

...

Thanks for the quick reply!

2 use-cases.

1- we can generate or retrieve a dataset's content to use natively in the creative coding environment (Max Pd Sc) so that it integrates in other workflows of data mangling and drawing. We already have in our dataset object file support and single point entry and query but this allow batch dump and load. The structure is:

{ "cols" : 3, "data" : { "entry-0" : [ -0.06755, 0.44185, -0.33835 ], "entry-1" : [ -0.12305, -0.24085, 0.31295 ], "entry-2" : [ -0.0595, -0.2881, 0.0597 ] }

}

2- we can retrieve or set the state of a complex objects. Our object will spit out its internal status ( for instance, a neural net) and we can use the values of its parameters, like below. More interestingly, we can also query its structure and use that information too.

{ "layers" : [ { "activation" : 3, "biases" : [ -3.076234734727154, 0.772760846709679 ], "cols" : 2, "rows" : 1, "weights" : [ [ 6.015551733036155, -1.826803841455323 ] ] } , { "activation" : 3, "biases" : [ -0.490600074475542 ], "cols" : 1, "rows" : 2, "weights" : [ [ -3.115116035462417 ], [ -3.969281643687132 ] ] } ] }

The key-value nesting is quite powerful for this type of open structure...

...

On 7 Sep 2021, at 15:51, Christof Ressi info@christofressi.com wrote:

Can you give an example of how the data is structured?

In which ways are users supposed to interact with the data?

Christof

On 07.09.2021 16:37, Pierre Alexandre Tremblay wrote: > Dear all > > I am trying to find the most Pd-vanilla-way to interface with our Dataset object in the FluCoMa project. In Max and SuperCollider we use Dictionaries, which are nestable and queryable in powerful programmatic ways, working essentially like interfaces to JSON-like data structures. > > I’ve looked at [struct] but the [set] object does not allow to do symbols and (list of) floats, and [appends] seem to have the same limitations. In all cases, I’m not certain it is the best approach in any cases to create such a list in Pd... > > I wonder how people do it and if I am missing an obvious workflow, especially with nested structures. > > Any pointer (pun intended) welcome > > p > > >

is jmmmp/cellblock a model of what you could use? but adapted to lists and maybe not so complex?

I tried opening your first example in [text]. As I thought, the issue is that commas are already reserved in pd for line changes. The output is this (I added a line number at the start of each line) when using carriage returns:

print:  0 { print:  1 "cols" : 3 print: 2 print:  3 "data" : { print:  4 "entry-0" : [ -0.06755 print:  5 0.44185 print:  6 -0.33835 ] print: 7 print:  8 "entry-1" : [ -0.12305 print:  9 -0.24085 print:  10 0.31295 ] print: 11 print:  12 "entry-2" : [ -0.0595 print:  13 -0.2881 print:  14 0.0597 ] print:  15 } print: 16 print:  17 } print: 18

Or this without them:

print:  0 { "cols" : 3 print:  1 "data" : { "entry-0" : [ -0.06755 print:  2 0.44185 print:  3 -0.33835 ] print:  4 "entry-1" : [ -0.12305 print:  5 -0.24085 print:  6 0.31295 ] print:  7 "entry-2" : [ -0.0595 print:  8 -0.2881 print:  9 0.0597 ] } } print: 10

which makes it a bit hard to read json files without some hacking, as all those commas create a new line. Unless there is another method, is it better to code your own json interpreter?

One advantage of [text] is that it can be used to declare data structures directly - examples might be in my tutorial, I'm not sure anymore how far I got with it.

Files are attached.

Joao

Thanks for the quick reply!

2 use-cases.

1- we can generate or retrieve a dataset's content to use natively in the creative coding environment (Max Pd Sc) so that it integrates in other workflows of data mangling and drawing. We already have in our dataset object file support and single point entry and query but this allow batch dump and load. The structure is:

{ "cols" : 3, "data" : { "entry-0" : [ -0.06755, 0.44185, -0.33835 ], "entry-1" : [ -0.12305, -0.24085, 0.31295 ], "entry-2" : [ -0.0595, -0.2881, 0.0597 ] }

}

2- we can retrieve or set the state of a complex objects. Our object will spit out its internal status ( for instance, a neural net) and we can use the values of its parameters, like below. More interestingly, we can also query its structure and use that information too.

{ "layers" : [ { "activation" : 3, "biases" : [ -3.076234734727154, 0.772760846709679 ], "cols" : 2, "rows" : 1, "weights" : [ [ 6.015551733036155, -1.826803841455323 ] ] } , { "activation" : 3, "biases" : [ -0.490600074475542 ], "cols" : 1, "rows" : 2, "weights" : [ [ -3.115116035462417 ], [ -3.969281643687132 ] ] } ] }

The key-value nesting is quite powerful for this type of open structure...

...

On 7 Sep 2021, at 15:51, Christof Ressi info@christofressi.com wrote:

Can you give an example of how the data is structured?

In which ways are users supposed to interact with the data?

Christof

On 07.09.2021 16:37, Pierre Alexandre Tremblay wrote:

...

Dear all

I am trying to find the most Pd-vanilla-way to interface with our Dataset object in the FluCoMa project. In Max and SuperCollider we use Dictionaries, which are nestable and queryable in powerful programmatic ways, working essentially like interfaces to JSON-like data structures.

I’ve looked at [struct] but the [set] object does not allow to do symbols and (list of) floats, and [appends] seem to have the same limitations. In all cases, I’m not certain it is the best approach in any cases to create such a list in Pd...

I wonder how people do it and if I am missing an obvious workflow, especially with nested structures.

Any pointer (pun intended) welcome

p

can you post an example of a parsed content?

Thanks all for the pointers. Now it is on mer to check the other tutorials

and find a way around. Our object does parse json in and out to file and std if needs be, the only idea here is a batch interface with Pd so users can input and retrieve en masse to our dataset world (with mlp and umap and other fun ML goodies - all working so soon on a repo near you if I can crack that last interface problem :)

...

On 8 Sep 2021, at 08:26, João Pais jmmmpais@gmail.com wrote:

is jmmmp/cellblock a model of what you could use? but adapted to lists

and maybe not so complex?

...

I tried opening your first example in [text]. As I thought, the issue is

that commas are already reserved in pd for line changes. The output is this (I added a line number at the start of each line) when using carriage returns:

...

print: 0 { print: 1 "cols" : 3 print: 2 print: 3 "data" : { print: 4 "entry-0" : [ -0.06755 print: 5 0.44185 print: 6 -0.33835 ] print: 7 print: 8 "entry-1" : [ -0.12305 print: 9 -0.24085 print: 10 0.31295 ] print: 11 print: 12 "entry-2" : [ -0.0595 print: 13 -0.2881 print: 14 0.0597 ] print: 15 } print: 16 print: 17 } print: 18

Or this without them:

print: 0 { "cols" : 3 print: 1 "data" : { "entry-0" : [ -0.06755 print: 2 0.44185 print: 3 -0.33835 ] print: 4 "entry-1" : [ -0.12305 print: 5 -0.24085 print: 6 0.31295 ] print: 7 "entry-2" : [ -0.0595 print: 8 -0.2881 print: 9 0.0597 ] } } print: 10

which makes it a bit hard to read json files without some hacking, as

all those commas create a new line. Unless there is another method, is it better to code your own json interpreter?

...

One advantage of [text] is that it can be used to declare data

structures directly - examples might be in my tutorial, I'm not sure anymore how far I got with it.

...

Files are attached.

Joao

...

Thanks for the quick reply!

2 use-cases.

1- we can generate or retrieve a dataset's content to use natively in

the creative coding environment (Max Pd Sc) so that it integrates in other workflows of data mangling and drawing. We already have in our dataset object file support and single point entry and query but this allow batch dump and load. The structure is:

...

...

{ "cols" : 3, "data" : { "entry-0" : [ -0.06755, 0.44185, -0.33835 ], "entry-1" : [ -0.12305, -0.24085, 0.31295 ], "entry-2" : [ -0.0595, -0.2881, 0.0597 ] }

}

2- we can retrieve or set the state of a complex objects. Our object

will spit out its internal status ( for instance, a neural net) and we can use the values of its parameters, like below. More interestingly, we can also query its structure and use that information too.

...

...

{ "layers" : [ { "activation" : 3, "biases" : [ -3.076234734727154, 0.772760846709679

],

...

...

                 "cols" : 2,
                 "rows" : 1,
                 "weights" : [ [ 6.015551733036155,

-1.826803841455323 ] ]

...

...

         }

, { "activation" : 3, "biases" : [ -0.490600074475542 ], "cols" : 1, "rows" : 2, "weights" : [ [ -3.115116035462417 ], [

-3.969281643687132 ] ]

...

...

         }

] }

The key-value nesting is quite powerful for this type of open

structure...

...

...

...

On 7 Sep 2021, at 15:51, Christof Ressi info@christofressi.com

wrote:

...

...

...

Can you give an example of how the data is structured?

In which ways are users supposed to interact with the data?

Christof

On 07.09.2021 16:37, Pierre Alexandre Tremblay wrote:

...

Dear all

I am trying to find the most Pd-vanilla-way to interface with our

Dataset object in the FluCoMa project. In Max and SuperCollider we use Dictionaries, which are nestable and queryable in powerful programmatic ways, working essentially like interfaces to JSON-like data structures.

...

...

...

...

I’ve looked at [struct] but the [set] object does not allow to do

symbols and (list of) floats, and [appends] seem to have the same limitations. In all cases, I’m not certain it is the best approach in any cases to create such a list in Pd...

...

...

...

...

I wonder how people do it and if I am missing an obvious workflow,

especially with nested structures.

...

...

...

...

Any pointer (pun intended) welcome

p

<json-test.pd><test.json>