Hello all,
It has been several months since I first inquired about this idea, and it seems there are some out there who think this would be a good idea. So, in the interest of figuring out just how this could work with PD, I would like to start a discussion about what sort of interface objects should be created.
1.0 What the objects would look like. My first thought is to design them so they operate like other already existing PD objects. While there may not be a perfect match with other objects, I am thinking more along the lines of what commands they recognize and their basic functionality.
While I don't have a specific idea for how to implement these connecting objects, I think that there would be three (or more) different PD objects instantiated to work with a database. One object to handle the connectivity, one object to handle the querying, and a third to handle any results coming from a query.
A. While there could be multiple connectivity objects, each one would point to a different database. This could be used to construct several data paths, and each connectivity object would be passed a symbol to use as a name for its data path. This name would then also have to be passed onto each query and results object to link them to a particular data path. (Does this need more explaination?)
B. There could be more than one query object, each one handling a different query type (as PD doesn't really support strings, it might be difficult to build a whole SQL statement using commands via an inlet).
C. There could also be multiple result object, each "listening" for a different parameter set.
2.0 Data Types, SQL, and PD So far, the biggest hurdle that I see with this is Data Types. As PD doesn't have much support for strings, there would either have to be special handling involved, or we just agree that a string in this context would be a PD symbol. Also, all the other datatypes (with the possible exception of a BLOB) would not really have a PD equivelent. But then again, how might binary data be stored to a BLOB from within PD?
3.0 Linking data into PD Another idea for connecting the data from the database into a PD patch, would be to map each of the fields in the database table to a particular PD variable. While this might allow for increased interaction, it would eliminate the use of any symbols returned from the database.
4.0 Version 0.0 As this is still in this design stage, there are some things that I would like to put off dealing with until there is a working set of PD objects.
A. Currently works with one database. As there are so many different types of databases out there, and most of them require additional components to setup and configure, I would like to keep this as simple as possible. Currently my idea is to use ONLY SQLite, as a backend. This would be compiled directly into the PD external, as the SQLite code implements a full SQL database in a single source file, it is pretty easy to link this up directly without any other programs to install.
B. All databases need to be designed externally. This means that no CREATE TABLE functions or other commands that would manipulate the database schema. This might be a problem is you need to create tables on the fly, but I would think that we could get a good start on the PD interface, and then take this idea up later.
***************
So, if any of you have any ideas on this topic, please feel free to jump in and add your $0.02. It would be very much appreciated.
Thanks,
Mike
Hi,
this is interesting because I am fiddling about with exactly these sorts of things at precisely this moment.
There was the claim recently that an external was not necessary because pyext exists and SQL stuff can be done in python. However my experience in getting pyext to work has been negative. However I have happily made an external python program talk to a SQL database as well as speaking OSC to talk with PD and Max and many other things.
So I would claim that OSC makes this idea of an external less pressing.
However I would have preferred not to have had to jump to another program.
So I would find a PD sql external great!
I would also not like to have the database integrated into the PD external. One of the important things about a database is that it can be accessed by several processes. One process can be dumping data into it from somewhere, another can retrieve that data and use it for something in real time or later or whatever. So I would prefer to have an interface to an external database. However having a PD external that is a database server might be relevant for those who do not wish to install mySQL or some other database server on their machine.
I would think that functionality would include:
connecting to a database server selecting which database to use creating datasets in a table retrieving data sets from a table
This would probably be best done as two objects, one sending data to store in the DB, one receiving it in response to the queries, with both objects having commands to open a connection and select which database to use.
I agree that creating tables is not necessary, at least not initially.
Data types: everything that is not a number (which in PD is a float by definition) could be a string (strings, dates, times) which would be a symbol, with the probable exception of binary data. No idea what to do with this is PD, perhaps just ignore it? Auxiliary functions to turn a sql date and time strings into lists and vice versa would be useful.
Hmm. that is my 0.02 Euro for tonight.
Cheers,
tim
On 05/11/2007, at 10:26 PM, Mike McGonagle wrote:
Hello all,
It has been several months since I first inquired about this idea, and it seems there are some out there who think this would be a good idea. So, in the interest of figuring out just how this could work with PD, I would like to start a discussion about what sort of interface objects should be created.
1.0 What the objects would look like. My first thought is to design them so they operate like other already existing PD objects. While there may not be a perfect match with other objects, I am thinking more along the lines of what commands they recognize and their basic functionality.
While I don't have a specific idea for how to implement these connecting objects, I think that there would be three (or more) different PD objects instantiated to work with a database. One object to handle the connectivity, one object to handle the querying, and a third to handle any results coming from a query.
A. While there could be multiple connectivity objects, each one would point to a different database. This could be used to construct several data paths, and each connectivity object would be passed a symbol to use as a name for its data path. This name would then also have to be passed onto each query and results object to link them to a particular data path. (Does this need more explaination?)
B. There could be more than one query object, each one handling a different query type (as PD doesn't really support strings, it might be difficult to build a whole SQL statement using commands via an inlet).
C. There could also be multiple result object, each "listening" for a different parameter set.
2.0 Data Types, SQL, and PD So far, the biggest hurdle that I see with this is Data Types. As PD doesn't have much support for strings, there would either have to be special handling involved, or we just agree that a string in this context would be a PD symbol. Also, all the other datatypes (with the possible exception of a BLOB) would not really have a PD equivelent. But then again, how might binary data be stored to a BLOB from within PD?
3.0 Linking data into PD Another idea for connecting the data from the database into a PD patch, would be to map each of the fields in the database table to a particular PD variable. While this might allow for increased interaction, it would eliminate the use of any symbols returned from the database.
4.0 Version 0.0 As this is still in this design stage, there are some things that I would like to put off dealing with until there is a working set of PD objects.
A. Currently works with one database. As there are so many different types of databases out there, and most of them require additional components to setup and configure, I would like to keep this as simple as possible. Currently my idea is to use ONLY SQLite, as a backend. This would be compiled directly into the PD external, as the SQLite code implements a full SQL database in a single source file, it is pretty easy to link this up directly without any other programs to install.
B. All databases need to be designed externally. This means that no CREATE TABLE functions or other commands that would manipulate the database schema. This might be a problem is you need to create tables on the fly, but I would think that we could get a good start on the PD interface, and then take this idea up later.
So, if any of you have any ideas on this topic, please feel free to jump in and add your $0.02. It would be very much appreciated.
Thanks,
Mike
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Tim Boykett schrieb:
Hi,
this is interesting because I am fiddling about with exactly these sorts of things at precisely this moment.
There was the claim recently that an external was not necessary because pyext exists and SQL stuff can be done in python. However my experience in getting pyext to work has been negative. However I have happily made an external python program talk to a SQL database as well as speaking OSC to talk with PD and Max and many other things.
What exactly was your problem with pyext?
greetings, Thomas
I think it would be really nice to have a Pd-ish interface to MySQL, I am glad you are taking this project on. Here are some thoughts:
- As for the objects, I think one idea is to have one object that is the connection to the database, then everything gets sent thru that. Then you can have supporting objects that help build queries, etc. which are then fed to the connection object. The results will come out of the outlet of the connection object, and there again can be objects to parse the results, if need be.
- I think in order to make it work well, you are going to need to use a thread so that Pd doesn't hang waiting for a response from the database. That means the interface needs to represent that the object is not deterministic. That could be a bit tricky. For example, you send a command to [mysql], then send another request before the first one is finished, then you get a response. How do you know which request that response belongs to? Normally this isn't a problem in Pd since basically everything responds within its time slice, so it acts as if everything is instantaneous.
- If you need strings, you should look at the two different string methods that exist: moocow's pdstring and the mrpeach's string datatype that's included in the most recent pd-extended builds. You might be able to get away with using lists and just insert spaces between the atoms before sending to the database.
- I think it would be more useful to make this object an interface to existing databases than just an embedded sqlite object. There is already [pool] and PDContainer which basically do that. Talking to external databases means that you could power a website with Pd :).
.hc
On Nov 5, 2007, at 5:33 PM, Tim Boykett wrote:
Hi,
this is interesting because I am fiddling about with exactly these sorts of things at precisely this moment.
There was the claim recently that an external was not necessary because pyext exists and SQL stuff can be done in python. However my experience in getting pyext to work has been negative. However I have happily made an external python program talk to a SQL database as well as speaking OSC to talk with PD and Max and many other things.
So I would claim that OSC makes this idea of an external less pressing.
However I would have preferred not to have had to jump to another program.
So I would find a PD sql external great!
I would also not like to have the database integrated into the PD external. One of the important things about a database is that it can be accessed by several processes. One process can be dumping data into it from somewhere, another can retrieve that data and use it for something in real time or later or whatever. So I would prefer to have an interface to an external database. However having a PD external that is a database server might be relevant for those who do not wish to install mySQL or some other database server on their machine.
I would think that functionality would include:
connecting to a database server selecting which database to use creating datasets in a table retrieving data sets from a table
This would probably be best done as two objects, one sending data to store in the DB, one receiving it in response to the queries, with both objects having commands to open a connection and select which database to use.
I agree that creating tables is not necessary, at least not initially.
Data types: everything that is not a number (which in PD is a float by definition) could be a string (strings, dates, times) which would be a symbol, with the probable exception of binary data. No idea what to do with this is PD, perhaps just ignore it? Auxiliary functions to turn a sql date and time strings into lists and vice versa would be useful.
Hmm. that is my 0.02 Euro for tonight.
Cheers,
tim
On 05/11/2007, at 10:26 PM, Mike McGonagle wrote:
Hello all,
It has been several months since I first inquired about this idea, and it seems there are some out there who think this would be a good idea. So, in the interest of figuring out just how this could work with PD, I would like to start a discussion about what sort of interface objects should be created.
1.0 What the objects would look like. My first thought is to design them so they operate like other already existing PD objects. While there may not be a perfect match with other objects, I am thinking more along the lines of what commands they recognize and their basic functionality.
While I don't have a specific idea for how to implement these connecting objects, I think that there would be three (or more) different PD objects instantiated to work with a database. One object to handle the connectivity, one object to handle the querying, and a third to handle any results coming from a query.
A. While there could be multiple connectivity objects, each one would point to a different database. This could be used to construct several data paths, and each connectivity object would be passed a symbol to use as a name for its data path. This name would then also have to be passed onto each query and results object to link them to a particular data path. (Does this need more explaination?)
B. There could be more than one query object, each one handling a different query type (as PD doesn't really support strings, it might be difficult to build a whole SQL statement using commands via an inlet).
C. There could also be multiple result object, each "listening" for a different parameter set.
2.0 Data Types, SQL, and PD So far, the biggest hurdle that I see with this is Data Types. As PD doesn't have much support for strings, there would either have to be special handling involved, or we just agree that a string in this context would be a PD symbol. Also, all the other datatypes (with the possible exception of a BLOB) would not really have a PD equivelent. But then again, how might binary data be stored to a BLOB from within PD?
3.0 Linking data into PD Another idea for connecting the data from the database into a PD patch, would be to map each of the fields in the database table to a particular PD variable. While this might allow for increased interaction, it would eliminate the use of any symbols returned from the database.
4.0 Version 0.0 As this is still in this design stage, there are some things that I would like to put off dealing with until there is a working set of PD objects.
A. Currently works with one database. As there are so many different types of databases out there, and most of them require additional components to setup and configure, I would like to keep this as simple as possible. Currently my idea is to use ONLY SQLite, as a backend. This would be compiled directly into the PD external, as the SQLite code implements a full SQL database in a single source file, it is pretty easy to link this up directly without any other programs to install.
B. All databases need to be designed externally. This means that no CREATE TABLE functions or other commands that would manipulate the database schema. This might be a problem is you need to create tables on the fly, but I would think that we could get a good start on the PD interface, and then take this idea up later.
So, if any of you have any ideas on this topic, please feel free to jump in and add your $0.02. It would be very much appreciated.
Thanks,
Mike
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
------------------------------------------------------------------------ ----
Using ReBirth is like trying to play an 808 with a long stick. - David Zicarelli
On Mon, 2007-11-05 at 22:08 -0500, Hans-Christoph Steiner wrote:
I think it would be really nice to have a Pd-ish interface to MySQL, I am glad you are taking this project on. Here are some thoughts:
- As for the objects, I think one idea is to have one object that is
the connection to the database, then everything gets sent thru that. Then you can have supporting objects that help build queries, etc. which are then fed to the connection object. The results will come out of the outlet of the connection object, and there again can be objects to parse the results, if need be.
I agree that this is the right architecture. WRT the 'connection handler' object, I had an idea that wrapping one of the database abstraction libraries might be the way to go. Perhaps libdbi (http://libdbi.sourceforge.net/).
- I think in order to make it work well, you are going to need to use
a thread so that Pd doesn't hang waiting for a response from the database. That means the interface needs to represent that the object is not deterministic. That could be a bit tricky. For example, you send a command to [mysql], then send another request before the first one is finished, then you get a response. How do you know which request that response belongs to? Normally this isn't a problem in Pd since basically everything responds within its time slice, so it acts as if everything is instantaneous.
I don't understand what you mean here. Why would using threading stop the object from being deterministic? Surely you would have one thread for the database connection, and get results back in the order in that the queries were sent in. Or were you thinking of having a multithreaded object with one connection per thread?
Jamie
On Nov 6, 2007, at 5:40 AM, Jamie Bullock wrote:
On Mon, 2007-11-05 at 22:08 -0500, Hans-Christoph Steiner wrote:
I think it would be really nice to have a Pd-ish interface to MySQL, I am glad you are taking this project on. Here are some thoughts:
- As for the objects, I think one idea is to have one object that is
the connection to the database, then everything gets sent thru that. Then you can have supporting objects that help build queries, etc. which are then fed to the connection object. The results will come out of the outlet of the connection object, and there again can be objects to parse the results, if need be.
I agree that this is the right architecture. WRT the 'connection handler' object, I had an idea that wrapping one of the database abstraction libraries might be the way to go. Perhaps libdbi (http://libdbi.sourceforge.net/).
- I think in order to make it work well, you are going to need to use
a thread so that Pd doesn't hang waiting for a response from the database. That means the interface needs to represent that the object is not deterministic. That could be a bit tricky. For example, you send a command to [mysql], then send another request before the first one is finished, then you get a response. How do you know which request that response belongs to? Normally this isn't a problem in Pd since basically everything responds within its time slice, so it acts as if everything is instantaneous.
I don't understand what you mean here. Why would using threading stop the object from being deterministic? Surely you would have one thread for the database connection, and get results back in the order in that the queries were sent in. Or were you thinking of having a multithreaded object with one connection per thread?
Pre-emptive threads are by their very nature non-deterministic because there is no way to guarantee that the things in different threads will execute with the same order everytime. Yes, the order that you send the messages won't change if you have one thread for the message sending, but once you have the thread, you can't guarantee that the database will return it's answer within on logical Pd clock tick. Pd/Max is built around this idea, that each object does it's thing within one clock tick.
.hc
------------------------------------------------------------------------ ----
All mankind is of one author, and is one volume; when one man dies, one chapter is not torn out of the book, but translated into a better language; and every chapter must be so translated.... -John Donne
Hallo, Hans-Christoph Steiner hat gesagt: // Hans-Christoph Steiner wrote:
Pre-emptive threads are by their very nature non-deterministic because there is no way to guarantee that the things in different threads will execute with the same order everytime. Yes, the order that you send the messages won't change if you have one thread for the message sending, but once you have the thread, you can't guarantee that the database will return it's answer within on logical Pd clock tick. Pd/Max is built around this idea, that each object does it's thing within one clock tick.
I wonder, what use is a database result if it comes five minutes after I sent the retrieval command? The only use for such an object would be to slowly fill another container like [textfile], which has guaranteed, deterministic response time. OTOH if a db-object would be allowed to send results to its outlets non-deterministically, it would be useless as a replacement for deterministic containers like [textfile]. So we'd need to versions of that db-class, either two classes or one class with a switch. I think, the deterministic version is more useful, and if you use a fast/local DB, it's about as reliable as hard disk access or netreceive.
Ciao
On Nov 7, 2007, at 4:37 AM, Frank Barknecht wrote:
Hallo, Hans-Christoph Steiner hat gesagt: // Hans-Christoph Steiner wrote:
Pre-emptive threads are by their very nature non-deterministic because there is no way to guarantee that the things in different threads will execute with the same order everytime. Yes, the order that you send the messages won't change if you have one thread for the message sending, but once you have the thread, you can't guarantee that the database will return it's answer within on logical Pd clock tick. Pd/Max is built around this idea, that each object does it's thing within one clock tick.
I wonder, what use is a database result if it comes five minutes after I sent the retrieval command? The only use for such an object would be to slowly fill another container like [textfile], which has guaranteed, deterministic response time. OTOH if a db-object would be allowed to send results to its outlets non-deterministically, it would be useless as a replacement for deterministic containers like [textfile]. So we'd need to versions of that db-class, either two classes or one class with a switch. I think, the deterministic version is more useful, and if you use a fast/local DB, it's about as reliable as hard disk access or netreceive.
AFAIK, MySQL and the like do not guarantee realtime responses, and network connections certainly do not. So if want something that is designed to do that, then use [pool], [textfile], or perhaps an embedded SQLlite, if it is quick enough.
Unless there are ways to guarantee the database will reply within one clock tick (usually ~1.5ms), then the interface should represent the reality of the situation. Otherwise the database object will cause lots of clicks and interruptions when used.
.hc
Ciao
Frank Barknecht _ ______footils.org__
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
------------------------------------------------------------------------ ----
Mistrust authority - promote decentralization. - the hacker ethic
AFAIK, MySQL and the like do not guarantee realtime responses, and network connections certainly do not. So if want something that is designed to do that, then use [pool], [textfile], or perhaps an embedded SQLlite, if it is quick enough.
By the way... although py/pyext might be hard to build compared to pdlua it can detach its execution from the pd thread and deliver late responses without halting the execution of pd.
gr~~~
Hallo, Hans-Christoph Steiner hat gesagt: // Hans-Christoph Steiner wrote:
AFAIK, MySQL and the like do not guarantee realtime responses, and network connections certainly do not. So if want something that is designed to do that, then use [pool], [textfile], or perhaps an embedded SQLlite, if it is quick enough.
Unless there are ways to guarantee the database will reply within one clock tick (usually ~1.5ms), then the interface should represent the reality of the situation. Otherwise the database object will cause lots of clicks and interruptions when used.
SQLite is fast. In most cases, MySQL etc. will be fast enough as well, but of course no common DB gives realtime guarantees. But neither does [textfile], e.g. when responding to the "read"-command: Loading large textfiles will halt execution as well. (Btw: An important use of SQL in Pd would be as a replacement for [textfile] if you need to parse tons of data, like gigabytes, as you can read row by row in nanoseconds with a realtime response that would be better than [textile] by miles.)
Pyext's approach of offering threading as an option is the most flexible and it's what first inspired my remark. If you want to use a sql-db as a way to store and load data for use in your logic, you often *need* deterministic behaviour. Of course then it's in your own responsibility to make sure you meet the deadlines, for example by optimizing queries, using fast, local DBs etc.
But the need to do deterministic queries sometimes (or rather often) must not be ignored by designing a SQL class for Pd with only threaded use in mind.
As databases can be used in so many different ways, I still favour the DIY approach using Python or Lua or so anyway. It's again a use case, where not the interpreted language is the bottleneck, but an outside factor. A C-class for SQL querying wouldn't be any faster than the (existing) Python/Lua classes, if the database is too slow.
Ciao
On Nov 7, 2007, at 2:35 PM, Frank Barknecht wrote:
Hallo, Hans-Christoph Steiner hat gesagt: // Hans-Christoph Steiner wrote:
AFAIK, MySQL and the like do not guarantee realtime responses, and network connections certainly do not. So if want something that is designed to do that, then use [pool], [textfile], or perhaps an embedded SQLlite, if it is quick enough.
Unless there are ways to guarantee the database will reply within one clock tick (usually ~1.5ms), then the interface should represent the reality of the situation. Otherwise the database object will cause lots of clicks and interruptions when used.
SQLite is fast. In most cases, MySQL etc. will be fast enough as well, but of course no common DB gives realtime guarantees. But neither does [textfile], e.g. when responding to the "read"-command: Loading large textfiles will halt execution as well. (Btw: An important use of SQL in Pd would be as a replacement for [textfile] if you need to parse tons of data, like gigabytes, as you can read row by row in nanoseconds with a realtime response that would be better than [textile] by miles.)
Pyext's approach of offering threading as an option is the most flexible and it's what first inspired my remark. If you want to use a sql-db as a way to store and load data for use in your logic, you often *need* deterministic behaviour. Of course then it's in your own responsibility to make sure you meet the deadlines, for example by optimizing queries, using fast, local DBs etc.
But the need to do deterministic queries sometimes (or rather often) must not be ignored by designing a SQL class for Pd with only threaded use in mind.
As databases can be used in so many different ways, I still favour the DIY approach using Python or Lua or so anyway. It's again a use case, where not the interpreted language is the bottleneck, but an outside factor. A C-class for SQL querying wouldn't be any faster than the (existing) Python/Lua classes, if the database is too slow.
Using threads in a Pd objects without representing it in the interface is a recipe for very difficult bugs. Talk to Max/MSP about bugs in "Overdrive" mode. Most people solve them by turning off overdrive, from what I hear.
Making the interface reflect the threaded nature doesn't have to be complicated. It could just be that you give each query a message ID which is prepended to the result. A deterministic database query object will cause interruptions, especially if there is any network activity involved. It would be a shame to have a database object that couldn't be used with realtime audio, video, etc.
I think the bigger issue is that people want to use Pd to interface all sorts of things that can't be deterministic. So it would be good if we figure out how to make that work well in the context of programming in Pd. "It's pretty good, but it interrupts sometimes" doesn't sound like a goal to strive for.
.hc
------------------------------------------------------------------------ ----
You can't steal a gift. Bird gave the world his music, and if you can hear it, you can have it. - Dizzy Gillespie
Hallo, Tim Boykett hat gesagt: // Tim Boykett wrote:
There was the claim recently that an external was not necessary because pyext exists and SQL stuff can be done in python. However my experience in getting pyext to work has been negative. However I have happily made an external python program talk to a SQL database as well as speaking OSC to talk with PD and Max and many other things.
pdlua is easier to build than pyext so you may try Claude's pdlua with luasql: http://www.keplerproject.org/luasql/
Ciao
Hallo, Frank Barknecht hat gesagt: // Frank Barknecht wrote:
pdlua is easier to build than pyext so you may try Claude's pdlua with luasql: http://www.keplerproject.org/luasql/
So, I hacked together a quick example with not much error checking. Tested on Debian with the package: liblua5.1-sql-sqlite2
(Claude, feel free to include it in pdlua with whatever license.)
Ciao
Hi,
On Mon, 2007-11-05 at 23:33 +0100, Tim Boykett wrote: [snip]
So I would find a PD sql external great!
Well there is one. It's called [sqlsingle] by Iain Mott (http://www.reverberant.com/PD/index.htm).
This is basically a thin wrapper around the libpq PostgreSQL library, allowing you to perform a variety of queries using PD messages: basically CRUD.
I have also made a modified version called [psql], which I thought I committed to CVS ages ago, but obviously didn't - It's in there now. [psql] removes the 'single' nature of the external making it significantly faster for multiple queries ([sqlsingle] flushes the database after every query). I've also added a Mac OS build target, and renamed the object to [psql], which seems more accurate.
I would think that functionality would include:
connecting to a database server selecting which database to use creating datasets in a table retrieving data sets from a table
See above. I am happy to maintain this, so if you have any problems, let me know (unless you want to send a patch).
BTW, I do agree that this is not the best solution for PD/database connectivity, and that a more modular approach would be better.
Jamie
On Mon, 5 Nov 2007 23:33:42 +0100 Tim Boykett tim@timesup.org wrote:
One of the important things about a database is that it can be accessed by several processes. One process can be dumping data into it from somewhere, another can retrieve that data and use it for something in real time or later or whatever. So I would prefer to have an interface to an external database.
A nice general model might be the way Perls Class::DBI (or even tie) work, as object serialisers with templates like flatfile, SQL, XML to load for whatever data backend you want.
I think its a great idea and will make many possibilities.
As for how to use it?
[textfile] has a simple buffer you can fill up with <cr> delimited lines. Keep it simple with an insert cursor from either side, and a send message that does any handle opening and dispatches or reads the data. The query should be completely left to the user to do with messages and depend on the template loaded.
It would be nice to easily dump and load multiple arrays to a database for wavetables, so don't ignore binary data blobs or even the idea of an audio rate [stream2db~].
I like the two object model, it fits with other Pd object patterns.
Maybe think about locking/contention for multiple connection objects, only one usable read write pair would be annoying.
MySQLite might be a good choice to go for first.
2c
Hi Mike,
I wonder what things would be like if you could suck SQL data directly into data structures. This way they could be highly structured and perhaps altered in PD.
I've often thought about data structures to store data only, and not graphics, which I suppose would be a good idea with lots of lots of data in it.
Being able to put a huge table into a gridflow matrix (of symbols) could be interesting, I don't know if you can have a grid of symbols (or strings) in gridflow...
.b.
Mike McGonagle wrote:
Hello all,
It has been several months since I first inquired about this idea, and it seems there are some out there who think this would be a good idea. So, in the interest of figuring out just how this could work with PD, I would like to start a discussion about what sort of interface objects should be created.
1.0 What the objects would look like. My first thought is to design them so they operate like other already existing PD objects. While there may not be a perfect match with other objects, I am thinking more along the lines of what commands they recognize and their basic functionality.
While I don't have a specific idea for how to implement these connecting objects, I think that there would be three (or more) different PD objects instantiated to work with a database. One object to handle the connectivity, one object to handle the querying, and a third to handle any results coming from a query.
A. While there could be multiple connectivity objects, each one would point to a different database. This could be used to construct several data paths, and each connectivity object would be passed a symbol to use as a name for its data path. This name would then also have to be passed onto each query and results object to link them to a particular data path. (Does this need more explaination?)
B. There could be more than one query object, each one handling a different query type (as PD doesn't really support strings, it might be difficult to build a whole SQL statement using commands via an inlet).
C. There could also be multiple result object, each "listening" for a different parameter set.
2.0 Data Types, SQL, and PD So far, the biggest hurdle that I see with this is Data Types. As PD doesn't have much support for strings, there would either have to be special handling involved, or we just agree that a string in this context would be a PD symbol. Also, all the other datatypes (with the possible exception of a BLOB) would not really have a PD equivelent. But then again, how might binary data be stored to a BLOB from within PD?
3.0 Linking data into PD Another idea for connecting the data from the database into a PD patch, would be to map each of the fields in the database table to a particular PD variable. While this might allow for increased interaction, it would eliminate the use of any symbols returned from the database.
4.0 Version 0.0 As this is still in this design stage, there are some things that I would like to put off dealing with until there is a working set of PD objects.
A. Currently works with one database. As there are so many different types of databases out there, and most of them require additional components to setup and configure, I would like to keep this as simple as possible. Currently my idea is to use ONLY SQLite, as a backend. This would be compiled directly into the PD external, as the SQLite code implements a full SQL database in a single source file, it is pretty easy to link this up directly without any other programs to install.
B. All databases need to be designed externally. This means that no CREATE TABLE functions or other commands that would manipulate the database schema. This might be a problem is you need to create tables on the fly, but I would think that we could get a good start on the PD interface, and then take this idea up later.
So, if any of you have any ideas on this topic, please feel free to jump in and add your $0.02. It would be very much appreciated.
Thanks,
Mike
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Hallo, B. Bogart hat gesagt: // B. Bogart wrote:
I've often thought about data structures to store data only, and not graphics, which I suppose would be a good idea with lots of lots of data in it.
Oh, I use data structures for that a lot, it's very useful. Even when Pd's data structures are implemented as *graphical* data structures, in the first place they are *data structures* liks "struct" in C and can be used in many ways without their graphical representation to do calculations like cellular automata or L-Systems.
Ciao
-
Andy Farnell -
B. Bogart -
Frank Barknecht -
Hans-Christoph Steiner -
Jamie Bullock -
Mike McGonagle -
Thomas Grill -
Tim Boykett