On Dec 7, 2007 6:44 PM, Hans-Christoph Steiner <hans@eds.org> wrote:

Since both the current Pd-SQL devs (Mike and Jamie :) have posted to this thread, I want to bring up the interface again.  What do you guys think about the two inlet interface for SQL objects?  I just find the sql/sqlend thing very strange, and it could end up being difficult to work with.  I see the interface like this: the hot inlet is for meta commands, like "open", "close", "export", "connect", etc.  Then the second, cold inlet is for building SQL queries.  Having a cold inlet devoted to building SQL queries would mean that you would not need any special "sql/sqlend" style tags at all.

Hans, I still think you are missing the point. The end tag 'endsql' and 'submitsql' are needed so that we can reinsert the commas that PD strips out of the messages. Without having the end tag, there would be no way to tell exactly where the SQL statement ends.

And as far as it being difficult to work with, I just think of the closing tag as being a replacement for a semi-colon in the SQL. Pretty much everything else between the two tags is assumed to be PURE SQL, and gets placed into the SQL buffer. As PD splits up the messages based on where the comma's fall, we are assuming that each time the 'anything' method gets called, that we have to reinsert the comma into the SQL. The doesn't happen with the first 'sql' tag.

As far as the second inlet, I can see doing that, but only for the reason that this 'syntax' is a little different than what PD normally expects. Other than that, I really don't see any reason to have a second inlet.

Instead, you'd just send messages to the cold inlet, those messages would be appended to the existing SQL message buffer.  Then when the query is ready to be sent, send a "submit" or "run" or whatever to the hot inlet, and it would run the buffer from the cold inlet and clear it.  There would also be a "clear" command to just clear the buffer without sending it.

So, then, how would we allow the ability to insert multiple SQL statements into a single buffer? The point of having the sql/endsql tag pairs is to delimit a single statement. I don't think that Jamie's external will allow multiple statements, I could be wrong, but I think it would be very desirable to have the ability to build an SQL buffer, for instance to do multiple 'inserts'. I have also worked with the idea of being able to insert arbitrary text into the buffer, so you can build dynamic SQL statements (for instance, adding [or leaving out] a 'where' clause, or a 'limit/offset' clause).

If we do it the way you are asking for, there would be no way to insert more than a single statement into the SQL buffer.

This would allow for the comma trick, etc. and I think it would fit better into Pd's message processing.  In addition, there could be a single line message for the hot inlet, like this:

"submit insert into mine (name) values ('the_text')" 

I'll help implement this if you want.  I just have a strong aversion to the "sqlend" stuff since I think it could end up being strange to use within Pd.  Plus I think these are very valuable objects to have, so I think it's important to get the interface right.  Basically, I am thinking of something along the lines of the [textfile] interface, as much as possible. Here's a sketch:

Why is it strange? We can't use the characters ',' and ';' due to PD, so we have to come up with some other way of dealing with them.

Also, how would you handle something like this... (using your method)

"submit insert into mine (id, name, something) values (1, 'santa', 'more stuff')"

How would your external know when to actually submit the SQL buffer to the database, as this is actually a series of 5 PD messages being sent to the external?

__hot inlet__

[open $1(

[close(

[export $1(   - this could be used to export the database to a .sql file

[clear (

[submit (

[submit insert into mytable (name) values ('$1') (

__cold inlet__

[insert into mytable (name) values ('$1') (

[CREATE TABLE datatable(id INTEGER, duration FLOAT, type VARCHAR, datetime DATETIME)  (

[SELECT id, ABS((duration - 1500)/1500) AS error FROM datatable ORDER BY error LIMIT 1(

And the cold inlet would actually get these PD message...

insert into mytable (name) values ('$1')

CREATE TABLE datatable(id INTEGER

duration FLOAT

type VARCHAR

datetime DATETIME

SELECT id

ABS((duration - 1500)/1500 AS error FROM datatable ODER BY ERROR LIMIT 1

Exactly how would the external know when to terminate an SQL statement and insert a semi-colon? There really is nothing there to indicate that there is more than one SQL statement.

 

Also, on a different note, I think it could be useful to have various status and meta data queries output on the second inlet.  Currently in [psql], there is just a bang on complete.  But there could be things information about the database that aren't available via SQL.

SQLite already does this. Any messages coming back from the database are output as a list on the second outlet. There are also symbols that are output, more or less as a good indication (indicating that a statement processed, for things that don't return a result set). A bang on the second outlet indicates that the current result set has been exhausted.

One other thing that I have thought about is other database layers, upon submitting an INSERT, will return the ROWID of the row that was just inserted. I was thinking that to implement this, this number could be output on the second outlet as a float.

So this would be:

bang: indicates result set done

symbol/list: status message from database

float: ROWID of the last insert

Finally, there is the source code for the Max/MSP mysql external, so I think it should be pretty easy to make a mysql version of this interface as well.  Then we'll have a nice trio of SQL options: embedded, PGSQL, and MySQL.

 

Well, as far as this goes, I think we should make every effort to have the interfaces for ALL of these objects be the same, so that these databases can be swapped on the fly.