On Sep 28, 2011, at 3:15 AM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-09-27 21:55, Hans-Christoph Steiner wrote:
That's a good break down of the issues. I'll checkout @loader_path. The @executable_path technique has been in Pd-extended for a long time, check out all of the libs in Pd-extended.app/Contents/lib by doing something like:
otool -L /Applications/Pd-extended.app/Contents/lib/libavcodec. 51.dylib
But this still leaves us with Windows and GNU/Linux, right? I think the original idea of setting the LD_LIBRARY_PATH when loading, then unsetting, is a good one. It would be good to have all platforms use the same technique.
seems like my other mail got unnoticed: i find that the LD_LIBRARY_PATH trick does NOT work as expected, as this variable is only evaluated at strttime of the application, and setting it via setenv() seems to not effect dlopen().
Ah, ok, that changes things. So you are sure there isn't a way to make this work? If so, do you think there are other shared library techniques you think could work? Otherwise, it looks like we are back to my original manual dlopen() proposal.
.hc
----------------------------------------------------------------------------
As we enjoy great advantages from inventions of others, we should be glad of an opportunity to serve others by any invention of ours; and this we should do freely and generously. - Benjamin Franklin
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/28/2011 03:07 PM, Hans-Christoph Steiner wrote:
Ah, ok, that changes things. So you are sure there isn't a way to make this work?
on, i'm not sure. i only reported issues i came across.
If so, do you think there are other shared library techniques you think could work?
i hope, but i do not know.
Otherwise, it looks like we are back to my original manual dlopen() proposal.
i'm still no friend of this. i'm afraid that all real-life solutions that involve dlopen(), will require significantly more code to be crammed into each external (only to make this work) than will be put into the "shared code library".
which would make the entire attempt futile.
fgmasdr IOhannes
On Sep 29, 2011, at 2:49 AM, IOhannes m zmölnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/28/2011 03:07 PM, Hans-Christoph Steiner wrote:
Ah, ok, that changes things. So you are sure there isn't a way to make this work?
on, i'm not sure. i only reported issues i came across.
If so, do you think there are other shared library techniques you think could work?
i hope, but i do not know.
Otherwise, it looks like we are back to my original manual dlopen() proposal.
i'm still no friend of this. i'm afraid that all real-life solutions that involve dlopen(), will require significantly more code to be crammed into each external (only to make this work) than will be put into the "shared code library".
which would make the entire attempt futile.
So how about the chdir technique?
.hc
----------------------------------------------------------------------------
Access to computers should be unlimited and total. - the hacker ethic
On Thursday, September 29, 2011 8:49 AM, "IOhannes m zmölnig" zmoelnig@iem.at wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/28/2011 03:07 PM, Hans-Christoph Steiner wrote:
Ah, ok, that changes things. So you are sure there isn't a way to make this work?
on, i'm not sure. i only reported issues i came across.
If so, do you think there are other shared library techniques you think could work?
i hope, but i do not know.
Otherwise, it looks like we are back to my original manual dlopen() proposal.
i'm still no friend of this. i'm afraid that all real-life solutions that involve dlopen(), will require significantly more code to be crammed into each external (only to make this work) than will be put into the "shared code library".
which would make the entire attempt futile.
Just tested with tkwidgets on MinGW. It works fine. It builds a libtkwidgets.dll, and entry.dll links against it. And it seems that Windows automatically looks in the same folder for other dependent DLLs. And Mac OS X is working using @loader_path.
So we just need to figure out GNU/Linux and we have it. tkwidgets builds and links to libtkwidgets.so, and if I set LD_LIBRARY_PATH before running Pd, it finds libtkwidgets.so. Anyone have any ideas?
.hc
On Thursday, September 29, 2011 8:49 AM, "IOhannes m zmölnig" zmoelnig@iem.at wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/28/2011 03:07 PM, Hans-Christoph Steiner wrote:
Ah, ok, that changes things. So you are sure there isn't a way to make this work?
on, i'm not sure. i only reported issues i came across.
If so, do you think there are other shared library techniques you think could work?
i hope, but i do not know.
Otherwise, it looks like we are back to my original manual dlopen() proposal.
i'm still no friend of this. i'm afraid that all real-life solutions that involve dlopen(), will require significantly more code to be crammed into each external (only to make this work) than will be put into the "shared code library".
which would make the entire attempt futile.
I found something promising for GNU/Linux to complete this project: http://grahamwideman.wordpress.com/2009/02/09/the-linux-loader-and-how-it-fi...
DT_RUNPATH: Using the DT_RUNPATH dynamic section attribute of the binary if present. (Ie: the executable can provide a list of paths t search for objects to load. However, DT_RUNPATH is not applied at the point those objects load other objects. — GW)
.hc
On Tuesday, October 18, 2011 6:50 PM, "Hans-Christoph Steiner" hans@at.or.at wrote:
On Thursday, September 29, 2011 8:49 AM, "IOhannes m zmölnig" zmoelnig@iem.at wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/28/2011 03:07 PM, Hans-Christoph Steiner wrote:
Ah, ok, that changes things. So you are sure there isn't a way to make this work?
on, i'm not sure. i only reported issues i came across.
If so, do you think there are other shared library techniques you think could work?
i hope, but i do not know.
Otherwise, it looks like we are back to my original manual dlopen() proposal.
i'm still no friend of this. i'm afraid that all real-life solutions that involve dlopen(), will require significantly more code to be crammed into each external (only to make this work) than will be put into the "shared code library".
which would make the entire attempt futile.
I found something promising for GNU/Linux to complete this project: http://grahamwideman.wordpress.com/2009/02/09/the-linux-loader-and-how-it-fi...
DT_RUNPATH: Using the DT_RUNPATH dynamic section attribute of the binary if present. (Ie: the executable can provide a list of paths t search for objects to load. However, DT_RUNPATH is not applied at the point those objects load other objects. — GW)
.hc
Yes! I got it working. I see the way to use it for this situation, basically set DT_RUNPATH in the .pd_linux file to the special string $ORIGIN. $ORIGIN is then replaced with the CWD of the .pd_linux file. This was the source of info that finally nailed this down for me:
http://blog.lxgcc.net/?tag=dt_runpath
The set of directories specified by a given DT_RUNPATH entry is used to find only the immediate dependencies of the executable or shared object containing the DT_RUNPATH entry. That is, it is used only for those dependencies contained in the DT_NEEDED entries of the dynamic structure containing the DT_RUNPATH entry, itself. One object's DT_RUNPATH entry does not affect the search for any other object's dependencies. http://www.sco.com/developers/gabi/2000-07-17/ch5.dynamic.html#shobj_depende...
I committed the change to tkwidgets and the template/Makefile.
.hc
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-19 01:29, Hans-Christoph Steiner wrote:
Yes! I got it working. I see the way to use it for this situation,
this looks like great news!
fgmadr IOhannes
On Oct 19, 2011, at 3:30 AM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-19 01:29, Hans-Christoph Steiner wrote:
Yes! I got it working. I see the way to use it for this situation,
this looks like great news!
Does it work for you?
.hc
----------------------------------------------------------------------------
Terrorism is not an enemy. It cannot be defeated. It's a tactic. It's about as sensible to say we declare war on night attacks and expect we're going to win that war. We're not going to win the war on terrorism. - retired U.S. Army general, William Odom
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-19 18:33, Hans-Christoph Steiner wrote:
Does it work for you?
it seems to work ok. at least the objects are loading (though i get a lot of tcl/tk errors with some of the examples, but i guess that's unrelated to the main problem)
fgmasdr IOhannes
On Oct 20, 2011, at 9:32 AM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-19 18:33, Hans-Christoph Steiner wrote:
Does it work for you?
it seems to work ok. at least the objects are loading (though i get a lot of tcl/tk errors with some of the examples, but i guess that's unrelated to the main problem)
Yeah, those objects are rough. I just ported the 'chaos' lib to this, you can try it at: externals/chaos
.hc
----------------------------------------------------------------------------
"[W]e have invented the technology to eliminate scarcity, but we are deliberately throwing it away to benefit those who profit from scarcity." -John Gilmore
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 10/19/2011 01:29 AM, Hans-Christoph Steiner wrote:
I committed the change to tkwidgets and the template/Makefile.
i tried to use the new template/Makefile for this feature and noticed a few issues: - - object sources are called SOURCES, whereas shared sources are called SHARED_SOURCE (singular); same goes for SHARED_HEADER for consistency sake, i propose to use plural throughout.
- - the output name of the shared lib cannot be simply overridden (it derives from the value of SHARED_SOURCE)
- - using multiple C-files for the shared code, doesn't really work: it tries to create a shared library from each C-file (so if you SHARED_SOURCE="foo.c bar.c", you will end up with both foo.so and bar.so, and your objecctclass will link against all those. it seems like you only used a "single C-file" library, but i guess a number of libs will also want to use multiple C-files for building a single library, which should then be linked to all objectclasses. (this get's you all the features from creating a shared lib from each SHARED_SOURCE file, and additionally allows the shared code to share code with itself)
therefore i propose a few small amendments to template/Makefile: - - rename SHARED_SOURCE to SHARED_SOURCES - - rename SHARED_HEADER to SHARED_HEADERS - - build all shared sources into a single library - - as default name, use "lib$(LIBRARY).so" for this library - - allow the user to easily override this name, if they want to
mdfgt IOhannes
On Oct 29, 2011, at 10:26 AM, IOhannes m zmölnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 10/19/2011 01:29 AM, Hans-Christoph Steiner wrote:
I committed the change to tkwidgets and the template/Makefile.
i tried to use the new template/Makefile for this feature and noticed a few issues:
- object sources are called SOURCES, whereas shared sources are called
SHARED_SOURCE (singular); same goes for SHARED_HEADER for consistency sake, i propose to use plural throughout.
- the output name of the shared lib cannot be simply overridden (it
derives from the value of SHARED_SOURCE)
- using multiple C-files for the shared code, doesn't really work: it
tries to create a shared library from each C-file (so if you SHARED_SOURCE="foo.c bar.c", you will end up with both foo.so and bar.so, and your objecctclass will link against all those. it seems like you only used a "single C-file" library, but i guess a number of libs will also want to use multiple C-files for building a single library, which should then be linked to all objectclasses. (this get's you all the features from creating a shared lib from each SHARED_SOURCE file, and additionally allows the shared code to share code with itself)
therefore i propose a few small amendments to template/Makefile:
- rename SHARED_SOURCE to SHARED_SOURCES
- rename SHARED_HEADER to SHARED_HEADERS
- build all shared sources into a single library
- as default name, use "lib$(LIBRARY).so" for this library
- allow the user to easily override this name, if they want to
These things are really all by design. This makefile template is not trying to be all things to all people, but instead provide the easiest solution for 98% of the cases. By not handling that 2%, it can be make much simpler for everyone else.
- SHARED_SOURCE is lib$(LIBRARY_NAME).c, if you need multiple files, you need to modify things
- the shared lib name is deliberately standardized, therefore its not settable
This means that a dev can easily tell whether a lib contains a shared lib, and where its source code is. Its self-documenting once you know it. You just look at the file system, and you know where things are.
.hc
----------------------------------------------------------------------------
The arc of history bends towards justice. - Dr. Martin Luther King, Jr.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 10/29/2011 04:35 PM, Hans-Christoph Steiner wrote:
These things are really all by design. This makefile template is not trying to be all things to all people, but instead provide the easiest solution for 98% of the cases. By not handling that 2%, it can be make much simpler for everyone else.
- SHARED_SOURCE is lib$(LIBRARY_NAME).c, if you need multiple files, you need to modify things
but why is it then that the SHARED_SOURCE is assigned with "?=", indicating that the user should be able to set the files to whatever fits their needs?
why is it not documented?
- the shared lib name is deliberately standardized, therefore its not settable
the thing is, it is _not_ standardized, it uses whatever SHARED_SOURCE is set to (and replaces .c by .$EXT), which is settable, and therefore can be set to multiple source files, making the template/Makefile simply not work.
it is exactly my suggestion to standardize it to lib$(LIBNAME).$(EXT).
This means that a dev can easily tell whether a lib contains a shared lib, and where its source code is. Its self-documenting once you know it. You just look at the file system, and you know where things are.
what is wrong with my suggestions? i really have the feeling, that it is mainly rejected because the suggestion comes from me (e.g. "not trying to be all things to all people" basically boils down to "please don't fit into IOs little world")
mfgasdr IOhannes
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 10/30/2011 09:13 PM, IOhannes m zmölnig wrote:
it is exactly my suggestion to standardize it to lib$(LIBNAME).$(EXT).
to be more precise, my suggestion basically contains exactly 2 suggestions: - - rename *_SOURCE to *_SOURCES (for consistency) - - standardize the shared library name to lib$(LIBNAME).$(EXT)
that is all.
if it helps, try to forget that with these changes I could use the template Makefile as is.
fgamsdr IOhannes
On Oct 30, 2011, at 4:23 PM, IOhannes m zmölnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 10/30/2011 09:13 PM, IOhannes m zmölnig wrote:
it is exactly my suggestion to standardize it to lib$(LIBNAME).$(EXT).
to be more precise, my suggestion basically contains exactly 2 suggestions:
- rename *_SOURCE to *_SOURCES (for consistency)
Having the name SHARED_SOURCE singular highlights that its a single file, which I think it should be. But since its an internal variable, I would be OK with it having a different name.
I don't see a gain to making it multiple files, unless you are doing something very complicated, in which case, you'll need to customize the Makefile. At that point, the Makefile is easy enough to read for someone who does Makefiles, so they don't need it documented.
- standardize the shared library name to lib$(LIBNAME).$(EXT)
that is all.
if it helps, try to forget that with these changes I could use the template Makefile as is.
It already is:
SHARED_SOURCE ?= $(wildcard lib$(LIBRARY_NAME).c) SHARED_HEADER ?= $(shell test ! -e $(LIBRARY_NAME).h || echo $(LIBRARY_NAME).h) SHARED_LIB = $(SHARED_SOURCE:.c=.$(SHARED_EXTENSION)) SHARED_TCL_LIB = $(wildcard lib$(LIBRARY_NAME).tcl)
I figured you'd want something more complicated, so I put in the ?= for you. Like before with ALL_*, take the template and modify it for your library. If its implemented in a way that still meets the goal of a simple interface, I'll merge it in.
.hc
----------------------------------------------------------------------------
Access to computers should be unlimited and total. - the hacker ethic
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-30 22:41, Hans-Christoph Steiner wrote:
Having the name SHARED_SOURCE singular highlights that its a single file, which I think it should be. But since its an internal variable, I would be OK with it having a different name.
i really cannot understand, why SHARED_SOURCE _should_ only be a single file. do you have any reference that claims that single-file libraries are somewhat "superior" (e.g. less confusing to the casual onlooker)?
anyhow...
I don't see a gain to making it multiple files, unless you are doing something very complicated, in which case, you'll need to customize the Makefile. At that point, the Makefile is easy enough to read for someone who does Makefiles, so they don't need it documented.
- standardize the shared library name to lib$(LIBNAME).$(EXT)
that is all.
It already is:
SHARED_SOURCE ?= $(wildcard lib$(LIBRARY_NAME).c) SHARED_HEADER ?= $(shell test ! -e $(LIBRARY_NAME).h || echo $(LIBRARY_NAME).h) SHARED_LIB = $(SHARED_SOURCE:.c=.$(SHARED_EXTENSION))
not really. SHARED_LIB is generated by replacing all ".c" extension by ".$(SHARED_EXTENSION) within SHARED_SOURCE, which is a space delimited list of tokens. your SHARED_SOURCE happens to consist only of a single token, but at the same time it is settable by the user (and indeed documents that it could be overridden by the use of "?=")
whereas LIBRARY_NAME is documented to be a single name (well i thought so; i cannot find any reference to this either)
if we leave variable names as they are, my proposed change is a one-liner (see attachment)
SHARED_TCL_LIB = $(wildcard lib$(LIBRARY_NAME).tcl)
does this already do anything? it seems like the only use of SHARED_TCL_LIB is within the "showsetup" target.
I figured you'd want something more complicated, so I put in the ?= for you. Like before with ALL_*, take the template and modify it for your library. If its implemented in a way that still meets the goal of a simple interface, I'll merge it in.
ah, we should create some code for me to say "this is _really_ a simple change"
mfgasdr IOhannes
On Oct 31, 2011, at 4:53 AM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-30 22:41, Hans-Christoph Steiner wrote:
Having the name SHARED_SOURCE singular highlights that its a single file, which I think it should be. But since its an internal variable, I would be OK with it having a different name.
i really cannot understand, why SHARED_SOURCE _should_ only be a single file. do you have any reference that claims that single-file libraries are somewhat "superior" (e.g. less confusing to the casual onlooker)?
anyhow...
I don't see a gain to making it multiple files, unless you are doing something very complicated, in which case, you'll need to customize the Makefile. At that point, the Makefile is easy enough to read for someone who does Makefiles, so they don't need it documented.
- standardize the shared library name to lib$(LIBNAME).$(EXT)
that is all.
It already is:
SHARED_SOURCE ?= $(wildcard lib$(LIBRARY_NAME).c) SHARED_HEADER ?= $(shell test ! -e $(LIBRARY_NAME).h || echo $(LIBRARY_NAME).h) SHARED_LIB = $(SHARED_SOURCE:.c=.$(SHARED_EXTENSION))
not really. SHARED_LIB is generated by replacing all ".c" extension by ".$(SHARED_EXTENSION) within SHARED_SOURCE, which is a space delimited list of tokens. your SHARED_SOURCE happens to consist only of a single token, but at the same time it is settable by the user (and indeed documents that it could be overridden by the use of "?=")
whereas LIBRARY_NAME is documented to be a single name (well i thought so; i cannot find any reference to this either)
if we leave variable names as they are, my proposed change is a one-liner (see attachment)
SHARED_TCL_LIB = $(wildcard lib$(LIBRARY_NAME).tcl)
does this already do anything? it seems like the only use of SHARED_TCL_LIB is within the "showsetup" target.
Oops, messed that one up. It now works. Thanks for catching that.
I figured you'd want something more complicated, so I put in the ?= for you. Like before with ALL_*, take the template and modify it for your library. If its implemented in a way that still meets the goal of a simple interface, I'll merge it in.
ah, we should create some code for me to say "this is _really_ a simple change"
You should create some code that solves your issue in your library. Then we'll take it from there.
.hc
----------------------------------------------------------------------------
All mankind is of one author, and is one volume; when one man dies, one chapter is not torn out of the book, but translated into a better language; and every chapter must be so translated.... -John Donne
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-31 15:35, Hans-Christoph Steiner wrote:
You should create some code that solves your issue in your library. Then we'll take it from there.
hmm, the diff i posted was a cleaned up diff taken from iem/iemnet/Makefile.
the code is already working (well kindof, if we don't take the w32 problems into account which are a different problem)
fgmasdr IOhannes
-
Hans-Christoph Steiner -
IOhannes m zmoelnig -
IOhannes m zmölnig