IOhannes and I were discussing how to load shared code for a libdir on IRC. The sticking point is that Mac OS X seems to require dylibs to have a hard-coded path. IOhannes posted a test lib to try to work out how to do it using DYLD_LIBRARY_PATH:
http://iem.at/~zmoelnig/OSX/test.tgz
So here are my results from Mac OS X 10.5.8:
creating a [test] object: -------------------------
/private/tmp/test/tmp/test.pd_darwin: dlopen(/private/tmp/test/tmp/ test.pd_darwin, 10): no suitable image found. Did find: /private/tmp/test/tmp/test.pd_darwin: unknown required load command 0x80000022 test ... couldn't create
hans@dhcp-10 tmp $ ~/code/pd-extended.git/src/pd -noprefs -lib test -------------------------------------------------------------------
./test.pd_darwin: dlopen(./test.pd_darwin, 10): no suitable image found. Did find: ./test.pd_darwin: unknown required load command 0x80000022 test: can't load library
Here is the IRC transcript, for those who might be interested: --------------------------------------------------------------
but it seems, you are trying to find arguments against single-object- per-lib 11:23:51 buscon [~chatzilla@orange.demos-europe.eu] entered the room.
_hc IRC 11:24:47 the only thing they currently fix IMHO is the hexloader issue and shared code 11:25:05 both of these can be handled in the libdir style, but its just not implemented yet... 11:25:26 I have an idea that should work for the shared code
IOhannes IRC 11:26:28 ?
_hc IRC 11:27:17 just have an easy way for each class to call dlopen on a shared library as part of setup() 11:27:45 based on c_externdir
IOhannes IRC 11:28:54 uäh 11:29:12 dlopen() is not really about sharing code, but about externalizing it
_hc IRC 11:29:22 eh?
IOhannes IRC 11:29:24 (plugins)
_hc IRC 11:29:33 its about opening a dynamic library 11:29:38 hence d-l-open
IOhannes IRC 11:29:56 exactly, and this is not what you usually do when you want to share code 11:30:04 mamalala left the room (quit: Ping timeout: 256 seconds).
_hc IRC 11:30:13 you mean you want it private to the project 11:30:27 when I share code, it is very much what I usually od
IOhannes IRC 11:30:58 nah, i mean when i hear s"sharing code" i think "dynamically linking against a library" 11:31:08 this does not require to dlopen() a file
_hc IRC 11:31:22 well, it does on Mac OS X for deployment 11:31:35 because dylibs have static paths coded in them
IOhannes IRC 11:31:36 it requires to tell the linker that i want to use a library
_hc IRC 11:31:40 that doesn't work for externals 11:31:45 you have to load them directly with dlopen 11:32:09 mamalala [~mamalala@89.204.137.162] entered the room.
IOhannes IRC 11:32:14 Gem uses a big shared library (openGL) and i haven't seen it dlopen()end on any platform (yet)
_hc IRC 11:32:17 so either use a hard-coded path or dlopn
IOhannes IRC 11:32:31 argh
_hc IRC 11:32:33 yes, because Gem has a hardcoded path to OpenGL 11:32:43 and it is in the same path on all Mac O SX systems
IOhannes IRC 11:33:04 so it won't find libGL if it was in /usr/local/lib(?
_hc IRC 11:33:35 that's the dynamic lookup of symbols in the lib path
IOhannes IRC 11:33:44 so?
_hc IRC 11:34:06 how are you going to put ~/Library/Pd/Gem into the lib path?
IOhannes IRC 11:34:24 i think it would be nice if the loader would: 11:34:42 find Gem.pd_darwin in ~/Library/Pd/Gem 11:34:53 then add ~/Library/Pd/Gem to the LD_LIBRARY_PATH 11:35:05 then delopen() Gem.pd_darwin 11:35:19 emoc left the room (quit: Quit: .:*%##~¤#*° °° ° °).
IOhannes IRC 11:35:20 wants to say dlopen(), of course 11:35:27 and finally restore the LD_LIBRARY_PATH
_hc IRC 11:35:31 LD_LIBRARY_PATH is global 11:35:41 that sounds like a recipe for disaster
IOhannes IRC 11:35:45 i don't see a problem 11:35:49 tonussi left the room (quit: Quit: Ex-Chat).
_hc IRC 11:36:03 same issues as Pd-extended has loading all libs by default
IOhannes IRC 11:36:06 you restore LD_LIBRARY_PATH to the original value after trying to dlopen() Gem.pd_darwin
_hc IRC 11:36:07 namespace conflicts 11:36:15 why not just dlopen() then? 11:36:30 buscon left the room (quit: Quit: ChatZilla 0.9.87 [Firefox 6.0.2/20110902133214]).
IOhannes IRC 11:36:35 because the logic needs to be implemented only once
_hc IRC 11:36:40 if you know in advance whhich DLL you need, why bother searching fo rit 11:36:46 same with the dlopen() logic
IOhannes IRC 11:36:56 and you don't have to work with error-prone function pointers 11:37:19 how is your development workflow (on systems that do not have weak- linking built-in)?
_hc IRC 11:37:24 hmm, I think function pointers work quite well 11:37:39 cd mylibdir && make
IOhannes IRC 11:37:40 it works fine, if you don't have to repeat the logic
_hc IRC 11:37:41 and it'll work
IOhannes IRC 11:37:48 _that_ sounds like disaster
_hc IRC 11:37:57 what?
IOhannes IRC 11:38:24 having _all_ devs have to implement their function pointers again and again 11:38:43 "cd mylibdir && make" is approximately how you build each and every external 11:38:50 i don't see how this is related to dlopen()
_hc IRC 11:39:06 you asked about the dev workflow
IOhannes IRC 11:39:07 and then dlopen() is different on w32 and un*x 11:39:16
_hc IRC 11:39:18 MinGW provides libdl
IOhannes IRC 11:39:26 vendor lock in
_hc IRC 11:39:37 lol
IOhannes IRC 11:39:51 back to "development workflow"
_hc IRC 11:39:52 except you have the sources and right to do whatever you want with it 11:40:05 yes, you are correct, I don't care to support MSVC
IOhannes IRC 11:40:07 like with "libreoffice"? 11:40:21 back to "development workflow"
_hc IRC 11:40:24 its too much work and not worth it 11:40:32 but I'd be happy if someone else wants to do it 11:40:36 its all free software
IOhannes IRC 11:40:44 to do what?
_hc IRC 11:40:55 handle MSVC compatibility
IOhannes IRC 11:41:14 after deliberately breaking it by depending on dlopen()? 11:41:26 i still don't see the problem with LD_LIBRARY_PATH
_hc IRC 11:41:30 someone could build libdl using MSVC, I am sure
IOhannes IRC 11:41:48 but why would anybody want to do that, if they have LoadLibrary 11:41:59 which is not really better or worse
_hc IRC 11:42:16 libdl for Windwos is a wrapper on LoadLibrary 11:42:23 Mac OS X did the same thing, albeit officially
IOhannes IRC 11:42:27 so?
_hc IRC 11:42:39 it means you don't need #ifdef _WIN32 11:42:41 very nice
IOhannes IRC 11:43:06 i don't want to "#ifdef _WIN32" because i don't see why i would need any platform dependent code in my setup()
_hc IRC 11:43:12 IOhannes: you want to add the currently loading lib to LD_LIBRARY_PATH, and then remove it?
IOhannes IRC 11:43:13 _very_ nice
IOhannes IRC 11:43:29 _hc: that's about it
_hc IRC 11:43:40 won't work on Mac OS X because of the hardcoded paths issue 11:43:44 there are two paths: 11:43:53 library id, i.e... "myself" 11:43:59 and libraries I am linked ot
IOhannes IRC 11:44:47 but iirc, those hardcoded paths can be relative, so on OSX we don't even need the LD_LIBRARY_PATH
_hc IRC 11:45:08 only relative to the Pd exe 11:45:34 then libs would only work if installed into ~/Library/Pd and Pd installed into /Applications with the same exact name 11:45:38 all dev builds would break 11:46:01 as would the ability to run Pd from any folder in Mac OS X, which is normal 11:46:07 loadbang left the room (quit: Read error: Operation timed out).
IOhannes IRC 11:46:28 but what does "man dyld" mean with: "DYLD_LIBRARY_PATH This is a colon separated list of directories that contain libraries. The dynamic linker searches these directories before it searches the default locations for libraries. It allows you to test new versions of existing libraries."
_hc IRC 11:46:31 its not relative paths really, there is a single variable @executable_path, which is the path to pd 11:46:40 loadbang [~loadbang@host86-177-236-179.range86-177.btcentralplus.com] entered the room.
_hc IRC 11:46:51 sure, that is also the case 11:46:58 that doesn't contradict what I say
IOhannes IRC 11:47:31 for me it sounds like this is the mechanism i propose with manipulating LD_LIBRARY_PATH
_hc IRC 11:48:03 right, but libs in DYLD_LINRARY_PATH still need to have their ID as a hardcoded path 11:48:16 so export = DYLD_LIBRARY_PATH=/foo 11:48:27 the library is linked to be /foo/mylib.dylib
IOhannes IRC 11:49:00 i don't exactly get what "is linked to be" means here 11:49:50 are you trying to say, that there is a conflict between the libraries "real" and "self" path, and that the "self" path (the one the library believes it resides in), has to be correct?
_hc IRC 11:49:59 they are called "install names"
IOhannes IRC 11:50:03 wonders whether he makes sense
_hc IRC 11:50:54 yes, the "install name" must be the actual path it resides in 11:50:57 or it doesn't work 11:51:43 you can see it by doing "otool -L /usr/lib/libc.dylib" 11:52:12 _ /usr/lib/libc.dylib: _ /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.7) _ /usr/lib/system/libmathCommon.A.dylib (compatibility version 1.0.0, current version 292.4.0) 11:52:20 the first line is the install name 11:52:22 the others the links
IOhannes IRC 11:52:57 but if i copy libc.dylib to $(PWD) and do a "otool -L libc.dylib", the "install name" says: 11:53:00 libc.dylib: 11:53:53 (forget the trailing ":")
_hc IRC 11:56:02 oops, bad example 11:56:26 otool -L /usr/lib/libfcgi.dylib 11:56:32 tonussi [~lucaspt@177.40.10.96] entered the room.
_hc IRC 11:56:41 _/usr/lib/libfcgi.dylib: _ /usr/lib/libfcgi.0.dylib (compatibility version 1.0.0, current version 1.0.0) _ /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.0.0) _ /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) 11:56:46 libc I guess is special 11:56:52 libc is weird on Mac OS X 11:57:14 hans@dhcp-10 pd-extended.git $ ls -l /usr/lib/libc.dylib lrwxr-xr-x 1 root wheel 15 2011-05-02 04:59 /usr/lib/libc.dylib -> libSystem.dylib 11:57:43 actually, libc is actually libSystem, and you can see the first entry is libSystem 11:57:47 that's the ID
IOhannes IRC 11:58:07 i'll have to try this out
_hc IRC 11:58:12 please do
IOhannes IRC 11:58:15 is sorry for being a disbeliever
_hc IRC 11:58:21 it sucks, but its true
IOhannes IRC 12:09:15 hmm, i cannot reproduce this 12:10:11 i created a library libtest, and compiled it 2 times with different flags, so it's sole function will print something different depending on which compilation you are using 12:10:26 then i created a binary, that uses this library 12:10:52 with DYLD_LIBRARY_PATH, i can chose which version of the library is used 12:11:18 the .dylibs are anywhere but in the path where they are expected (which would be /usr/local/lib)
IOhannes IRC 12:17:36 now trying with an external
_hc IRC 12:40:09 IOhannes: any luck? 12:40:23 dto left the room (quit: Quit: Leaving.).
IOhannes IRC 12:40:58 http://iem.at/~zmoelnig/OSX/test.tgz 12:41:25 i can tell an external which dylib to use by setting DYLD_LIBRARY_PATH 12:41:37 i have to run now
----------------------------------------------------------------------------
"[T]he greatest purveyor of violence in the world today [is] my own government." - Martin Luther King, Jr.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/26/2011 06:52 PM, Hans-Christoph Steiner wrote:
IOhannes and I were discussing how to load shared code for a libdir on IRC. The sticking point is that Mac OS X seems to require dylibs to have a hard-coded path. IOhannes posted a test lib to try to work out how to do it using DYLD_LIBRARY_PATH:
http://iem.at/~zmoelnig/OSX/test.tgz
So here are my results from Mac OS X 10.5.8:
creating a [test] object:
/private/tmp/test/tmp/test.pd_darwin: dlopen(/private/tmp/test/tmp/test.pd_darwin, 10): no suitable image found. Did find: /private/tmp/test/tmp/test.pd_darwin: unknown required load command 0x80000022 test ... couldn't create
hans@dhcp-10 tmp $ ~/code/pd-extended.git/src/pd -noprefs -lib test
./test.pd_darwin: dlopen(./test.pd_darwin, 10): no suitable image found. Did find: ./test.pd_darwin: unknown required load command 0x80000022 test: can't load library
just to clarify a bit: the binaries posted under the link above were compiled under OSX-10.6, without any precautions for backward compatibility. they don't work on 10.5, nor 10.4
i also put the same set of binaries online, but this time compiled under 10.4, and they work and function as expected under 10.4 and 10.6 (and i expect them to work on 10.5 as well)
on w32, the search paths for dll's should be modifiable via the %PATH% environment, whereas on linux (and most i think BSD and hurd) the envvar in question is LD_LIBRARY_PATH
after the technical details, let's step back and have a look at what this is all about: the main motivation is to be able to bundle shared code and 3rd party libraries with externals.
the dynamic linkers of the various OSs, will automatically search for dylinked libraries your external depends on, in various places (e.g. /usr/lib).
this is great if you have an easy way to install the needed library into this place (e.g. a dependy-informed installer system like debian's "apt"). it's not so great, if you don't have such a system, and the user is expected to find and download the given dependencies and put them into your system's search path. even if an external ships with all dependencies, they still have to be installed in the "correct" places, which makes installation more complicated than just dragging a folder onto your desktop. why?
because your OS's dynamic linker doesn't look for 3rd party libraries in weird places like a folder on your desktop.
now the idea, hans and me came up with, was to make Pd tell the OS to look into those places. e.g. if library "foo.pd_darwin" gets installed into ~/Library/Pd/foo/, then it would be nice, if the dynamic linker would search for any depending libraries in the ~/Library/Pd/foo/ directory. likewise "bar.pd_linux" (installed in ~/my-externals/pub/) might have it's dependencies searched for in ~/my-externals/pub/.
if both "foo" and "bar" depend on "libdudelsack", and both provide a "libdudelsack.dll" in their homedirectories, then of course we want both libraries to choose "their" correct one.
so the idea is, to modify the automatic search path of the dynamic linker whenever a library is loaded.
example: the user installs the "frotzel" library, by downloading it, and unzipping it into /usr/local/pd-externals/
now, if the user requests a new object [frotzel], Pd will search for a matching binary and eventually find frotzel.l_ia64 in /usr/local/pd-externals/frotzel/ before dlopen()ing this binary, it will read the LD_LIBRARY_PATH variable, push it to some stack, and then add /usr/local/pd-externals/frotzel/ at the beginning of the LD_LIBRARY_PATH. it will then dlopen() the frotzel.l_ia64 binary, and the linker discovers, that it has to load the libknurbel.so library, which it will (magically) find in /usr/local/pd-externals/frotzel/ because all dependencies are fullfilled, [frotzel] will successfully be loaded and will work. after fortzel.l_ia64 has been successfully loaded, we no longer need to search for libraries in /usr/local/pd-externals/frotzel/, and therefore the original content of LD_LIBRARY_PATH is restored (popped from the stack).
the exact name of "LD_LIBRARY_PATH" is system dependent, but it seems that LD_LIBRARY_PATH, DYLD_LIBRARY_PATH and PATH cover most systems.
fasdmrt IOhannes
On Sep 26, 2011, at 2:42 PM, IOhannes m zmölnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/26/2011 06:52 PM, Hans-Christoph Steiner wrote:
IOhannes and I were discussing how to load shared code for a libdir on IRC. The sticking point is that Mac OS X seems to require dylibs to have a hard-coded path. IOhannes posted a test lib to try to work out how to do it using DYLD_LIBRARY_PATH:
http://iem.at/~zmoelnig/OSX/test.tgz
So here are my results from Mac OS X 10.5.8:
creating a [test] object:
/private/tmp/test/tmp/test.pd_darwin: dlopen(/private/tmp/test/tmp/test.pd_darwin, 10): no suitable image found. Did find: /private/tmp/test/tmp/test.pd_darwin: unknown required load command 0x80000022 test ... couldn't create
hans@dhcp-10 tmp $ ~/code/pd-extended.git/src/pd -noprefs -lib test
./test.pd_darwin: dlopen(./test.pd_darwin, 10): no suitable image found. Did find: ./test.pd_darwin: unknown required load command 0x80000022 test: can't load library
just to clarify a bit: the binaries posted under the link above were compiled under OSX-10.6, without any precautions for backward compatibility. they don't work on 10.5, nor 10.4
i also put the same set of binaries online, but this time compiled under 10.4, and they work and function as expected under 10.4 and 10.6 (and i expect them to work on 10.5 as well)
on w32, the search paths for dll's should be modifiable via the %PATH% environment, whereas on linux (and most i think BSD and hurd) the envvar in question is LD_LIBRARY_PATH
after the technical details, let's step back and have a look at what this is all about: the main motivation is to be able to bundle shared code and 3rd party libraries with externals.
the dynamic linkers of the various OSs, will automatically search for dylinked libraries your external depends on, in various places (e.g. /usr/lib).
this is great if you have an easy way to install the needed library into this place (e.g. a dependy-informed installer system like debian's "apt"). it's not so great, if you don't have such a system, and the user is expected to find and download the given dependencies and put them into your system's search path. even if an external ships with all dependencies, they still have to be installed in the "correct" places, which makes installation more complicated than just dragging a folder onto your desktop. why?
because your OS's dynamic linker doesn't look for 3rd party libraries in weird places like a folder on your desktop.
now the idea, hans and me came up with, was to make Pd tell the OS to look into those places. e.g. if library "foo.pd_darwin" gets installed into ~/Library/Pd/foo/, then it would be nice, if the dynamic linker would search for any depending libraries in the ~/Library/Pd/foo/ directory. likewise "bar.pd_linux" (installed in ~/my-externals/pub/) might have it's dependencies searched for in ~/my-externals/pub/.
if both "foo" and "bar" depend on "libdudelsack", and both provide a "libdudelsack.dll" in their homedirectories, then of course we want both libraries to choose "their" correct one.
so the idea is, to modify the automatic search path of the dynamic linker whenever a library is loaded.
example: the user installs the "frotzel" library, by downloading it, and unzipping it into /usr/local/pd-externals/
now, if the user requests a new object [frotzel], Pd will search for a matching binary and eventually find frotzel.l_ia64 in /usr/local/pd-externals/frotzel/ before dlopen()ing this binary, it will read the LD_LIBRARY_PATH variable, push it to some stack, and then add /usr/local/pd-externals/frotzel/ at the beginning of the LD_LIBRARY_PATH. it will then dlopen() the frotzel.l_ia64 binary, and the linker discovers, that it has to load the libknurbel.so library, which it will (magically) find in /usr/local/pd-externals/frotzel/ because all dependencies are fullfilled, [frotzel] will successfully be loaded and will work. after fortzel.l_ia64 has been successfully loaded, we no longer need to search for libraries in /usr/local/pd-externals/frotzel/, and therefore the original content of LD_LIBRARY_PATH is restored (popped from the stack).
I got this building and working on Mac OS X, since that's where the discussion started. It was really just a matter of getting the build flags right. I am using tkwidgets as my test bed, then I'll integrate the changes into the library template. You can see my work here in the 'newentry' branch:
https://github.com/pd-projects/tkwidgets
the exact name of "LD_LIBRARY_PATH" is system dependent, but it seems that LD_LIBRARY_PATH, DYLD_LIBRARY_PATH and PATH cover most systems.
fasdmrt IOhannes
My guess is that it'll be something like this:
#ifdef _WIN32 PATH #elif defined(__APPLE__) DYLD_LIBRARY_PATH #else LD_LIBRARY_PATH #endif
And the IRC transcript for the record:
i don't exactly know how to do LD_LIBRARY_PATH on w32 yet
_hc IRC 1:54:17 IOhannes: the annoyances don't end there... 1:54:30 if libtest10.dylib links to something esle....
IOhannes IRC 1:54:31 i _thought_ it's a matter of %PATH% or so, but i never checked
_hc IRC 1:54:52 there seems to be some vars for dyld to handle this:
IOhannes IRC 1:54:58 i'm pretty sure there is aa mechanism on w32 that allows us to do that
_hc IRC 1:54:59 DYLD_FALLBACK_LIBRARY_PATH This is a colon separated list of directories that contain libraries. It is used as the default location for libraries not found in their install path. By default, it is set to $(HOME)/lib:/usr/local/lib:/lib:/usr/lib.
DYLD_ROOT_PATH This is a colon separated list of directories. The dynamic linker will prepend each of this directory paths to every image access until a file is found.
IOhannes IRC 1:55:20 would you mind posting to pastie? 1:55:37 it's hard to read on irssie
_hc IRC 1:55:41 http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ ManPages/man1/dyld.1.html
IOhannes IRC 1:55:46 irsee, that is 1:56:11 excalibas left the room (quit: Remote host closed the connection).
IOhannes IRC 1:56:50 irssi that is
_hc IRC 1:56:50 so you are proposing that each time Pd loads a lib, it sets the *_PATH, then unsets it?
IOhannes IRC 1:57:13 i fail to see the difference between DYLD_LIBRARY_PATH and DYLD_FALLBACK_PATH 1:57:36 and yes, this is what my proposal is about
_hc IRC 1:57:47 I don't quite understand it, but it seems to be related to the "install name" thing
IOhannes IRC 1:58:39 the big pro (imho), is, that it allows devs to work as they are used to work with dylibs, and not invent some ramba zama magic dlopen() hack for a standard problem 1:59:14 but don't my DYLD_LIBRARY_PATH scripts work for you?
_hc IRC 2:00:05 some dlopen hack like externals?
IOhannes IRC 2:00:54 no, an external is a plugin, and dlopen() is designed for plugins 2:01:52 in contrast, the automatic dynamic linker is designed for dynamic sharing of code 2:02:07 which is what we want when we talk about "code-sharing" 2:02:16 and not: plugins 2:02:45 bot are closely related on a technical level, but this doesn't make them interchangeable 2:03:27 s/bot/both/ 2:03:36 is on a bad connection 2:05:12 according to http://msdn.microsoft.com/en-us/library/7d83bc18%28v=VS.100%29.aspx , %PATH% is the envvar to use to trick the w32 linker 2:15:21 pob [~pob@gar31-4-82-240-199-62.fbx.proxad.net] entered the room.
_hc IRC 3:07:09 IOhannes: what about when the libdir's .dylib is linked to other libs? 3:07:22 say /sw/bin/libspeex.dylib 3:09:59 r33p left the room (quit: Quit: ronflette mode).
IOhannes IRC 3:14:56 what about it?
_hc IRC 3:15:17 I feel like that's where there will be problems
IOhannes IRC 3:15:26 why? 3:15:37 olsen [~sesselast@xdsl-188-155-179-205.adslplus.ch] entered the room.
IOhannes IRC 3:15:51 maybe this is unclear: i'm talking about _adding_ to LD_LIBRARY_PATH, not about replacing it 3:16:12 the dynamic linker will then additionally search in yet another directory
_hc IRC 3:16:14 did you run your tests on 10.4? 3:16:17 or just build them?
IOhannes IRC 3:16:25 both
_hc IRC 3:16:47 hmm, my decision on this was informed by some experience a long time ago that i can't remember 3:17:00 so I feel like something is not going to work with this
IOhannes IRC 3:18:08 maybe it was something trivial back then
_hc IRC 3:18:48 with luck... 3:20:18 IOhannes: this project was already setup with this idea in mind, so I'm trying it: github.com/pd-projects/tkwidgets 3:20:25 it has shared code in tkwidgets.c
IOhannes IRC 3:22:16 i cannot checkout with my uplink right now...
_hc IRC 3:22:41 wow, that's slow
IOhannes IRC 3:24:26 what? my connection or your download speed?
_hc IRC 3:25:06 your uplink
IOhannes IRC 3:28:27 i know; officially i'm without internet at home
_hc IRC 3:32:09 IOhannes: the test thing doesn't build for me on 10.5: http://pastebin.com/ 3:32:24 http://pastebin.com/reTPU8MR
IOhannes IRC 3:33:32 try "make distclean & autoreconf -fiv && ./configure && make" 3:33:42 (and get the ampersands rights)
_hc IRC 3:34:57 oy, there were go: make distclean; /sw/bin/autoreconf-2.63 -fiv && ./ configure && make 3:36:18 IOhannes: so you link it normally, without DYLD_LIBRARY_PATH? 3:36:26 then set DLYD_ later? 3:36:50 lorenzosu [~lorenzosu@ppp-79-0.24-151.libero.it] entered the room.
IOhannes IRC 3:36:58 DYLD_LIBRARY_PATH is an environment variable that allows you to tweak the dynamic linker at runtime
_hc IRC 3:37:19 I know
IOhannes IRC 3:37:38 "dynamic linker at runtime" means: the piece of software that resolves dynamic libraries when you run your executable 3:37:59 so DYLD_LIBRARY_PATH is never set in the compilation/linking stage
_hc IRC 3:38:11 does it affect the linkign is my question
IOhannes IRC 3:38:14 only when you run the actual program 3:38:21 which linking? 3:39:27 (there's the gcc linking the object files into a binary and adding information on where to find external references) 3:39:58 (and theres the dynamic linker that resolves the external references when you run the binary) 3:40:11 the first thing happens on the dev machine 3:40:19 the second thing happens on the deployment machine 3:40:31 DYLIB_LIBRARY_PATH has nothing to do with the first stop 3:40:44 it's _only_ about deployment
_hc IRC 3:41:08 can you make libtool show you its command line? 3:41:22 I can't get this workign in the Makefile yet
IOhannes IRC 3:41:28 usually it print's out the cmdline
_hc IRC 3:42:25 this seems incomplete to me: libtool: link: gcc -g -O2 -o .libs/test test.o libtest/.libs/libtest.dylib 3:42:42 oh wait, wrong one 3:43:02 that's the one: /bin/sh ./libtool --tag=CC --mode=link gcc -g -O2 - module -avoid-version -shared -shrext .pd_darwin libtest/libtest.la - o test.la -rpath /usr/local/lib test_la-test.lo
IOhannes IRC 3:43:40 the line following that, is the actual gcc call 3:43:54
gcc -Wl,-undefined -Wl,dynamic_lookup -o .libs/test.pd_darwin -
bundle .libs/test_la-test.o libtest/.libs/libtest.0.0.0.dylib
_hc IRC 3:43:55 libtool: link: gcc -dynamiclib -Wl,-undefined -Wl,dynamic_lookup - o .libs/libtest.0.dylib .libs/libtest.o -O2 -install_name /usr/ local/lib/libtest.0.dylib -compatibility_version 1 -current_version 1.0 -Wl,-single_module 3:43:56 yup 3:44:15 it has to have an install_name...
IOhannes IRC 3:44:20 so it sets the install_name
_hc IRC 3:44:22 and version 3:44:23 yup
IOhannes IRC 3:44:39 but you can override that with DYLIB_LIBRARY_PATH
_hc IRC 3:44:53 and a .dylib won't work without a install_name 3:44:55 that's the pisser
IOhannes IRC 3:44:59 the libtool libraries are supposed to be as standard as can be
_hc IRC 3:45:00 even tho you can override it
IOhannes IRC 3:45:10 so what's the problem then?
_hc IRC 3:45:19 that's probably what got me in the past
IOhannes IRC 3:45:55 because our "overriding" is really only telling the linker to search for all libraries in an additional place 3:46:28 it's not forcing anything (apart from a certain search _order_) 3:46:40 i have no clue what apple want the install_name to do
_hc IRC 3:49:37 got it! 3:49:43 well, it built 3:50:11 doh! 3:50:23 the install_name is what is used in the linking 3:51:09 http://pastebin.com/Anf0BnUf
IOhannes IRC 3:51:10 in which linking?
_hc IRC 3:51:24 where would you do the DYLD_LIBRARY_PATH linking?
IOhannes IRC 3:52:07 in the pd-loader 3:52:44 oiata [~oiata@cable-86-56-113-6.cust.telecolumbus.net] entered the room.
_hc IRC 3:52:52 right, where in that process?
IOhannes IRC 3:54:55 s_loader.c:172 (push (DY)LD_LIBRARY_PATH) 3:55:13 s_loader.c:173 pop (DY)LD_LIBRARY_PATH 3:55:25 s_loader.c:183: push PATH 3:55:33 s_loader.c:184: pop PATH 3:55:54 oops, s_loader.c:185: pop PATH 3:57:35 and actually it should be done for the scheduler-loader as well (s_loader.c:246) 3:57:42 and for pd~ 3:57:58 i wish miller had created a dlopen() wrapper
_hc IRC 3:59:23 just read your email on this, I didn't think about including random other libs, that's true it should work also 3:59:30 that would be very helpful on Windows and Mac OS X
_hc IRC 4:06:23 IOhannes: strange, it seems to work without the DYLD_LIBRARY_PATH set
IOhannes IRC 4:07:10 what are you doing? 4:07:40 maybe you have the library sitting around somewhere
_hc IRC 4:07:56 to link the dylib: 4:07:57 -dynamiclib -undefined dynamic_lookup -install_name $(SHARED_LIB) - compatibility_version 1 -current_version 1.0 4:08:05 to link the objects: 4:08:15 ./$(SHARED_LI 4:08:27 cc -arch ppc -arch i386 -arch x86_64 -mmacosx-version-min=10.4 - bundle -bundle_loader /Applications/Pd-extended.app/Contents/Resources/ bin/pd -undefined dynamic_lookup -o "text.pd_darwin" "text.o" -lc -L/ sw/lib ./libtkwidgets.dylib 4:11:19 here's the actual dylib link: 4:11:19 cc -arch ppc -arch i386 -arch x86_64 -mmacosx-version-min=10.4 - dynamiclib -undefined dynamic_lookup -install_name libtkwidgets.dylib - compatibility_version 1 -current_version 1.0 -o libtkwidgets.dylib tkwidgets.o -lc -L/sw/lib 4:12:29 IOhannes: I figured it out, I was starting pd in the tkwidgets/ folder, that's how it found it
IOhannes IRC 4:15:39 the only problem i have right now, is that it is crashing on linux
_hc IRC 4:16:34 IOhannes: does the lib need a versioned name in Linux? ie. can we just have libtkwidgets.so or must it be libtkwidgets.so.1.
IOhannes IRC 4:18:52 i _think_ the latter (though i'm not 100% sure) 4:19:27 ah, i think if the library has no so_version, you might get away with .sl 4:19:31 .so, i meant
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-09-26 23:29, Hans-Christoph Steiner wrote:
My guess is that it'll be something like this:
#ifdef _WIN32 PATH #elif defined(__APPLE__) DYLD_LIBRARY_PATH #else LD_LIBRARY_PATH #endif
i would go for something with less logic involved: push(PATH); push(LD_LIBRARY_PATH); push(DYLD_LIBRARY_PATH); ... pop(PATH); pop(LD_LIBRARY_PATH); pop(DYLD_LIBRARY_PATH);
the only drawback i see right now is, that is doesn't work. at least on linux, any modification to LD_LIBRARY_PATH is nicely ignored and only the value taken when starting the application is taken into account :-(
maybe there is a soltuin for this problem, but i haven't come across one yet.
fgmasdr IOhannes
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
http://cocoaconvert.net/2009/08/02/frameworks-with-private-frameworks/
fgmasdr IOhannes
_______________________________________________ Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
That's a good break down of the issues. I'll checkout @loader_path. The @executable_path technique has been in Pd-extended for a long time, check out all of the libs in Pd-extended.app/Contents/lib by doing something like:
otool -L /Applications/Pd-extended.app/Contents/lib/libavcodec.51.dylib
But this still leaves us with Windows and GNU/Linux, right? I think the original idea of setting the LD_LIBRARY_PATH when loading, then unsetting, is a good one. It would be good to have all platforms use the same technique.
.hc
On Sep 27, 2011, at 3:42 AM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
http://cocoaconvert.net/2009/08/02/frameworks-with-private-frameworks/
fgmasdr IOhannes
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6BflIACgkQkX2Xpv6ydvSK+gCgqhyxJVkNuR8hjsB4NWfGnmgi QXgAoMrmStRdLgq/WGf6XVgUSWBXa6BP =9UUb -----END PGP SIGNATURE-----
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
----------------------------------------------------------------------------
"Making boring techno music is really easy with modern tools, but with live coding, boring techno is much harder." - Chris McCormick
I found this very useful too:
http://www.mikeash.com/pyblog/friday-qa-2009-11-06-linking-and-install-names...
I want to continue to support 10.4 as the minimum, but 10.5 introduced @rpath, which is supposed to work better:
http://www.dribin.org/dave/blog/archives/2009/11/15/rpath/
.hc
On Sep 27, 2011, at 3:42 AM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
http://cocoaconvert.net/2009/08/02/frameworks-with-private-frameworks/
fgmasdr IOhannes
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6BflIACgkQkX2Xpv6ydvSK+gCgqhyxJVkNuR8hjsB4NWfGnmgi QXgAoMrmStRdLgq/WGf6XVgUSWBXa6BP =9UUb -----END PGP SIGNATURE-----
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
----------------------------------------------------------------------------
“We must become the change we want to see. - Mahatma Gandhi
I just pushed a commit to 'newentry' branch of tkwidgets that implements the @loader_path thing. It seems to work! Next, GNU/ Linux! tkwidgets builds and links to the tkwidgets.so now on GNU/Linux.
.hc
On Sep 27, 2011, at 3:42 AM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
http://cocoaconvert.net/2009/08/02/frameworks-with-private-frameworks/
fgmasdr IOhannes
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6BflIACgkQkX2Xpv6ydvSK+gCgqhyxJVkNuR8hjsB4NWfGnmgi QXgAoMrmStRdLgq/WGf6XVgUSWBXa6BP =9UUb -----END PGP SIGNATURE-----
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
----------------------------------------------------------------------------
If you are not part of the solution, you are part of the problem.
-
Hans-Christoph Steiner -
IOhannes m zmoelnig -
IOhannes m zmölnig