IOhannes and I were discussing how to load shared code for a libdir on IRC. The sticking point is that Mac OS X seems to require dylibs to have a hard-coded path. IOhannes posted a test lib to try to work out how to do it using DYLD_LIBRARY_PATH:
http://iem.at/~zmoelnig/OSX/test.tgz
So here are my results from Mac OS X 10.5.8:
creating a [test] object: -------------------------
/private/tmp/test/tmp/test.pd_darwin: dlopen(/private/tmp/test/tmp/ test.pd_darwin, 10): no suitable image found. Did find: /private/tmp/test/tmp/test.pd_darwin: unknown required load command 0x80000022 test ... couldn't create
hans@dhcp-10 tmp $ ~/code/pd-extended.git/src/pd -noprefs -lib test -------------------------------------------------------------------
./test.pd_darwin: dlopen(./test.pd_darwin, 10): no suitable image found. Did find: ./test.pd_darwin: unknown required load command 0x80000022 test: can't load library
Here is the IRC transcript, for those who might be interested: --------------------------------------------------------------
but it seems, you are trying to find arguments against single-object- per-lib 11:23:51 buscon [~chatzilla@orange.demos-europe.eu] entered the room.
_hc IRC 11:24:47 the only thing they currently fix IMHO is the hexloader issue and shared code 11:25:05 both of these can be handled in the libdir style, but its just not implemented yet... 11:25:26 I have an idea that should work for the shared code
IOhannes IRC 11:26:28 ?
_hc IRC 11:27:17 just have an easy way for each class to call dlopen on a shared library as part of setup() 11:27:45 based on c_externdir
IOhannes IRC 11:28:54 uäh 11:29:12 dlopen() is not really about sharing code, but about externalizing it
_hc IRC 11:29:22 eh?
IOhannes IRC 11:29:24 (plugins)
_hc IRC 11:29:33 its about opening a dynamic library 11:29:38 hence d-l-open
IOhannes IRC 11:29:56 exactly, and this is not what you usually do when you want to share code 11:30:04 mamalala left the room (quit: Ping timeout: 256 seconds).
_hc IRC 11:30:13 you mean you want it private to the project 11:30:27 when I share code, it is very much what I usually od
IOhannes IRC 11:30:58 nah, i mean when i hear s"sharing code" i think "dynamically linking against a library" 11:31:08 this does not require to dlopen() a file
_hc IRC 11:31:22 well, it does on Mac OS X for deployment 11:31:35 because dylibs have static paths coded in them
IOhannes IRC 11:31:36 it requires to tell the linker that i want to use a library
_hc IRC 11:31:40 that doesn't work for externals 11:31:45 you have to load them directly with dlopen 11:32:09 mamalala [~mamalala@89.204.137.162] entered the room.
IOhannes IRC 11:32:14 Gem uses a big shared library (openGL) and i haven't seen it dlopen()end on any platform (yet)
_hc IRC 11:32:17 so either use a hard-coded path or dlopn
IOhannes IRC 11:32:31 argh
_hc IRC 11:32:33 yes, because Gem has a hardcoded path to OpenGL 11:32:43 and it is in the same path on all Mac O SX systems
IOhannes IRC 11:33:04 so it won't find libGL if it was in /usr/local/lib(?
_hc IRC 11:33:35 that's the dynamic lookup of symbols in the lib path
IOhannes IRC 11:33:44 so?
_hc IRC 11:34:06 how are you going to put ~/Library/Pd/Gem into the lib path?
IOhannes IRC 11:34:24 i think it would be nice if the loader would: 11:34:42 find Gem.pd_darwin in ~/Library/Pd/Gem 11:34:53 then add ~/Library/Pd/Gem to the LD_LIBRARY_PATH 11:35:05 then delopen() Gem.pd_darwin 11:35:19 emoc left the room (quit: Quit: .:*%##~¤#*° °° ° °).
IOhannes IRC 11:35:20 wants to say dlopen(), of course 11:35:27 and finally restore the LD_LIBRARY_PATH
_hc IRC 11:35:31 LD_LIBRARY_PATH is global 11:35:41 that sounds like a recipe for disaster
IOhannes IRC 11:35:45 i don't see a problem 11:35:49 tonussi left the room (quit: Quit: Ex-Chat).
_hc IRC 11:36:03 same issues as Pd-extended has loading all libs by default
IOhannes IRC 11:36:06 you restore LD_LIBRARY_PATH to the original value after trying to dlopen() Gem.pd_darwin
_hc IRC 11:36:07 namespace conflicts 11:36:15 why not just dlopen() then? 11:36:30 buscon left the room (quit: Quit: ChatZilla 0.9.87 [Firefox 6.0.2/20110902133214]).
IOhannes IRC 11:36:35 because the logic needs to be implemented only once
_hc IRC 11:36:40 if you know in advance whhich DLL you need, why bother searching fo rit 11:36:46 same with the dlopen() logic
IOhannes IRC 11:36:56 and you don't have to work with error-prone function pointers 11:37:19 how is your development workflow (on systems that do not have weak- linking built-in)?
_hc IRC 11:37:24 hmm, I think function pointers work quite well 11:37:39 cd mylibdir && make
IOhannes IRC 11:37:40 it works fine, if you don't have to repeat the logic
_hc IRC 11:37:41 and it'll work
IOhannes IRC 11:37:48 _that_ sounds like disaster
_hc IRC 11:37:57 what?
IOhannes IRC 11:38:24 having _all_ devs have to implement their function pointers again and again 11:38:43 "cd mylibdir && make" is approximately how you build each and every external 11:38:50 i don't see how this is related to dlopen()
_hc IRC 11:39:06 you asked about the dev workflow
IOhannes IRC 11:39:07 and then dlopen() is different on w32 and un*x 11:39:16
_hc IRC 11:39:18 MinGW provides libdl
IOhannes IRC 11:39:26 vendor lock in
_hc IRC 11:39:37 lol
IOhannes IRC 11:39:51 back to "development workflow"
_hc IRC 11:39:52 except you have the sources and right to do whatever you want with it 11:40:05 yes, you are correct, I don't care to support MSVC
IOhannes IRC 11:40:07 like with "libreoffice"? 11:40:21 back to "development workflow"
_hc IRC 11:40:24 its too much work and not worth it 11:40:32 but I'd be happy if someone else wants to do it 11:40:36 its all free software
IOhannes IRC 11:40:44 to do what?
_hc IRC 11:40:55 handle MSVC compatibility
IOhannes IRC 11:41:14 after deliberately breaking it by depending on dlopen()? 11:41:26 i still don't see the problem with LD_LIBRARY_PATH
_hc IRC 11:41:30 someone could build libdl using MSVC, I am sure
IOhannes IRC 11:41:48 but why would anybody want to do that, if they have LoadLibrary 11:41:59 which is not really better or worse
_hc IRC 11:42:16 libdl for Windwos is a wrapper on LoadLibrary 11:42:23 Mac OS X did the same thing, albeit officially
IOhannes IRC 11:42:27 so?
_hc IRC 11:42:39 it means you don't need #ifdef _WIN32 11:42:41 very nice
IOhannes IRC 11:43:06 i don't want to "#ifdef _WIN32" because i don't see why i would need any platform dependent code in my setup()
_hc IRC 11:43:12 IOhannes: you want to add the currently loading lib to LD_LIBRARY_PATH, and then remove it?
IOhannes IRC 11:43:13 _very_ nice
IOhannes IRC 11:43:29 _hc: that's about it
_hc IRC 11:43:40 won't work on Mac OS X because of the hardcoded paths issue 11:43:44 there are two paths: 11:43:53 library id, i.e... "myself" 11:43:59 and libraries I am linked ot
IOhannes IRC 11:44:47 but iirc, those hardcoded paths can be relative, so on OSX we don't even need the LD_LIBRARY_PATH
_hc IRC 11:45:08 only relative to the Pd exe 11:45:34 then libs would only work if installed into ~/Library/Pd and Pd installed into /Applications with the same exact name 11:45:38 all dev builds would break 11:46:01 as would the ability to run Pd from any folder in Mac OS X, which is normal 11:46:07 loadbang left the room (quit: Read error: Operation timed out).
IOhannes IRC 11:46:28 but what does "man dyld" mean with: "DYLD_LIBRARY_PATH This is a colon separated list of directories that contain libraries. The dynamic linker searches these directories before it searches the default locations for libraries. It allows you to test new versions of existing libraries."
_hc IRC 11:46:31 its not relative paths really, there is a single variable @executable_path, which is the path to pd 11:46:40 loadbang [~loadbang@host86-177-236-179.range86-177.btcentralplus.com] entered the room.
_hc IRC 11:46:51 sure, that is also the case 11:46:58 that doesn't contradict what I say
IOhannes IRC 11:47:31 for me it sounds like this is the mechanism i propose with manipulating LD_LIBRARY_PATH
_hc IRC 11:48:03 right, but libs in DYLD_LINRARY_PATH still need to have their ID as a hardcoded path 11:48:16 so export = DYLD_LIBRARY_PATH=/foo 11:48:27 the library is linked to be /foo/mylib.dylib
IOhannes IRC 11:49:00 i don't exactly get what "is linked to be" means here 11:49:50 are you trying to say, that there is a conflict between the libraries "real" and "self" path, and that the "self" path (the one the library believes it resides in), has to be correct?
_hc IRC 11:49:59 they are called "install names"
IOhannes IRC 11:50:03 wonders whether he makes sense
_hc IRC 11:50:54 yes, the "install name" must be the actual path it resides in 11:50:57 or it doesn't work 11:51:43 you can see it by doing "otool -L /usr/lib/libc.dylib" 11:52:12 _ /usr/lib/libc.dylib: _ /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.7) _ /usr/lib/system/libmathCommon.A.dylib (compatibility version 1.0.0, current version 292.4.0) 11:52:20 the first line is the install name 11:52:22 the others the links
IOhannes IRC 11:52:57 but if i copy libc.dylib to $(PWD) and do a "otool -L libc.dylib", the "install name" says: 11:53:00 libc.dylib: 11:53:53 (forget the trailing ":")
_hc IRC 11:56:02 oops, bad example 11:56:26 otool -L /usr/lib/libfcgi.dylib 11:56:32 tonussi [~lucaspt@177.40.10.96] entered the room.
_hc IRC 11:56:41 _/usr/lib/libfcgi.dylib: _ /usr/lib/libfcgi.0.dylib (compatibility version 1.0.0, current version 1.0.0) _ /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.0.0) _ /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) 11:56:46 libc I guess is special 11:56:52 libc is weird on Mac OS X 11:57:14 hans@dhcp-10 pd-extended.git $ ls -l /usr/lib/libc.dylib lrwxr-xr-x 1 root wheel 15 2011-05-02 04:59 /usr/lib/libc.dylib -> libSystem.dylib 11:57:43 actually, libc is actually libSystem, and you can see the first entry is libSystem 11:57:47 that's the ID
IOhannes IRC 11:58:07 i'll have to try this out
_hc IRC 11:58:12 please do
IOhannes IRC 11:58:15 is sorry for being a disbeliever
_hc IRC 11:58:21 it sucks, but its true
IOhannes IRC 12:09:15 hmm, i cannot reproduce this 12:10:11 i created a library libtest, and compiled it 2 times with different flags, so it's sole function will print something different depending on which compilation you are using 12:10:26 then i created a binary, that uses this library 12:10:52 with DYLD_LIBRARY_PATH, i can chose which version of the library is used 12:11:18 the .dylibs are anywhere but in the path where they are expected (which would be /usr/local/lib)
IOhannes IRC 12:17:36 now trying with an external
_hc IRC 12:40:09 IOhannes: any luck? 12:40:23 dto left the room (quit: Quit: Leaving.).
IOhannes IRC 12:40:58 http://iem.at/~zmoelnig/OSX/test.tgz 12:41:25 i can tell an external which dylib to use by setting DYLD_LIBRARY_PATH 12:41:37 i have to run now
----------------------------------------------------------------------------
"[T]he greatest purveyor of violence in the world today [is] my own government." - Martin Luther King, Jr.