Numeric/numpy revisited
I'll post this long-winded thing to the users list although it's really for the developers... I'm revisiting the issue of wanting to have access to the numpy pkg (numpy.scipy.org) for a Chimera plugin - a delayed continuation of the problem discussed here: http://www.cgl.ucsf.edu/pipermail/chimera-users/2006-April/000743.html Here's my situation: I installed chimera-1.2304-osx_x11.dmg. I then built the numpy pkg (from svn) using my system's python (2.4) via 'python setup.py build' and copied the resulting /numpy dir into the chimera /share dir. I then discover/verify 2 things: 1) using my system's python, I can successfully import both Numeric and numpy - Numeric coming from chimera and the numpy that I just built and copied into chimera's /share dir 2) using the python2.4 bundled with chimera, I cannot import both Numeric and numpy. I can always import one (either one), but then get a bizarre error when trying to import the other. --------------------------- More verbosely: I set PYTHONPATH to /Applications/Chimera.app/Contents/Resources/lib/python2.4/site-packages/Num eric:/Applications/Chimera.app/Contents/Resources/share Then check the system python: [macmini:~] heiland% pwd /Users/heiland [macmini:~] heiland% which python /usr/local/bin/python [macmini:~] heiland% ll /usr/local/bin/python lrwxr-xr-x 1 root heiland 68 Jul 27 06:19 /usr/local/bin/python@ -> ../../../Library/Frameworks/Python.framework/Versions/2.4/bin/python [macmini:~] heiland% python Python 2.4.3 (#1, Apr 7 2006, 10:54:33) [GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin Type "help", "copyright", "credits" or "license" for more information.
import Numeric import numpy from numpy.core.multiarray import typeinfo
Then check the chimera python: /Applications/Chimera.app/Contents/Resources/share [macmini:Contents/Resources/share] heiland% ../bin/python2.4 Python 2.4.3 (#1, Oct 16 2006, 09:43:37) [GCC 4.0.1 (Apple Computer, Inc. build 5247)] on darwin Type "help", "copyright", "credits" or "license" for more information.
import Numeric import numpy Traceback (most recent call last): File "<stdin>", line 1, in ? File "/Applications/Chimera.app/Contents/Resources/share/numpy/__init__.py", line 36, in ? import core File "numpy/core/__init__.py", line 8, in ? import numerictypes as nt File "numpy/core/numerictypes.py", line 83, in ? from multiarray import typeinfo, ndarray, array, empty, dtype ImportError: cannot import name typeinfo
----------------------------- OK, it then dawns on me that probably what I really should be doing is building numpy using chimera's python. So, after downloading the chimera (1.2318) OSX headers (http://www.cgl.ucsf.edu/chimera/sourcecode.html ), putting the /include in the /Resources dir, and doing: /Applications/Chimera.app/Contents/Resources/bin/python2.4 setup.py build It all builds fine, but then I have the same bizarre error when trying to import both Numeric and numpy as before. Crap. So I then use 'ldd' to see the dependencies in the different python executables: ldd /Applications/Chimera.app/Contents/Resources/bin/python2.4 /Applications/Chimera.app/Contents/Resources/bin/python2.4: /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 71.1.3) /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.3.0) /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current version 47.1.0) [macmini:~] heiland% ldd /Library/Frameworks/Python.framework/Versions/2.4/bin/python /Library/Frameworks/Python.framework/Versions/2.4/bin/python: /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current version 47.1.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.1.3) Note the different versions on the libSystem.B.dylib. Some more poking... [macmini:~] heiland% locate libgcc_s.1.dylib /Developer/SDKs/MacOSX10.3.9.sdk/usr/lib/libgcc_s.1.dylib /Developer/SDKs/MacOSX10.4u.sdk/usr/lib/libgcc_s.1.dylib /usr/lib/libgcc_s.1.dylib [macmini:~] heiland% ldd /Developer/SDKs/MacOSX10.4u.sdk/usr/lib/libgcc_s.1.dylib /Developer/SDKs/MacOSX10.4u.sdk/usr/lib/libgcc_s.1.dylib: /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.3) [macmini:~] heiland% ldd /Developer/SDKs/MacOSX10.3.9.sdk/usr/lib/libgcc_s.1.dylib /Developer/SDKs/MacOSX10.3.9.sdk/usr/lib/libgcc_s.1.dylib: /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 71.1.3) [macmini:~] heiland% ldd /usr/lib/libgcc_s.1.dylib /usr/lib/libgcc_s.1.dylib: /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.1.2) ----------------------------- One thing worth noting is the different gcc builds (my system python used a later build, 5250, cf 5247 for chimera's python). I also installed the latest xcode (2.4.1) before doing all this. If you've stayed with me to the end, I congratulate you :) Basic question - any ideas? Would it be possible for the Chimera OSX developers to verify they have the latest xcode? And secondly, do you know how one can the chimera python have the dependency on libSystem.B.dylib v.88.1.2 instead of 71.1.3 ? Thanks, -Randy Fwiw, this link http://mail.python.org/pipermail/pythonmac-sig/2006-April/017310.html seems related to this thread also.
Hi Randy, I have spent the past several days switching Chimera from Numeric to NumPy. I observe the same problem that you do trying to import both packages on Mac OS 10.4.8 using Chimera version 1.2328. I think the issue is a namespace clash between site-packages/Numeric/multiarray.so site-packages/numpy/core/multiarray.so and I think it only happens with the Chimera Python because we build our Python with a compilation flag -DUSE_DYLD_GLOBAL_NAMESPACE. That is in Chimera source code chimera/foreign/Python/GNUmakefile. Thanks to Greg Couch for pointing that out. Because of that flag only one of the two initmultiarray() routines from the Numeric and numpy libraries is being called -- probably just the first one loaded. Here's the comment about USE_DYLD_GLOBAL_NAMEPSPACE from Python source code file Python-2.4.3/Python/dynload_next.c /* ** Python modules are Mach-O MH_BUNDLE files. The best way to load these ** is each in a private namespace, so you can load, say, a module bar and a ** module foo.bar. If we load everything in the global namespace the two ** initbar() symbols will conflict. ** However, it seems some extension packages depend upon being able to access ** each others' global symbols. There seems to be no way to eat our cake and ** have it, so the USE_DYLD_GLOBAL_NAMESPACE define determines which behaviour ** you get. */ Why do we use -DUSE_DYLD_GLOBAL_NAMESPACE? I couldn't remember so I compiled Python without -DUSE_DYLD_GLOBAL_NAMESPACE and found that Chimera will not start using it. It fails to find _init_chimera(), the python initialization function fo the _chimera.so module. Why is that? There is an _chimera.dylib (dynamic library) and a _chimera.so (mach bundle). The latter is for importing using Python, and the former is for other Chimera modules (eg _surface.so) that need to link against it. The linking and Python loading could not be done with the same object a year ago when we investigated this. The _chimera.so just links against the _chimera.dylib. And for some reason the Python import (a dlopen() system call) then sees no symbols in the _chimera.so unless the USE_DYLD_GLOBAL_NAMESPACE is used. We have considered complicated fixes for that but have not pursued them. So the only two options I see are to wait a bit for our numpy Chimera, or wait for us to work out a way to drop USE_DYLD_GLOBAL_NAMESPACE. Both are not so good, involve waiting. I think the conversion to numpy will happen faster. That conversion may take some weeks or a month. We are having problems compiling numpy on IRIX and Tru64 platforms and about 1/3 of the Chimera Numeric code remains to be converted to numpy. The other 2/3 which is the volume data code has been converted and is working fine under numpy on Linux and Mac. The total effort for the conversion is probably just another 3 days but we are busy with many things and that work will be spread out over a longer time. Tom
participants (2)
-
Randy Heiland -
Thomas Goddard