numpy library compiled to run with other libraries inside Chimera

Terry Lang

12 Jul 2011 12 Jul '11

1:02 p.m.

Hey Guys!

I use Chimera as a back end library for a bunch of my code. I occasionally run into a function that an external library, scipy for example, has already implemented. I have found the chimera flag that allows me to import python libraries for use. However, because numpy is already distributed with Chimera, importing libraries that depend on numpy is almost _impossible_. We run into a variety of version and compiler issues that are very difficult to work around. Would it be possible to get more transparent documentation on how the numpy library is compiled and when it will be upgraded in a particular release? (NOTE: We actually have this problem right now, so even an email with a description of the current nightly build set up would be SUPER helpful!).

Thanks in advance, Terry

-- Paula Therese Lang Postdoctoral Scholar Alber Lab UC Berkeley/QB3

Show replies by date

Tom Goddard

12 Jul 12 Jul

3:35 p.m.

New subject: numpy library compiled to run with other libraries inside Chimera

Hi Terry,

Chimera daily builds are using numpy 1.4.1 compiled with just the standard "python setup.py install". Here's the Chimera numpy make file in the Chimera SVN source code repository

http://plato.cgl.ucsf.edu/trac/chimera/browser/trunk/foreign/numpy

The other day I needed a numpy 1.6 function and was tempted to update Chimera to numpy 1.6 but have not done it yet.

It is difficult for us to predict the numpy version of the next Chimera production release because we don't know when Chimera 1.6 will be released and we don't know when future numpy versions will be released. We are not likely to update numpy in the 3 months prior to a Chimera production release. And I would like to be up to the most current numpy that meets that requirement.

Tom

...

Hey Guys!
      I use Chimera as a back end library for a bunch of my code.  I
occasionally run into a function that an external library, scipy for example, has already implemented. I have found the chimera flag that allows me to import python libraries for use. However, because numpy is already distributed with Chimera, importing libraries that depend on numpy is almost _impossible_. We run into a variety of version and compiler issues that are very difficult to work around. Would it be possible to get more transparent documentation on how the numpy library is compiled and when it will be upgraded in a particular release? (NOTE: We actually have this problem right now, so even an email with a description of the current nightly build set up would be SUPER helpful!).

Thanks in advance, Terry

Greg Couch

3:59 p.m.

New subject: numpy library compiled to run with other libraries inside Chimera

On 07/12/2011 01:02 PM, Terry Lang wrote:

...

      I use Chimera as a back end library for a bunch of my code.  I
occasionally run into a function that an external library, scipy for example, has already implemented. I have found the chimera flag that allows me to import python libraries for use. However, because numpy is already distributed with Chimera, importing libraries that depend on numpy is almost _impossible_. We run into a variety of version and compiler issues that are very difficult to work around. Would it be possible to get more transparent documentation on how the numpy library is compiled and when it will be upgraded in a particular release? (NOTE: We actually have this problem right now, so even an email with a description of the current nightly build set up would be SUPER helpful!)

So as you can tell, it's has not been a priority for chimera to be binary compatible with other software packages. And if you are installing your own Python packages to work with chimera, it is much easier to install them with chimera's Python into chimera's Python library (using chimera --noguil --script "setup.py ...") and use "chimera --nogui" to run scripts that use both chimera and other packages. Python packages, that have C code in them, should work, but ones that have C++ and Fortran code might not work.

numpy is updated on an ad hoc basis. We are generally conservative about updating it, but it can be updated for any reason (bug fixes, new version of Python needs new version, some other foreign package needs a newer version, etc.). And that can be said for any of the 30+ foreign software packages that are included with chimera. In the near term, I don't anticipate any updates unless we get requests for them (hint), except possibly for updating Python to the latest 2.7 version. The version numbers of the foreign packages are found in http://plato.cgl.ucsf.edu/trac/chimera/browser/trunk/mk/foreign.make, except for Python, Tcl/Tk, Tix, and Togl, which are in http://plato.cgl.ucsf.edu/trac/chimera/browser/trunk/mk/config.make. And configuration options for the foreign packages are in the per-package GNUmakefile in http://plato.cgl.ucsf.edu/trac/chimera/browser/trunk/foreign/ http://plato.cgl.ucsf.edu/trac/chimera/browser/trunk/foreign.

The compiler flags for various platforms are found in http://plato.cgl.ucsf.edu/trac/chimera/browser/trunk/mk/os.make. The compiler versions used are not documented and may change. That said, on Windows, we use Microsoft Visual Studio 2008 because that is what the python.org version uses. On Mac OS X we use the standard compiler for the OS version, so the 32-bit version is built on Mac OS X 10.5.8 with Xcode 3.1.2 and the 64-bit version on Mac OS X 10.6 with Xcode 3.2.6. And then there's Linux. There are so many different versions of Linux that we build on a lowest common denominator system to get the forward-compatibility advantage of linking with an older C runtime. Currently, we are building 32-bit Linux binaries on a Debian 4 system with gcc 4.1.2 and plan to move the 64-bit builds to a Debian 4 64-bit system, but currently build on Red Hat Enterprise Linux 5.6 system with gcc 4.2.3. I don't see us changing from Debian 4 for a long time, but we might update our version of gcc to get C++0x features and other improvements -- no timetable for that though.

Our nightly build setup checks out a fresh copy of chimera on each platform every evening and then compiles everything and builds a chimera installer. The "mother" process waits for each build to complete and decides if the build was a success if the installer was built or not. Then the successful builds are added to the web site. The identical mechanism is used to build releases by adding an argument saying what kind of release it is. For the gory details, start with http://plato.cgl.ucsf.edu/trac/chimera/browser/trunk/tools/build_chimera.

HTH,

Greg

Konrad Hinsen

13 Jul 13 Jul

1:55 a.m.

New subject: numpy library compiled to run with other libraries inside Chimera

On 12 Jul 2011, at 22:02, Terry Lang wrote:

...

    I use Chimera as a back end library for a bunch of my code.  I
occasionally run into a function that an external library, scipy for example, has already implemented. I have found the chimera flag that allows me to import python libraries for use. However, because numpy is already distributed with Chimera, importing libraries that depend on numpy is almost _impossible_.

What I do is install such third-party modules from source, using Chimera's Python. Here's the script I use under MacOS:

#!/bin/bash chimera_py=/Applications/Chimera.app/Contents/Frameworks/ Python.framework/Versions/2.7 export PYTHONPATH=$chimera_py/lib/python2.7 export DYLD_LIBRARY_PATH=$chimera_py $chimera_py/bin/python2.7 $*

Just use this instead of the "python" command to run the setup.py coming with various Python packages. A similar approach should work under Linux, for Windows I don't know.

Konrad. -- --------------------------------------------------------------------- Konrad Hinsen Centre de Biophysique Moléculaire, CNRS Orléans Synchrotron Soleil - Division Expériences Saint Aubin - BP 48 91192 Gif sur Yvette Cedex, France Tel. +33-1 69 35 97 15 E-Mail: research at khinsen dot fastmail dot net ---------------------------------------------------------------------

dweber＠cgl.ucsf.edu

15 Jul 15 Jul

6:22 p.m.

New subject: numpy library compiled to run with other libraries inside Chimera

Hi Terry,

It might help to use virtualenv, which can encapsulate or sandbox your additions to the chimera python environment. Please see my notes on this at http://plato.cgl.ucsf.edu/trac/chimera/wiki/ChimeraDevEnv (and links therein). If you have a list of packages your working with, it might be possible to create a script that will "easy_install" them into a virtualenv.

Take care, Darren

...

Hey Guys!
     I use Chimera as a back end library for a bunch of my code.  I
occasionally run into a function that an external library, scipy for example, has already implemented. I have found the chimera flag that allows me to import python libraries for use. However, because numpy is already distributed with Chimera, importing libraries that depend on numpy is almost _impossible_. We run into a variety of version and compiler issues that are very difficult to work around. Would it be possible to get more transparent documentation on how the numpy library is compiled and when it will be upgraded in a particular release? (NOTE: We actually have this problem right now, so even an email with a description of the current nightly build set up would be SUPER helpful!).

Thanks in advance, Terry

-- Paula Therese Lang Postdoctoral Scholar Alber Lab UC Berkeley/QB3

Chimera-dev mailing list Chimera-dev@cgl.ucsf.edu http://plato.cgl.ucsf.edu/mailman/listinfo/chimera-dev

4691

Age (days ago)

4695

Last active (days ago)

List overview

Download

4 comments

5 participants

tags (0)

participants (5)

dweber＠cgl.ucsf.edu
Greg Couch
Konrad Hinsen
Terry Lang
Tom Goddard