Central path for bundles on shared computing environment?

Hi, We are setting up a new cluster at our university, and I stumbled upon the following problem: our nodes used for remote visualization don't have access to the internet. Therefore, users running an interactive graphical session won't be able to install bundles by themselves, which was the case in our old cluster. One workaround is to start ChimeraX either without GUI or with X forwarding over SSH on one of the normal login nodes (no high-end visualization capability) just for the purpose of downloading and installing bundles from the toolshed. This is however inconvenient to most users. Another approach is to install using the 'toolshed' command with a pre-downloaded wheel, but also not ideal. My thought would be to preinstall the most common plugins we use (like ArtiaX and ISOLDE) into a shared location, and ChimeraX would then find the bundles installed there upon startup. Therefore, my question is: is any other path other than the user's home folder supported for installing bundles? If yes, what is the order of search paths? Alternatively, is there some environment variable or other setting that I can use to tell ChimeraX where to find installed bundles? Thanks in advance, -- Ricardo Diogo Righetto

It would be simplest if you downloaded the bundles from the ChimeraX toolshed using your regular web browser. The downloaded files are Python wheels. Then they can be installed using the "toolshed install" command. You could prepare a defaults.cxc file that would install all of the default bundles from the shared location. If the bundles depend on a package from pypi.org, then you will need to download those too, but install using the "pip install" command. For example (untested): # load pypi packages first pip install /path/to/pandas-2.2.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_x86_64.whl # next load bundles toolshed install /path/to/ChimeraX_BundleName-Version-PythonTag-ABITag-PlatformTag.whl Then "ChimeraX defaults.cxc" would run the command script, or the command script could be opened (either on the command line or in Open dialog). The difference between "toolshed install" and "pip install" is that that "toolshed install" reads the added ChimeraX metadata in the wheel. So the commands, file formats, and tools get integrated into the interface. A totally untested alternative is to run your own pypi server, put the wheels you downloaded there, and use the --toolshed command line argument to point to it. The toolshed has some additional APIs that a pypi server doesn't, but it might work HTH, Greg On 4/8/2024 7:13 AM, Ricardo Righetto via ChimeraX-users wrote:
Hi,
We are setting up a new cluster at our university, and I stumbled upon the following problem: our nodes used for remote visualization don't have access to the internet. Therefore, users running an interactive graphical session won't be able to install bundles by themselves, which was the case in our old cluster. One workaround is to start ChimeraX either without GUI or with X forwarding over SSH on one of the normal login nodes (no high-end visualization capability) just for the purpose of downloading and installing bundles from the toolshed. This is however inconvenient to most users. Another approach is to install using the 'toolshed' command with a pre-downloaded wheel, but also not ideal.
My thought would be to preinstall the most common plugins we use (like ArtiaX and ISOLDE) into a shared location, and ChimeraX would then find the bundles installed there upon startup. Therefore, my question is: is any other path other than the user's home folder supported for installing bundles? If yes, what is the order of search paths? Alternatively, is there some environment variable or other setting that I can use to tell ChimeraX where to find installed bundles?
Thanks in advance,
-- Ricardo Diogo Righetto
_______________________________________________ ChimeraX-users mailing list --chimerax-users@cgl.ucsf.edu To unsubscribe send an email tochimerax-users-leave@cgl.ucsf.edu Archives:https://mail.cgl.ucsf.edu/mailman/archives/list/chimerax-users@cgl.ucsf.edu/

Hi Ricardo, I think ChimeraX packages are always installed in the user directory, and ChimeraX does not look in any other location. You could probably install them normally and then copy them into the application itself (chimerax/lib/python3.11/site-packages). You should copy both the *.dist_info directory for the package and the directory with the package itself since the *.dist_info directory is used for ChimeraX to know what is installed and create menu entries. If you updated ChimeraX you would need to reinstall packages like ISOLDE because they contain shared libraries that are linked against the ChimeraX libraries so a new ChimeraX version requires a new ISOLDE version. By the way, we do not support running ChimeraX with remote display. X windows remote display fails so often when trying to use OpenGL graphics that we discourage this use. While it can be convenient for researchers to run ChimeraX on a cluster where their data files are, we instead recommend they copy their data files to their personal computer and use ChimeraX on the personal computer for a much better experience. Tom
On Apr 9, 2024, at 12:56 AM, Greg Couch via ChimeraX-users <chimerax-users@cgl.ucsf.edu> wrote:
It would be simplest if you downloaded the bundles from the ChimeraX toolshed using your regular web browser. The downloaded files are Python wheels. Then they can be installed using the "toolshed install" command. You could prepare a defaults.cxc file that would install all of the default bundles from the shared location. If the bundles depend on a package from pypi.org, then you will need to download those too, but install using the "pip install" command. For example (untested):
# load pypi packages first pip install /path/to/pandas-2.2.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_x86_64.whl # next load bundles toolshed install /path/to/ChimeraX_BundleName-Version-PythonTag-ABITag-PlatformTag.whl Then "ChimeraX defaults.cxc" would run the command script, or the command script could be opened (either on the command line or in Open dialog). The difference between "toolshed install" and "pip install" is that that "toolshed install" reads the added ChimeraX metadata in the wheel. So the commands, file formats, and tools get integrated into the interface.
A totally untested alternative is to run your own pypi server, put the wheels you downloaded there, and use the --toolshed command line argument to point to it. The toolshed has some additional APIs that a pypi server doesn't, but it might work
HTH,
Greg
On 4/8/2024 7:13 AM, Ricardo Righetto via ChimeraX-users wrote:
Hi,
We are setting up a new cluster at our university, and I stumbled upon the following problem: our nodes used for remote visualization don't have access to the internet. Therefore, users running an interactive graphical session won't be able to install bundles by themselves, which was the case in our old cluster. One workaround is to start ChimeraX either without GUI or with X forwarding over SSH on one of the normal login nodes (no high-end visualization capability) just for the purpose of downloading and installing bundles from the toolshed. This is however inconvenient to most users. Another approach is to install using the 'toolshed' command with a pre-downloaded wheel, but also not ideal.
My thought would be to preinstall the most common plugins we use (like ArtiaX and ISOLDE) into a shared location, and ChimeraX would then find the bundles installed there upon startup. Therefore, my question is: is any other path other than the user's home folder supported for installing bundles? If yes, what is the order of search paths? Alternatively, is there some environment variable or other setting that I can use to tell ChimeraX where to find installed bundles?
Thanks in advance,
-- Ricardo Diogo Righetto
_______________________________________________ ChimeraX-users mailing list -- chimerax-users@cgl.ucsf.edu <mailto:chimerax-users@cgl.ucsf.edu> To unsubscribe send an email to chimerax-users-leave@cgl.ucsf.edu <mailto:chimerax-users-leave@cgl.ucsf.edu> Archives: https://mail.cgl.ucsf.edu/mailman/archives/list/chimerax-users@cgl.ucsf.edu/
ChimeraX-users mailing list -- chimerax-users@cgl.ucsf.edu To unsubscribe send an email to chimerax-users-leave@cgl.ucsf.edu Archives: https://mail.cgl.ucsf.edu/mailman/archives/list/chimerax-users@cgl.ucsf.edu/

Hi Tom, hi Greg, Thank you very much for your suggestions! I think Tom's suggestion of copying the bundles into the ChimeraX installation path itself is the closest to what I was hoping to achieve. However, after putting more thought into it I concluded that, at least for the time being, it will be fine that users launch ChimeraX on the login node, install whatever bundles they want from the internet, and do the heavy work on the visualization nodes later on. Regarding performance, it's true that X forwarding performs very poorly for OpenGL graphics. We use, however, a VNC server <https://osc.github.io/ood-documentation/latest/enable-desktops.html> with VirtualGL <https://www.virtualgl.org/> enabled for 3D acceleration within a remote graphical desktop session running on the cluster computing nodes with GPU. We have been happily using this setup for the past ~2 years and have to say performance is almost as smooth as running ChimeraX on my local machine, at least for our intents and purposes (visualization of cryo-EM maps, tomograms and PDB models, including making figures and movies). I understand this adds several layers of complication for you to support this kind of environment, but it's definitely possible to get it going. Cheers, -- Ricardo Diogo Righetto Em ter., 9 de abr. de 2024 às 20:01, Tom Goddard <goddard@sonic.net> escreveu:
Hi Ricardo,
I think ChimeraX packages are always installed in the user directory, and ChimeraX does not look in any other location. You could probably install them normally and then copy them into the application itself (chimerax/lib/python3.11/site-packages). You should copy both the *.dist_info directory for the package and the directory with the package itself since the *.dist_info directory is used for ChimeraX to know what is installed and create menu entries. If you updated ChimeraX you would need to reinstall packages like ISOLDE because they contain shared libraries that are linked against the ChimeraX libraries so a new ChimeraX version requires a new ISOLDE version.
By the way, we do not support running ChimeraX with remote display. X windows remote display fails so often when trying to use OpenGL graphics that we discourage this use. While it can be convenient for researchers to run ChimeraX on a cluster where their data files are, we instead recommend they copy their data files to their personal computer and use ChimeraX on the personal computer for a much better experience.
Tom
On Apr 9, 2024, at 12:56 AM, Greg Couch via ChimeraX-users < chimerax-users@cgl.ucsf.edu> wrote:
It would be simplest if you downloaded the bundles from the ChimeraX toolshed using your regular web browser. The downloaded files are Python wheels. Then they can be installed using the "toolshed install" command. You could prepare a defaults.cxc file that would install all of the default bundles from the shared location. If the bundles depend on a package from pypi.org, then you will need to download those too, but install using the "pip install" command. For example (untested): # load pypi packages first pip install /path/to/pandas-2.2.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_x86_64.whl # next load bundles toolshed install /path/to/ChimeraX_BundleName-Version-PythonTag-ABITag-PlatformTag.whl
Then "ChimeraX defaults.cxc" would run the command script, or the command script could be opened (either on the command line or in Open dialog). The difference between "toolshed install" and "pip install" is that that "toolshed install" reads the added ChimeraX metadata in the wheel. So the commands, file formats, and tools get integrated into the interface.
A totally untested alternative is to run your own pypi server, put the wheels you downloaded there, and use the --toolshed command line argument to point to it. The toolshed has some additional APIs that a pypi server doesn't, but it might work
HTH,
Greg On 4/8/2024 7:13 AM, Ricardo Righetto via ChimeraX-users wrote:
Hi,
We are setting up a new cluster at our university, and I stumbled upon the following problem: our nodes used for remote visualization don't have access to the internet. Therefore, users running an interactive graphical session won't be able to install bundles by themselves, which was the case in our old cluster. One workaround is to start ChimeraX either without GUI or with X forwarding over SSH on one of the normal login nodes (no high-end visualization capability) just for the purpose of downloading and installing bundles from the toolshed. This is however inconvenient to most users. Another approach is to install using the 'toolshed' command with a pre-downloaded wheel, but also not ideal.
My thought would be to preinstall the most common plugins we use (like ArtiaX and ISOLDE) into a shared location, and ChimeraX would then find the bundles installed there upon startup. Therefore, my question is: is any other path other than the user's home folder supported for installing bundles? If yes, what is the order of search paths? Alternatively, is there some environment variable or other setting that I can use to tell ChimeraX where to find installed bundles?
Thanks in advance,
-- Ricardo Diogo Righetto
_______________________________________________ ChimeraX-users mailing list -- chimerax-users@cgl.ucsf.edu To unsubscribe send an email to chimerax-users-leave@cgl.ucsf.edu Archives: https://mail.cgl.ucsf.edu/mailman/archives/list/chimerax-users@cgl.ucsf.edu/
_______________________________________________ ChimeraX-users mailing list -- chimerax-users@cgl.ucsf.edu To unsubscribe send an email to chimerax-users-leave@cgl.ucsf.edu Archives: https://mail.cgl.ucsf.edu/mailman/archives/list/chimerax-users@cgl.ucsf.edu/
participants (3)
-
Greg Couch
-
Ricardo Righetto
-
Tom Goddard