Poor performance with 30 molecules
Hi Ben, The main trick to improve efficiency in Chimera with many copies of a molecule is to use only the C-alpha coordinates. This is usually acceptable with working with EM data resolutions. You create a PDB file with only the CA atoms (can use "select @CA" command and Chimera File / Save PDB...) reducing the total number of atoms by 10x or more. Chimera is a very memory inefficient with molecules taking about 2 Kbytes per atom. This is about 10x worse than PyMol and VMD. You start feeling this at about 100,000 atoms and up and run into real trouble around 500,000 atoms if you are using 32-bit Chimera (no 64-bit available on Windows or Mac) because you run out of address space. The ability to rotate the models depends on rendering speed. Above 100,000 atoms only "wire" style will be fast and you must have a graphics driver installed (not Mesa on Linux). With a 4 year old graphics card (Radeon 9800 Pro) I get 10 frames per second with 200,000 atoms. With a newer card you should handle 500,000 atoms in wire at reasonable frame rates. Chimera session files include all the PDB atom coordinates. Chimera has no way to show 30 copies of a PDB file at different positions without having 30 independent copies of the data, except using surfaces with the multiscale tool. The multiscale tool does not interoperate easily with fitting in maps. Tom Elaine Meng wrote:
Begin forwarded message:
From: Benoit Zuber <bzuber@mrc-lmb.cam.ac.uk> Date: October 2, 2008 10:24:34 AM PDT To: Elaine Meng <meng@cgl.ucsf.edu> Subject: Re: [Chimera-users] volume viewer window
Hi Elaine, I just aligned a tiny subtomogram to a model with a lot of different parameter, which generated many volumes. Then I wanted to visualize all the results at once. Since each volume was tiny, chimera was not struggling, it was fine :-)
However that reminds me another issue about memory I had sometime ago. I have a tomogram into which I have fitted many copies (more than 30) of the same pdb file. There are two problems: 1) compiling the session file takes ages (around 10 minutes I would say). Opening the compiled session file takes more than 1 minute. 2) if I display all the copies of the pdb, everything is very slow (I mean if I want to move anything in space, it takes very long). If I display only a few copies (plus 1 or two slices through the tomogram), then it works fast.
For the 2nd problem, there is maybe no easy solution to speed up the calcultations, but for the first one, since we are dealing with copies of a single pdb, wouldn't it be possible to make the process faster? I reckon that the coordinates of each atom of each copy of the pdb are saved. Would there be a way to save only one copy and transformation matrix for every copy? Would it speed up file saving-compiling-opening operations?
Cheers, Ben
Hi Ben, I know Tom Goddard has made many recent improvements in Volume Viewer memory usage and speed, but it sounds like your main issue is many atomic structures rather than lots of volume data.
If the PDB copies are identical except in transformation, there is some capability in Multiscale Models to only store one copy of the coordinates and show the others as low-res surfaces. However, as soon as you try to show any more detailed display such as ribbons or atoms and bonds, the full coordinates for that copy are loaded. If you actually want to view ribbons/atoms/bonds for each copy, I don't know of any mechanism to save memory. Also, to use Multiscale Models in this way, you would have to generate an input PDB file with matrices describing all of the transformations, which sounds difficult. Yet another way to handle multiple coordinate sets for the same atoms is as a trajectory. However, then you can only view one member of ensemble (one "time step" from the trajectory) at a time, so that also would not solve your problem.
I CC'd Tom and Eric in case they have any ideas. Best, Elaine
Thank you Tom and Elaine for your detailed explanations! Since the main point is to fit a high resolution model in a low resolution map, and considering the fact that the multiscale tool does not interoperate easily with fittings in maps, I will rather use C-alpha coordinates only as you suggest Tom. I have just tried it and it made a great improvement! Also I tested fitting before and after removing the non CA atoms and it did not make a significant difference. I was wondering, if after fitting say 50 Calpha chains I wanted to replace them by the full molecular model (for showing in a talk for example), would there be an easy way to do that? Cheers, Ben Thomas Goddard wrote:
Hi Ben,
The main trick to improve efficiency in Chimera with many copies of a molecule is to use only the C-alpha coordinates. This is usually acceptable with working with EM data resolutions. You create a PDB file with only the CA atoms (can use "select @CA" command and Chimera File / Save PDB...) reducing the total number of atoms by 10x or more.
Chimera is a very memory inefficient with molecules taking about 2 Kbytes per atom. This is about 10x worse than PyMol and VMD. You start feeling this at about 100,000 atoms and up and run into real trouble around 500,000 atoms if you are using 32-bit Chimera (no 64-bit available on Windows or Mac) because you run out of address space. The ability to rotate the models depends on rendering speed. Above 100,000 atoms only "wire" style will be fast and you must have a graphics driver installed (not Mesa on Linux). With a 4 year old graphics card (Radeon 9800 Pro) I get 10 frames per second with 200,000 atoms. With a newer card you should handle 500,000 atoms in wire at reasonable frame rates.
Chimera session files include all the PDB atom coordinates. Chimera has no way to show 30 copies of a PDB file at different positions without having 30 independent copies of the data, except using surfaces with the multiscale tool. The multiscale tool does not interoperate easily with fitting in maps.
Tom
Elaine Meng wrote:
Begin forwarded message:
From: Benoit Zuber <bzuber@mrc-lmb.cam.ac.uk> Date: October 2, 2008 10:24:34 AM PDT To: Elaine Meng <meng@cgl.ucsf.edu> Subject: Re: [Chimera-users] volume viewer window
Hi Elaine, I just aligned a tiny subtomogram to a model with a lot of different parameter, which generated many volumes. Then I wanted to visualize all the results at once. Since each volume was tiny, chimera was not struggling, it was fine :-)
However that reminds me another issue about memory I had sometime ago. I have a tomogram into which I have fitted many copies (more than 30) of the same pdb file. There are two problems: 1) compiling the session file takes ages (around 10 minutes I would say). Opening the compiled session file takes more than 1 minute. 2) if I display all the copies of the pdb, everything is very slow (I mean if I want to move anything in space, it takes very long). If I display only a few copies (plus 1 or two slices through the tomogram), then it works fast.
For the 2nd problem, there is maybe no easy solution to speed up the calcultations, but for the first one, since we are dealing with copies of a single pdb, wouldn't it be possible to make the process faster? I reckon that the coordinates of each atom of each copy of the pdb are saved. Would there be a way to save only one copy and transformation matrix for every copy? Would it speed up file saving-compiling-opening operations?
Cheers, Ben
Hi Ben, I know Tom Goddard has made many recent improvements in Volume Viewer memory usage and speed, but it sounds like your main issue is many atomic structures rather than lots of volume data.
If the PDB copies are identical except in transformation, there is some capability in Multiscale Models to only store one copy of the coordinates and show the others as low-res surfaces. However, as soon as you try to show any more detailed display such as ribbons or atoms and bonds, the full coordinates for that copy are loaded. If you actually want to view ribbons/atoms/bonds for each copy, I don't know of any mechanism to save memory. Also, to use Multiscale Models in this way, you would have to generate an input PDB file with matrices describing all of the transformations, which sounds difficult. Yet another way to handle multiple coordinate sets for the same atoms is as a trajectory. However, then you can only view one member of ensemble (one "time step" from the trajectory) at a time, so that also would not solve your problem.
I CC'd Tom and Eric in case they have any ideas. Best, Elaine
-- Benoît Zuber MRC Laboratory of Molecular Biology Hills Road Cambridge CB2 0QH United Kingdom +44 1223 402209 bzuber@mrc-lmb.cam.ac.uk
On Oct 3, 2008, at 1:31 AM, Benoit Zuber wrote:
I was wondering, if after fitting say 50 Calpha chains I wanted to replace them by the full molecular model (for showing in a talk for example), would there be an easy way to do that?
Hi Ben, This probably does not qualify as easy, but I would just open the full copies and transform them to match their respective alpha-carbon traces, then close the alpha-carbon trace models. Definitely save a session file before doing this part! Something like: matrixcopy #0 #50 (apply transformation matrix of CA model 0 to full model 50, which assumes their untransformed CA coords are identical) <http://www.cgl.ucsf.edu/chimera/docs/UsersGuide/midas/matrixcopy.html> or match #50@ca #0@ca (fit CA atoms of model 50 to those of model 0, which assumes the models have the same number of CA atoms in the same order) <http://www.cgl.ucsf.edu/chimera/docs/UsersGuide/midas/match.html> then possibly close 0 You could make a Chimera command file to do all 50. Alternatively, for the programming expert, there must be a sneaky way to substitute the full coordinates of each structure for the CA-trace coordinates within the python session file, but this is beyond my skill set, and session-editing can be a risky sport. Best, Elaine ----- Elaine C. Meng, Ph.D. meng@cgl.ucsf.edu UCSF Computer Graphics Lab (Chimera team) and Babbitt Lab Department of Pharmaceutical Chemistry University of California, San Francisco http://www.cgl.ucsf.edu/home/meng/index.html
Hi Ben, Elaine's solution would be done by making a Chimera command file (suffix ".cmd") with lines open 100 allatoms.pdb ; matrixcipy #0 #100 ; close #0 open 101 allatoms.pdb ; matrixcipy #1 #101 ; close #1 open 102 allatoms.pdb ; matrixcipy #2 #102 ; close #2 .... To save some typing you can make a Python script (file suffix ".py") instead like: from chimera import runCommand for i in range(50): runCommand('open %d allatoms.pdb ; matrixcipy #%d #%d ; close #%d' % (100+i, i, 100+i, i)) Opening that file with Chimera File/Open... to replace all your models. Tom Elaine Meng wrote:
On Oct 3, 2008, at 1:31 AM, Benoit Zuber wrote:
I was wondering, if after fitting say 50 Calpha chains I wanted to replace them by the full molecular model (for showing in a talk for example), would there be an easy way to do that?
Hi Ben, This probably does not qualify as easy, but I would just open the full copies and transform them to match their respective alpha-carbon traces, then close the alpha-carbon trace models. Definitely save a session file before doing this part! Something like:
matrixcopy #0 #50 (apply transformation matrix of CA model 0 to full model 50, which assumes their untransformed CA coords are identical) <http://www.cgl.ucsf.edu/chimera/docs/UsersGuide/midas/matrixcopy.html>
or
match #50@ca #0@ca (fit CA atoms of model 50 to those of model 0, which assumes the models have the same number of CA atoms in the same order) <http://www.cgl.ucsf.edu/chimera/docs/UsersGuide/midas/match.html>
then possibly
close 0
You could make a Chimera command file to do all 50. Alternatively, for the programming expert, there must be a sneaky way to substitute the full coordinates of each structure for the CA-trace coordinates within the python session file, but this is beyond my skill set, and session-editing can be a risky sport. Best, Elaine ----- Elaine C. Meng, Ph.D. meng@cgl.ucsf.edu UCSF Computer Graphics Lab (Chimera team) and Babbitt Lab Department of Pharmaceutical Chemistry University of California, San Francisco http://www.cgl.ucsf.edu/home/meng/index.html
_______________________________________________ Chimera-users mailing list Chimera-users@cgl.ucsf.edu http://www.cgl.ucsf.edu/mailman/listinfo/chimera-users
participants (4)
-
Benoit Zuber -
Elaine Meng -
Thomas Goddard -
Tom Goddard