- Chimera-users - RBVI Mailing Lists

Representation of volume data in memory
by Thomas Goddard 30 Jan '07

30 Jan '07

Hi Jeff, Chimera keeps volume data in memory using the same numeric value types that are present in the file, for example, 32-bit float, 16-bit integer, 8-bit integer. Volume data is copied only when necessary -- for instance, using the Gaussian Filter tool creates a floating point copy of the data since Gaussian convolution of integer data produces a floating point result. Tom Jeff wrote: > does chimera store all values as float internally (even if the original > map is in 2 byte integers), making the mode of the MRC file irrelevant > to chimera's performance? > > -jeff

1 0

Re: [Chimera-users] zone tool and volume rendering
by Thomas Goddard 30 Jan '07

30 Jan '07

Hi Jeff, I'll tell you my experience looking at a tomogram (2048 x 2048 x 76) of an immune synapse for a demonstration I gave last month at a cell tomography workshop. The file (mrc) is too large to conveninently display at full resolution. So I made binned versions 2x and 4x smaller along each axis (file size 8x or 64x smaller). Then I used the precomputed subsamples feature I described before. For some added convenience I then saved the map to a NetCDF file using Chimera because this saves the full matrix and binned versions in a single file (new in Chimera 1.2318). I use the volume dialog data display options to "Adjust step size to show at most 1 Mvoxels", or however many million voxels my computer can handle without getting too slow. Then I use the full view of the data to find regions I'm interested in, then use the subregion selection feature and Crop button to look at a smaller piece and the step size automatically is reduced. If I choose a small region I get full resolution. When doing this you have to keep in mind that once you manually change the step size at the top of the volume dialog it turns off the automatic step size adjustment. So I try to avoid that. Of course I also use the volume planes tool. I think you know all this and probably are using Chimera to the best of its current capabilities. You want to push it to use better compression (like resampling to have better aligned boxes) so you can see more data at full resolution. We don't have that yet. It is not hard to envision much fancier compression implementations that improve performance. Chandra Bajaj at U. Texas is very interested in this and may have software you can use. You might want to try his VolumeRover program. The Chimera EM capabilities were developed for single particle reconstructions. We have done little in Chimera to support work with EM tomograms which are very different in size and content (non-axis aligned structures, membranes, ...). We plan to develop tomogram capabilities and my volume planes tool created last month is a tiny step. But it will take years to implement the obvious stuff we can think of now. Your comments are helpful in guiding the priorities. I don't understand why you are "jotting down displacements" when saving subregions. Chimera puts that info into the MRC header. So why do you need to write it down? Tom Jeff wrote: > thanks Tom, > > with regards to the rotation/resampling, I was thinking mainly in > terms of visualizing large #'s of filaments and membrane, which in > themselves only constitute a small fraction of the actual volume they > traverse. even if re-sampling were done by cubic interpolation onto a > the same dimensioned grid, in some cases the loss in resolution would > be acceptable (definitely would be acceptable for some of my current > needs). i have problems with visualizing my whole maps unbinned but > was thinking that the savings such an approach would represent could > allow me to do that. for instance, this is already a problem for me > with trying to volume render a slab of membrane, which I think might > contain interesting features but I can't get the whole thing to go > unbinned; if it were rotated into the minimal enclosing volume, > interpolation artifacts or otherwise, my feeling was that I might > overcome that (there are also a bunch of filaments attached to the > membrane, and running underneath the membrane, so it is a real > difficulty for me to go back and forth between cutting out subvolumes > and binning them to be manageable, segment out filaments, or membrane, > etc, then try to put all the segmented items back into the same volume > and have a look-see). > > the volume planes tool has proven very helpful in segmenting out the > membranes, though. to get at the filaments that are attached to the > membrane, I was zoning out a region around the membrane (placed a > bunch of markers on the membrane using volume planes tool) about 400 A > in radius, but this proved difficult to render in the entire tomogram > b/c the zoned membrane still encompassed most of the original volume, > etc. so I was back to having to cut regions out of that segmented > membrane map to look further. (thus necessitating me jotting down the > displacements of each subregion, etc.) > > -jeff

1 0

Re: [Chimera-users] using a marker file to zone a volume from python prompt
by Thomas Goddard 29 Jan '07

29 Jan '07

Hi Jeff, I forgot to mention that to run the volumezone.py script without display the Chimera user interface you would invoke it from a shell: % chimera --nogui volumezone.py The --nogui flag tells Chimera not to start any graphical user interface. Tom

1 0

Re: [Chimera-users] zone tool and volume rendering
by Thomas Goddard 29 Jan '07

29 Jan '07

Hi Jeff, You are right that masking a filament that runs along the long diagonal of a box does not reduce the disk space or memory use any. (If you compressed the file then it would make a big difference. But you need to uncompress it to use it so that is of limited value.) So to achieve a space savings you need to resample the volume using a grid aligned with the axes of the filament. The resampling will degrade the data resolution some and it might be reasonable to do it on a grid with half the spacing (making the file 8x bigger). With such a resampled map the rotation and shift could be stored so that it can be placed in the original location in the full map. We don't support that now but when we add resampling we will store those rotation parameters in the map file header (maybe in the "labels" field for an MRC file). The time for Chimera to display a map depends on how large the volume grid is because it scans the entire grid in one pass when using the marching cubes algorithm to compute a contour surface. It also depends on how many triangles make up the contour surface. By eliminating the 99% of a masked file that is zeros using a new orientation and resampling it could compute contour surfaces faster. And the solid (volumetric) rendering should be faster directly proportional to the file size. That may be the more important case for your EM tomograms. I plan on supporting resampling on a rotated grid. But I think in many real uses it will not save much space because you will want to resample with finer grid spacing. But it could be convenient for exporting to analysis or visualization software that is limited to xy-planes. Tom Jeff wrote: > got it, > > what you say makes sense, I can definitely workaround, especially given > that the zone tool does a mask upon saving. > > I have been thinking about the problem I have with filaments (or > surfaces) spanning non-orthogonal regions of a volume, which means that > zoning out a long filament (or surface) that might take a much smaller > amount of space/memory once zoned actually takes up almost the same > amount currently, if I am thinking right. > > as an example, say you have a volume that is 100x100x100. inside the > volume, you have a filament, that takes up 10x10x244 voxels (effectively > spanning from one corner to the mirror/opposite corner of the volume). > in theory, zoning out that filament could take up ~1/40th the space, if > rotated to line up with one of the cartesian axes. I don't know how much > logic it would take to do this; I also don't know how chimera reads > origin information from an MRC file, or how much work it would take to > enable it to read (and use) an euler convention from an MRC file, for > displaying later. This would enable cutting a whole bunch of filaments > out of a volume, and then displaying them all concurrently without > running into the problem of loading in large rectangular prisms of space > that are mainly zeros (if this is currently happening, I don't know). I > guess at that point it makes sense to consider the geometric center of > such a rotated object at its origin, to reduce re-displaying the > object/filament properly in the original orientation to just storing a > shift and the rotation angles (6 parameters). > > the same holds true for membranes - right now, I have an entire membrane > that I segmented with markers, but it essentially runs along a hypotenuse. > > one question that probably negates the above paragraph would be 'does > the masked volume (with mostly zeros) display any faster / take less > resources than an equivalent volume without the zeros'? I have been > assuming this is not the case, that the zoned/masked volume still > entails reading in / accounting for that full rectangular prism, despite > the fact that it is mostly empty / full of zeros. > > thanks for you help, > > -Jeff

1 0

Re: [Chimera-users] using a marker file to zone a volume from python prompt
by Thomas Goddard 29 Jan '07

29 Jan '07

Hi Jeff, I've attached a script that opens a density map and a Chimera marker file and writes a new volume corresponding to a zone around the markers. It does not require showing the Chimera user interface to do this. Perhaps you don't need this now (I just got your follow-up email.) but perhaps you will find it useful in writing other scripts. Karin Gross at Max Planck Institute of Biochemistry has written a Chimera extension to slice tomograms at arbitrary angles. I saw it when I visited Munich in December. They also were planning on allowing marker placement on those planes. The volume planes tool could be made to average neighboring planes. Currently it simply displays a NxMx1 subvolume in solid rendering mode. Since there is no existing code in Chimera to support averaging adjacent planes that would have to be added. Probably the easiest solution is to add the adjacent planes in Python and create a new single plane volume on the fly. Tom Jeff wrote: > thanks Tom, > > I will try that. alongside that, is there a way for me to run a 'zone' > function and save a subregion from a python command line (no chimera > launched, just import a library?)? > > I have used the volume planes tool to generate a marker set of a > membrane, it was handy. But, it only handles orthogonal directions, it > can't provide an arbitrary interpolant (which is a large reason I rely > heavily on IMOD's slicer tool); this functionality would be very useful. > A second addition to the volume planes tool would simply be allowing > averaging several slices together (reduces noise) in the direction > normal to the planes, which may already be there (the documentation > online said something about this feature, but it wasn't clear to me) > > it the volume planes tool were able to handle an arbitrary > view/interpolant, then I wouldn't have to depend on the slicer tool, and > the previous request about rotating markers that was related to that > requirement (different features available in different slicer > orientations in IMOD, necessitating a rotation prior to visualization > with the volume planes tool in chimera, thus requiring old marker set > from original volume to be rotated to be used in new volume, etc) would > effectively be voided > > > -Jeff # ---------------------------------------------------------------------------- # Save a volume zone around a given set of markers as a new volume file. # def save_zone(volume_path, markers_path, radius, volume_out_path): # Read zone points atoms, bonds = marker_set_atoms_and_bonds(markers_path) from chimera import Xform xform = Xform() # Identity transform for marker points from SurfaceZone import path_points points = path_points(atoms, bonds, xform) from VolumeData import open_file_type vol = open_file_type(volume_path, 'mrc') # Get containing volume subregion ijk_min, ijk_max = containing_subregion(vol, points, radius) from VolumeData import Grid_Subregion subvol = Grid_Subregion(vol, ijk_min, ijk_max) # Make new volume with zeros outside radius from VolumeViewer import zone_masked_grid_data masked_vol = zone_masked_grid_data(subvol, points, radius) # Write new volume from VolumeData.mrc import write_mrc2000_grid_data write_mrc2000_grid_data(masked_vol, volume_out_path) # ---------------------------------------------------------------------------- # def marker_set_atoms_and_bonds(markers_path): from VolumePath.markerset import load_marker_set_xml msets = load_marker_set_xml(markers_path) atoms = [] bonds = [] for mset in msets: atoms.extend(map(lambda m: m.atom, mset.markers())) bonds.extend(map(lambda l: l.bond, mset.links())) return atoms, bonds # ---------------------------------------------------------------------------- # def containing_subregion(vol, points, radius): from VolumeViewer import points_ijk_bounds, clamp_region ijk_min, ijk_max = points_ijk_bounds(points, radius, vol) ijk_min, ijk_max = clamp_region((ijk_min, ijk_max, None), vol.size)[:2] from math import floor, ceil ijk_min = map(lambda x: int(floor(x)), ijk_min) ijk_max = map(lambda x: int(ceil(x)), ijk_max) return ijk_min, ijk_max # ---------------------------------------------------------------------------- # volume_path = '/usr/local/src/chimera-demos/volume/examples/tricorn_map.mrc' markers_path = '/usr/local/src/staff/goddard/chimera/devel/ZoneScript/test.cmm' radius = 5 volume_out_path = '/usr/local/src/staff/goddard/chimera/devel/ZoneScript/test.mrc' save_zone(volume_path, markers_path, radius, volume_out_path)

1 0

access to pdb parser
by Jean Didier Pie Marechal 29 Jan '07

29 Jan '07

2 1

Re: [Chimera-users] zone tool and volume rendering
by Thomas Goddard 29 Jan '07

29 Jan '07

Hi Jeff, The volume dialog "zone" feature does not work with solid (volumetric) rendering. The current implementation just hides surface triangles having vertices outside the given range so it only works on surfaces. When you use volume dialog File / Save map as... it makes a copy of the data with grid values set to zero outside the radius. Creating a copy of the map in memory or masking the map for use with solid rendering are not currently supported. We wish to add this in the future and it should not be hard to achieve although performance when changing the radius will be significantly worse than with the current surface zoning. The documentation does not mention this limitation and I will update the volume dialog description to describe it. Tom Jeff wrote: > hi Tom, > > is there a way to get the zone tool to operate when doing a solid/volume > rendering - in surface mode, if you set a zone radius, let's say 40 A, > you'll get a region that extends 40 A around the selected marker(s); > however, in 'solid' mode, it appears I get the entire orthogonal region > in the volume that encloses the markers plus that radius. thiis is > problematic if the markers span the hypotenuse of the volume; in that > case, zone with volume/'solid' appears to more or less display the > entire volume rather than just the surrounding region as in 'surface mode' > > do I have a buggy version? this is a recent release, but I've had this > happen with previous releases > > -jeff

1 0

Re: [Chimera-users] chimera startup problem
by Greg Couch 29 Jan '07

29 Jan '07

Just a FYI to the list, the problem here was an ancient OpenGL graphics driver. Upgrading it made the problem disappear. I'll be adding an entry to our Graphics Driver bugs web page, <http://www.cgl.ucsf.edu/chimera/graphicsbugs.html> with a few more details. - Greg ---------- Forwarded message ---------- Date: Tue, 23 Jan 2007 12:25:41 -0500 From: Chen Xu <xuchen66(a)gmail.com> To: chimera-users(a)cgl.ucsf.edu Subject: [Chimera-users] chimera startup problem Dear List, after upgrading to 1.2304 (1.2303 freeze the computer after several minute loaded with a volume pdb file) , I have problem to start it up on (RedHat): agnieszka@raptor ~]$ chimera Error starting chimera: ValueError: attempt to create string with null pointer (see reply log for Python traceback info) Traceback (most recent call last): File "/sware/chimera/chimera-1.2304/share/__main__.py", line 59, in ? value = chimeraInit.init(sys.argv) File "CHIMERA/share/chimeraInit.py", line 255, in init tkgui.initializeGUI(exitonquit) File "CHIMERA/share/chimera/tkgui.py", line 2407, in initializeGUI File "CHIMERA/share/chimera/tkgui.py", line 2368, in createApp File "CHIMERA/share/chimera/tkgui.py", line 1784, in __init__ File "CHIMERA/share/chimera/tkgui.py", line 1811, in makeGraphicsWindow ValueError: attempt to create string with null pointer any help?

1 0

Slow rendering in FC6
by Thomas Goddard 29 Jan '07

29 Jan '07

On Yifan's linux system a large molecule takes a second to redraw and gradually renders from one side to the other. Turned out he did not have the NVIDIA graphics driver installed, so MESA software rendering was being used. Determined this using linux "glxinfo" command. Solution was to install graphics driver. -------- Original Message -------- Subject: problem of running Chimera in FC6 Date: Sun, 28 Jan 2007 13:05:20 -0800 From: Yifan Cheng To: goddard(a)cgl.ucsf.edu Hi Tom, I have a question about running Chimera in Linux FC6 system and wondering if you could help me out? I have Linux boxes running FC6, with a Gigabyte GV-NX76G256D-RH Geforce 7600GS 256MB video card. But somehow the Chimera keep refreshing with any movement in the model or map I loaded into it. It runs this way in both thousands color mode as well as in millions color mode. This video card is not Quadro type and has no stereo function, but I thought this shouldn't matter. Do you have any idea what might be the problem? Thanks a lot for your help. best, Yifan

1 0

Re: [Chimera-users] align different scale models
by Thomas Goddard 29 Jan '07

29 Jan '07

Hi Pu, The PDB file uses units of Angstroms. To specify the spacing between grid planes of your EM map, open it in Chimera, then use menu entry Features / Origin and Scale from the volume dialog. The "voxel size" values listed indicate the size of one grid cell in the density map which is equivalent to the grid spacing. You can change the values and press Return (or Enter) key and the map will be shown at the new scale. Enter the values in Angstroms. (Chimera does not know what units you are using but all data sets must use the same units if you want them to be displayed at the correct relative scales.) The map file is not modified. To save the new scale you either can save a Chimera session (File / Save session as... on the main Chimera window) or save a new copy of the map (File / Save map as... on the volume dialog). Tom P Qian wrote: > Hi Tom, > > I have question to ask you about alignment of PDB model and EM density map. > > When I open both EM density map and its corresponding PDB model, I found they > have different size. I think that is due to different scale they are using. My > question is: Is it possible for me to re-scale them individually so that they > can match each other. I cannot do it at the momemt because zoom function will > affect all data loaded, not only active data but also unactive one. I'm trying > to produce a EM density map which has same scale with PDB model to solve it, > but if Chimera could do it as well, it will be helpful. Thanks > > Best regards > > P. Qian > > Dr. Pu Qian > Physics Department > The University of Sheffield > Hicks Building > Hounsfield Road, S3 7RH > UK >

1 0