Hi Lori, If you had enough memory the resample operation would be pretty fast (I'd guess a few tens of seconds). But the trouble is that Chimera is doing this operation in a very memory inefficient way. What it does is determine the x,y,z coordinates of all the grid points and then interpolates the volume at those points. The trouble is that those coordinates are 3 floats taking 12 bytes. Also it makes a floating point copy of the interpolated values for another 4 bytes per grid point. If your 2 Gbyte map is single byte values then these extra arrays take 16 times more memory, 32 Gbytes. If your maps are 4 byte floats you still use 4 times the memory that the map does (8 Gbytes). If this stuff doesn't fit in memory it starts getting paged to disk and it may be 100 times slower. The only reasonable answer here is that Chimera should not allocate memory for all the xyz grid points. It's doing that because the code is in Python/numpy instead of C++. Other code in Chimera instead works one data plane at a time. This reduces the temporary memory use for those xyz grid points and if the whole calculation can be done in memory it is much much faster. I just made that change for resampling and it will be in tomorrow's daily build. It won't be in the upcoming Chimera 1.4 release since are only making bug fixes to that version. The memory use is still quite high. It makes an extra temporary copy of the interpolated map with floating point values (4 bytes per grid point). That is unnecessary. But it's a bit more trouble to optimize that away so I'm not going to do it now. But I'll add it to the Chimera request list. With the change I made it should use 4 bytes per grid point temporary memory instead of 16 bytes and if your 64-bit Linux system has enough physical memory (8 or 16 Gbytes) the resample will run much faster. Tom -------- Original Message -------- Subject: [Chimera-users] binning large volumes From: Lori Nyland To: chimera-users Date: 10/28/09 1:07 PM
Hello,
I have two volumes with the same dimensions and each are nearly 2GB. I can use the vop resample command in a 64 bit Chimera for Linux and reassign one volumes coordinates relative to the other. However, it is slow to execute, but it does work. In order to speed things up is there a way to bin the volumes (to use less space), place it through the vop resample command, and then un-bin the volume without losing information?
Thanks! Lori