ImageSequence does not allow per-image models

[mpks]

Hi,
I am writing a format class that uses ImageSequence to set the per-image beam model. I looked into imageset.h file, and there all the set functions from the ImageSet class (set_beam_for_image, set_detector_for_image, etc) are overwritten in the ImageSequence class to throw DXTBX_ERROR("Cannot set per-image model in sequence");

So it seems that it is not possible to use per-image models with the ImageSequence.

[dagewa]

This follows discussion at the DIALS core meeting, in which I think the conclusion was that we believe ImageSequence should allow this behaviour, however it currently seems it does not by design.

[phyy-nx]

Without re-reading the code, it seems the solution would be to add an array of detector/beam/gonio models, which starts to smell a lot like an ImageSet. What's the real benefit of an ImageSequence in this case?

[mpks]

The main difference between ImageSet and ImageSequence is in the time it takes to load data (for example, with dials.image_viewer). A sequence takes a fraction of a second, whereas a set takes much longer.

Based on the discussion from last week's meeting, I got the impression that ImageSequence solves some of the problems of the ImageSet (the time it takes to load). The talk was that ImageSet is an older class, and that we should work to make it deprecated in the future. However, looking at the code, it seems that ImageSequence is just an additional name/label for a special case of ImageSet (meaning it doesn't add any extra functionality, except restricting everything to a single model).

I am still puzzled as to why one is so much faster than the other, so I'll spend some time trying to understand the difference. The reason may be that one is setting all the models in the C++ part of the code, whereas the other relies on Python. I am not sure does the speed depends on the restriction to a single model for all images, but I can make an ImageSet with a single beam and a single detector model, and it's still slow.

I still think the difference is somewhere in the Python vs. C++ part of the code. For example, the get_imageset function from the MultiImage format is called 900 times when reading my dataset of 300 images as an ImageSet. It gets called twice if I read it as a sequence.

[phyy-nx]

The usage of scan objects as a proxy for the ImageSet for stills is the key to the speed of the ImageSequence. But maybe the answer is to ditch the idea of models being stored on the ImageSet/Sequence and use the experiment list storage of the models instead. Something up like this:

dials.import dataset.h5 convert_stills_to_sequences=True
dials.python duplicate_the_models.py imported.expt

Where duplicate_the_models is

from dxtbx.experiment_list import ExperimentList
import sys, copy

experiments = ExperimentList.from_file(sys.argv[1])
detector = experiments[0].detector

for expt in experiments:
  expt.detector = copy.deepcopy(detector)

experiments.as_file('duplicated_models.expt')

The resultant expt file should have N experiments, N scans, 1 beam, 1 ImageSequence, N detectors.

Then, use this experiment list for downstream stuff. Is this useful?

[dagewa]

Hi @phyy-nx I don't understand why the models are stored on the ImageSet anyway. I understand ExperimentList, which has a pointer to the data location, and when you need to read the data (check_format=True) you have a format class with a method that allows the read of that data. To me, the ExperimentList is the primary object, and I have pretty much avoided using ImageSets wherever I can in code. I'm not even sure what an ImageSet is for.

Thanks for your example. I'm sure we can work around dials.import issues in any number of ways, but what we are very purposely trying to do with this project is provide canonical DIALS processing instructions for collaborators. We very much want to avoid bespoke workarounds.

[phyy-nx]

100% agree. Dropping the extra copy of the models stored on the imageset makes sense to me. Using it only as a pointer to the pixel data seems best.

[mpks]

I tried the workaround proposed by Aaron, and indeed, the duplicated_models.expt shows a different detector model for each experiment. However, when you write a different beam position for each detector, the image_viewer shows only one beam for all images. So the expt file is as expected, but somewhere in the processing, it seems that all these models get replaced with a single one.

Regarding your proposal to drop models from the imageset, I found this old discussion with the same conclusion. I will look more into this and try to figure out the solution.

[biochem-fan]

I encountered a related problem when I tried to use dials.ssx_index on an XFEL dataset, whose wavelength varied shot by shot. I reported it in dials/dials#2725. I made a PR to address this issue in dials.import dials/dials#2865, which unfortunately wasn't merged (why?). I think the per-image detector model can be implemented similarly. But if this is for drifting direct beam positions, it should be implemented as different beam vectors, not detector origins, shouldn't it?

[mpks]

We have both solutions for a drifting beam (one with a different detector model, and one with different beams), both using ImageSet. Although the beam position is the same for both approaches, there are still slight differences. With a changing beam model, the incoming angle on the detector is no longer 90 degrees, so the resolution rings turn into ellipses. The detector distance is usually much larger than the detector dimensions, so the deviation from the right angle is usually small.

[jbeilstenedmands]

Does the ImageSet also store the models because a long time ago, I recall that spotfinding produced the DataBlock data structure (based around the ImageSet?) and ExperimentLists were only created after indexing? Of course if you only had the ImageSet at spotfinding, you still needed a way to access beam, detector models etc. through it. Then we migrated away from DataBlocks in favour of the common ExperimentList structure everywhere, and perhaps the ImageSet storing models does not give any further functionality?

[biochem-fan]

@mpks Is this for serial electron diffraction? If so, changing the beam direction is the right approach, because the incident beam is indeed tilting albeit slightly. If the beam-tilt pivot point alignment of the electron microscope was perfect, beam shift would not induce any beam tilt and the direct beam position on the detector (in the diffraction mode) would remain stationary. In reality, pivot point alignment is never perfect and beam shift to scan crystals induces some beam tilt.

the resolution rings turn into ellipses

dials.image_viewer supports iso-resolution ellipses (and even hyperbolas!).

The detector distance is usually much larger than the detector dimensions, so the deviation from the right angle is usually small.

Yes. The effect of oblique incidence should be tiny.

[mpks]

@biochem-fan Yes, this was for serial electron diffraction.
@jbeilstenedmands Yes, the ImageSet keeps the models. This is probably why overwriting the experiment list does not work.