[Requirement] Batch reduction

[olihammond]

Executive summary

A user-friendly process is required for batch reduction of user-specified files

Context and background knowledge

Batch data reduction is a critical requirement of neutron instruments for large-scale structure determination, such as in Small-Angle Neutron Scattering (SANS) and Neutron Reflectometry (NR) experiments, which both involve processing large volumes of data. Currently, Scipp users rely on Jupyter notebook workflows and/or widget interfaces, which are inefficient for handling these types of high-throughput experiments. A dedicated processing interface, which may be a GUI or another user-oriented approach, will streamline the reduction process, improving accessibility, efficiency, and user experience.

Inputs

The precise implementation of this batch processing workflow is handled differently according to specific instrument and facility needs. A common theme is often a table input, such as in a tabular GUI-based system, where the user specifies an array of relevant files and input parameters and hits 'reduce' in fundamentally the same way as they are presently done in the Scipp workflow/widgets. This can also take the form of simple tabular (.csv) input files for Scipp, which specify run numbers for the pertinent values, ie SANS, TRANS, and background file run numbers. More global variables, such as wavelength ranges/bands, detector/monitor nexus files, masks and calibration measurements such as the empty beam which are likely to be held in common across multiple runs, could be specified in each row (cumbersome), or elsewhere in ie the GUI or csv header.

Methodology

A reduction GUI is an attractive potential long-term solution that could eventually be adapted to display and perform additional functionality from a more unified frontend (plotting of reduced data, wavelength/time-slicing, etc). On the other hand, a more basic implementation could involve a capability to take ie .csv or similar tabular data as an input, defining a looped series of individual reductions of different data files; this could then be adapted to a table-based GUI in the long term.

Outputs

Scipp should output the same reduced data as it currently produces; output files of corrected/reduced SANS or NR data, in the currently specified formts, should be saved for each separate run specified in the GUI/table. Instantaneously-generated plots of the reduced data would be a nice-to-have, and/or the option to plot or not.

Which interfaces are required?

Integrated into reduction workflow, Other (please describe in comments)

Test cases

The current set of collected LoKI test data would serve sufficiently for testing of reducing using a single input sent from the user to Scipp.

Comments

No response

[jokasimr]

I think it's better if this is open for anyone to pick up to avoid siloing. I'm happy to assist anyone who picks this up and help them get up to speed on the implementation of the batch reduction gui for reflectometry.