Pruning for repeated workflow calls

[SimonHeybrock]

This is mainly a note about something we could do relatively easily when moving to a more graph-based approach (see recent work). Frequently, we may want to run the same workflow on a series of inputs. However, not all inputs may change, and we would like to avoid re-computation of unchanged data. This can be done quite easily, I think:

• Given a task graph and input keys that are updated, find all affected nodes.
• Find all unaffected parents of affected nodes.
• Compute all the unaffected parents.
• Set these computed results in the pipeline.
• Rebuild the graph.

This is not a suggestion to implement the above in Sciline, rather something that should guide Sciline's data structure and interface design, to make it easy.

[SimonHeybrock]

Not exactly the same, but have a look at https://scipp.github.io/essreduce/generated/modules/ess.reduce.streaming.html#module-ess.reduce.streaming for an implementation of a very similar feature, on top of Sciline.