Generalize procedure to constrain parameter domain

[redeboer]

Currently, it's only possible to fix or 'unfix' parameters. Minuit allows setting parameter limits, while some of the scipy and nlopt allow more general constraints (bounds argument and non-linear constraints).

[jonas-eschle]

I think there are three things when it comes to constraints, at least what we thought about, practically:

• dependent parameters: If e.g. a parameter is a sum of two actually, then this is "constraint to be always the sum of the two". This I think is best done by implementing it as the actual some, such as these
• box constraints (while they are mathematically a (trivial) special case, they are often available and easy to implemented into the minimizer, many local minimizer even require this): the rectangular restrictions of the parameters, the lower and upper limits. These should help the minimization, but it should not be close to the minimum.
• arbitrary terms added to the likelihood such as these: auxiliary measurements such as a gaussian constraints. These are critical as they will change the minimum, they change the likelihood.

I'll try to get up an example so that you can maybe try out these things directly and see if that fits the need

[redeboer]

Thanks for your thoughts!
With the current setup, the first point would just be a matter of reformulating your expression (subs your parameters with some new parameter). So, ideally, you would just have to tackle the third point, then the box constraints are (as you say) just a special case.
I'm not sure yet about an interface here. The most abstract way would be to feed some expression (possibly only featuring parameter names) with some equality and then internally you reformulate that for the specific optimizer.

[jonas-eschle]

Yes, I agree on 1, that should work out well.

For two, while these are technically just a special case, they serve numerically also an important and quite different purpose: They restrict the search space (and therefore many optimizers profit from it, some global ones even require it. Locals often either need an initial step size or boundaries). Also, the implementation is trivial, compared to 3. But that should not be hard to do in this sense as it is just keeping track of boundaries for the minimizer (so they don't need to be dragged into the computation).

For three, in principle yes. In practice rather not, because many minimizers do not support arbitrary constraints (and if they do, they don't do magic: they just make sure that the costraint is minimally violated). Also, I don't think there is any gain compared to adding it to the likelihood except of it the very best case some speedup. However, it is more complicated usually to interface with the minimizer. Lastly, it has to be somewhere in the likelihood (or be respected by everyone), as even with a minimizer constraint, it indirectly goes into it, just not explicit (which can be dangerous, as maybe it won't be in the error estimate then if it is not explicitly added; the likelihood is our holy grail in the end).
Also, but maybe that was not even meant, e.g. a Gaussian constraint is maybe even harder or impossible to add to a minimizer as it is just an additional likelihood term.