Skip to content

Extend performance and data profiles to allow for optimal value #22

@wildsm

Description

@wildsm

In both
https://github.com/POptUS/BenDFO/blob/main/profiling/data_profile.m
and
https://github.com/POptUS/BenDFO/blob/main/profiling/perf_profile.m
the current best function value for each problem is determined entirely from the input data via

prob_min = min(min(H), [], 3);

This is reasonable for most applications, but in some cases one would like to specify what this "problem minimum" should be.
Examples for data profiles include:

  • When solving stochastic optimization problems and/or using a randomized algorithm, one might perform replications. In order to better capture the distribution of the performance due to this stochasticity/randomness, each replication should be treated as a separate problem. In these cases, the "problem minimum" might be better treated as being the least function seen across all solvers and replications.
  • When viewing a single solver's performance in isolation, and when a rough approximation of the desired "problem minimum" is known, using an absolute measure associate with such an external "problem minimum" could be valuable.

Initial work to address this issue could include:

  • Adding an additional optional struct to pass prob_min in
  • Performing a check that the default prob_min value obtained from the data is not exceeded by the passed optional argument. (One could imagine an error or warning identifying this.)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions