Experiment config_hook does not change ingredient's config

[cmditch]

It seems that any ingredient level changes within an experiment's config hook do not actually alter the ingredient's config. It seems they are being overwritten right about here.

Code to reproduce the issue:

import sacred

feats = sacred.Ingredient("feats")
feats.add_config({"name": "feats"})

ex = sacred.Experiment("coury_sacred_test", ingredients=[feats])
ex.add_config({"name": "experiment"})


@ex.config_hook
def ex_config_hook(config, command_name, logger):
    config["feats"]["name"] = "feats_changed"
    print("Config hook changes:", config)
    return config


@ex.automain
def main(_run):
    print(_run.config)

Output:

> python scratch_sacred_config_hook_debug.py print_config

Config hook changes: {'name': 'experiment', 'feats': {'name': 'feats_changed'}}
INFO - coury_sacred_test - Running command 'print_config'
INFO - coury_sacred_test - Started
Configuration (modified, added, typechanged, doc):
  name = 'experiment'
  seed = 149828750
  feats:
    name = 'feats'
INFO - coury_sacred_test - Completed after 0:00:00

[thequilo]

Hi @cmditch! Thanks for reporting this. Do you have an idea how to fix this issue and would prepare a PR? Unfortunately, I currently don't have much time to work on this.

[cmditch]

Hi @cmditch! Thanks for reporting this. Do you have an idea how to fix this issue and would prepare a PR? Unfortunately, I currently don't have much time to work on this.

I can't say I have a solution atm. I've dug into the code a fair amount and attempted some fixes, but the config update logic is wildly difficult to track since there's so many ways the config can be altered.

[thequilo]

You are right, the config resolution code is (too?) complex. I looked at it and there is a larger issue with config hooks.

If you add a no-op hook to the ingredient, strange things happen (the config is added twice?)

import sacred

feats = sacred.Ingredient("feats")
feats.add_config({"name": "feats"})

ex = sacred.Experiment("coury_sacred_test", ingredients=[feats])
ex.add_config({"name": "experiment"})


@feats.config_hook
def ex_config_hook(config, command_name, logger):
    return config


@ex.automain
def main(_run):
    print(_run.config)

Config hook changes: {'feats': {'name': 'feats'}}
WARNING - coury_sacred_test - No observers have been added to this run
INFO - coury_sacred_test - Running command 'main'
INFO - coury_sacred_test - Started
{'name': 'experiment', 'seed': 705608531, 'feats': {'name': 'feats', 'feats': {'name': 'feats'}}}
INFO - coury_sacred_test - Completed after 0:00:00

I'll have to look into this