Skip to content

Conversation

@rsmmr
Copy link
Member

@rsmmr rsmmr commented Apr 28, 2025

Syntax:

module Foo {
    option MySwitch = False;
    option MyValue: uint64 = 5;

    [...]

    MySwitch = True;
}

Options are meant to capture a tunable parameter where a parser picks
a suitable default for something, but allows changing that value
externally. An option is something "in between" a global constant
and a global variable: Like a global constant, it's "semantically
constant" in the sense that it's something that's explicitly set to
some chosen value; it's not meant for dynamically collecting state.
But like a global, its value can change during runtime.

Access to options works through normal scoping, so that their values
can modified externally through assignments inside other modules, in
particular at initialization time through global statements. If
multiple modules change an option at init time, it's undefined which
change will win out.

This isn't fully finished yet, and up for discussion at this point.
Some notes/thoughts:

- I have tested that this works from the Zeek side to feed
  script-level values back into a parser through global statements
  (`Foo::MyValue = zeek::get_count("Foo:MyValue");`)

- This commit currently does not implement the equivalent of
  `cxx_enable_dynamic_globals` for options. Like globals, options
  aren't thread-safe by default, and we'd need to take similar
  measures to make them so. In fact, at that point, options could
  just be merged into the set of globals during codegen, although
  we'd get similar overhead then. Alternatively, because changes
  are supposed to be rare, if we'd allow only assignments to
  change a value, it might be viable to use actual locks around
  accesses, not sure.

- The similarity to globals on the implementation side suggests
  that options are really primarily a semantic language construct,
  but nothing fundamentally new. I'm kind of torn if we actually
  need them, or should simply use globals for tunable parameters.

- I considered making options read-only during runtime, allowing
  changes only at module initialization time (i.e., through global
  statements). However, that would prevent updating them from
  inside a change handler for a Zeek-side option, which would be
  unfortunate.

TODO:
- Do we want this?
- More tests
- Documentation

Syntax:

    module Foo {
        option MySwitch = False;
        option MyValue: uint64 = 5;

        [...]

        MySwitch = True;
    }

Options are meant to capture a tunable parameter where a parser picks
a suitable default for something, but allows changing that value
externally. An `option` is something "in between" a global constant
and a global variable: Like a global constant, it's "semantically
constant" in the sense that it's something that's explicitly set to
some chosen value; it's not meant for dynamically collecting state.
But like a global, its value *can* change during runtime.

Access to options works through normal scoping, so that their values
can modified externally through assignments inside other modules, in
particular at initialization time through global statements. If
multiple modules change an option at init time, it's undefined which
change will win out.

This isn't fully finished yet, and up for discussion at this point.
Some notes/thoughts:

    - I have tested that this works from the Zeek side to feed
      script-level values back into a parser through global statements
      (`Foo::MyValue = zeek::get_count("Foo:MyValue");`)

    - This commit currently does not implement the equivalent of
      `cxx_enable_dynamic_globals` for options. Like globals, options
      aren't thread-safe by default, and we'd need to take similar
      measures to make them so. In fact, at that point, options could
      just be merged into the set of globals during codegen, although
      we'd get similar overhead then. Alternatively, because changes
      are supposed to be rare, if we'd allow only assignments to
      change a value, it might be viable to use actual locks around
      accesses, not sure.

    - The similarity to globals on the implementation side suggests
      that options are really primarily a semantic language construct,
      but nothing fundamentally new. I'm kind of torn if we actually
      need them, or should simply use globals for tunable parameters.

    - I considered making options read-only during runtime, allowing
      changes only at module initialization time (i.e., through global
      statements). However, that would prevent updating them from
      inside a change handler for a Zeek-side option, which would be
      unfortunate.

TODO:
    - Do we want this?
    - More tests
    - Documentation
@bbannier
Copy link
Member

This seems to be practically about #765 which I just linked.

@rsmmr
Copy link
Member Author

rsmmr commented May 6, 2025

Some notes on the plan going forward:

  • Conceptually, the notion of options seems valuable to have in addition to (and in distinction to) global variables.
  • The missing piece to really make this something novel, is an API to modify these values from inside a host application, so that one can change values dynamically, like from inside a Zeek change handler triggered by its configuration framework.
  • To make that work, we'll need a global runtime registry of all available options (which can then also be queried, "what options do my current analyzers provide?").
  • Semantically option values should be visible across HLTO boundaries: if two modules both define the same option (e.g., HTTP::MyTuningParameter) a change will be visible to both. This is different from global variables.
  • If/when we get HILTI-level threads, all threads will likewise see the same option values--which is again different from HILTI's globals, where each thread gets its own copy. This means we'll need to lock around accesses to options, which should be fine given that write operations will be very rare (hence basic spin locks should do well).

Next step: Write this up a bit more formally, with proposals for syntax and API.

@rsmmr
Copy link
Member Author

rsmmr commented May 6, 2025

I considered making options read-only during runtime, allowing
changes only at module initialization time (i.e., through global
statements). However, that would prevent updating them from
inside a change handler for a Zeek-side option, which would be
unfortunate.

If we get a host API to change values instead, maybe we can go back to this and allow Spicy-side changes only during module initialization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants