ingest: enable captive core online replay ledger sequence 2 when unbounded

[sreuland]

PR Checklist

PR Structure

• This PR has reasonably narrow scope (if not, break it down into smaller PRs).
• This PR avoids mixing refactoring changes with feature changes (split into two PRs
  otherwise).
• This PR's title starts with name of package that is most changed in the PR, ex.
  services/friendbot, or all or doc if the changes are broad or impact many
  packages.

Thoroughness

• This PR adds tests for the most critical parts of the new functionality or fixes.
• I've updated any docs (developer docs, .md
  files, etc... affected by this change). Take a look in the docs folder for a given service,
  like this one.

Release planning

• I've reviewed the changes in this PR and if I consider them worthwhile for being mentioned on release notes then I have updated the relevant CHANGELOG.md within the component folder structure. For example, if I changed horizon, then I updated (services/horizon/CHANGELOG.md. I add a new line item describing the change and reference to this PR. If I don't update a CHANGELOG, I acknowledge this PR's change may not be mentioned in future release notes.
• I've decided if this PR requires a new major/minor version according to
  semver, or if it's mainly a patch change. The PR is targeted at the next
  release branch if it's not a patch change.

What

Changed the captive core runner to modify toml to include CATCHUP_COMPLETE core config flag when an unbounded ledger range(end=0) and a from=2 is requested

Why

Client ingestion that uses the captive core runner with an unbounded range(end=0) and start=2 will get stuck waiting for sequence 2 to be emitted by core during run on the meta pipe which by default core will start emitting from 3, unless CATCHUP_COMPLETE=true is configured.

Closes #5866

[Copilot]

Pull request overview

This PR fixes an issue where captive core fails to replay ledger sequence 2 when configured with an unbounded range (end=0) starting from ledger 2. The fix modifies the captive core runner to set the CATCHUP_COMPLETE=true configuration flag in stellar-core when the starting ledger is less than 3.

Key changes:

• Added CatchupComplete field to the TOML configuration struct to support the CATCHUP_COMPLETE stellar-core config option
• Modified the run-from logic to detect when ledger sequence < 3 is requested and set CATCHUP_COMPLETE=true instead of running catchup
• Implemented TOML cloning in newWorkingDir to avoid side effects when modifying configuration for specific runs
• Added comprehensive error handling for the new TOML cloning operations

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
ingest/ledgerbackend/toml.go	Added CatchupComplete boolean field to support the stellar-core CATCHUP_COMPLETE configuration option
ingest/ledgerbackend/stellar_core_runner.go	Updated runFrom and catchup methods to handle errors from stream creation functions
ingest/ledgerbackend/run_from.go	Modified logic to skip catchup and enable CATCHUP_COMPLETE when starting from ledger sequence < 3; updated newRunFromStream to return errors
ingest/ledgerbackend/dir.go	Implemented TOML config cloning in newWorkingDir to prevent side effects; added enableCoreCatchupComplete method to set the flag
ingest/ledgerbackend/catchup.go	Updated newCatchupStream to handle and propagate errors from newWorkingDir
ingest/ledgerbackend/stellar_core_runner_test.go	Added TestRunFromRequestedSequence2 to verify correct behavior when starting from ledger 2
CHANGELOG.md	Documented the bug fix for ledger sequence 2 replay issue

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sreuland]

Hello @tamirms , I incorporated feedback, s/b ready for re-review, thanks!