Releases: sigp/lighthouse
Blue Vacuum Cleaner
DO NOT RUN THIS PRE-RELEASE ON MAINNET
This release contains a bug and we strongly recommend upgrading to v7.0.0-beta.5.
Summary
This new beta release brings support for the Hoodi testnet, and cleaned up versions of the fixes made to keep Holesky running.
It is recommended for all testnets: Sepolia, Holesky, Hoodi and Chiado (Gnosis testnet).
This release will form the basis of the upcoming stable v7.0.0 release for Pectra on mainnet, so your feedback is greatly appreciated.
Breaking Changes
There are no breaking changes introduced by this release.
Known Issues
This release contains a bug and we recommend upgrading to v7.0.0-beta.5.
Update Priority
This table provides priorities for which classes of users should update particular components.
| User Class | Beacon Node | Validator Client |
|---|---|---|
| Testnet Users | Low | Low |
| Mainnet Users | DO NOT UPGRADE | DO NOT UPGRADE |
See Update Priorities for more information about this table.
All Changes
- Release v7.0.0-beta.4 (#7162)
- Update ring to 0.17.14 to fix build compat (#7164)
- Reject attestations to blocks prior to the split (#7084)
- Manual compaction endpoint backport (#7104)
- Pseudo finalization endpoint (#7103)
- Support Hoodi testnet (#7145)
- State cache tweaks (#7095)
- Add block ban flag
--invalid-block-roots(#7042) - Ensure finalized block is the correct fork variant when constructing light client updates (#7085)
- feat: implement new beacon APIs(accessors for pending_deposits/pending_partial_withdrawals) (#7006)
- Address cargo audit failure
RUSTSEC-2024-0437(#7114) - Set
epochs-per-blob-prunedefault to 256 (#7113) - Change state cache size default to 32 (#7101)
- Address cargo audit failure
RUSTSEC-2025-0009(#7086) - Optimise status processing (#7082)
- Temporarily ignore cargo audit failures (#7092)
- Use
sync_tolerance_epochsflag to control the proposer prep routines (#7044) - Schedule Chiado testnet Electra hard fork (#7074)
- Make ExecutionBlock::total_difficulty Optional (#7050)
- Add
--long-timeouts-multiplierCLI flag (#7047) - Add
--disable-attestingflag to validator client (#7046) - Add test flag to override
SYNC_TOLERANCE_EPOCHSfor range sync testing (#7030) - Fix builder API headers (#7009)
- Rust 1.85 lints (#7019)
- Fix light client merkle proofs (#7007)
Binaries
See pre-built binaries documentation.
The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0
| System | Architecture | Binary | PGP Signature |
|---|---|---|---|
| x86_64 | lighthouse-v7.0.0-beta.4-x86_64-apple-darwin.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v7.0.0-beta.4-x86_64-unknown-linux-gnu.tar.gz | PGP Signature | |
| aarch64 | lighthouse-v7.0.0-beta.4-aarch64-unknown-linux-gnu.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v7.0.0-beta.4-x86_64-windows.tar.gz | PGP Signature | |
| System | Option | - | Resource |
| Docker | v7.0.0-beta.4 | sigp/lighthouse |
Memory Parasites
DO NOT RUN THIS PRE-RELEASE ON MAINNET
Summary
This is another hotfix for Holesky fixing a bug introduced in v7.0.0-beta.2. The bug caused nodes to get stuck after Holesky finalized again. This release is recommended over v7.0.0-beta.2 for all nodes running on Holesky.
Alternatively, users may checkpoint sync using v7.0.0-beta.0 with --state-cache-size 32, although this may become unstable if finality lapses again.
We are working on further testing and stabilising the hotfixes for the final v7.0.0 release and may release a new beta before then. Thank you for your patience.
All Changes
- Release v7.0.0-beta.3
- Fix descent from split check (#7105)
Binaries
See pre-built binaries documentation.
The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0
| System | Architecture | Binary | PGP Signature |
|---|---|---|---|
| x86_64 | lighthouse-v7.0.0-beta.3-x86_64-apple-darwin.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v7.0.0-beta.3-x86_64-unknown-linux-gnu.tar.gz | PGP Signature | |
| aarch64 | lighthouse-v7.0.0-beta.3-aarch64-unknown-linux-gnu.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v7.0.0-beta.3-x86_64-windows.tar.gz | PGP Signature | |
| System | Option | - | Resource |
| Docker | v7.0.0-beta.3 | sigp/lighthouse |
Frankenstein's Monster
v7.0.0-beta.3 is recommended instead
💡 Mainnet users can ignore this pre-release.
Summary
This is a second hot-fix release for the unsuccessful Electra upgrade on Holesky in February 2025. It contains patches to help reduce memory usage during non-finality, and to help nodes sync.
This release is only for Holesky users. Do not use this version on mainnet, Sepolia, or any other network.
Please see our informational issue for up-to-date advice on the Holesky situation: #7040
Forced Pseudo-Finalization
This release includes an experimental feature to forcefully "finalize" a block in Lighthouse's database so that Lighthouse can prune low-quality sidechains, remove finalized states and reclaim disk space. This feature is not safe in general, and should not be used except in emergencies.
Usage of the endpoint requires an epoch, block_root and state_root to pseudo-finalize, which must be from an epoch boundary. Skipped slots are OK, but the state_root must be the state_root of the epoch boundary state.
An example command to force pseudo-finalization on Holesky is:
curl -X POST --data '{"epoch": "117400", "state_root": "0x355fa23c9704fe346362c43a8fe43fba464fe63f20853bd3a87a8f465d52b4f4", "block_root": "0x06d788e593fd2b5b6fb6dcd63dfa4766201f05d948923dff6865d823246dd3c7" }' http://localhost:5052/lighthouse/finalize
If successful you should see a lot of logs of the form Pruning head, and then after some time (possibly multiple hours if your node knows all of the 800+ Holesky heads), it will complete. Completion is indicated via the debug log DEBG Database consolidation complete (look in $datadir/beacon/logs/beacon.log), and an update to the split field in the /lighthouse/database/info API.
Compaction will likely run after forced finalization, which is what reclaims the space permanently. If started you will see the log:
INFO Starting database compaction
When it completes (likely after ~1hr), you will see the log:
INFO Database compaction complete
If the finalization completes without compaction, you can trigger a manual compaction using the HTTP API:
curl -X POST http://localhost:5052/lighthouse/compaction
State Cache Tweaks
We've tweaked how old states interact with the state cache, removing several paths for state cache misses, and bad state cache interactions caused by BlocksByRange requests, sidechains and attestations to ancient blocks.
All of these optimisations are automatic and require no action from users to benefit. If you are feeling adventurous we've also found some benefit from using a new flag --state-cache-headroom 8, which prunes the state cache more aggressively (removing 8 states) when it gets full.
You may notice backtraces present in the State cache missed logs. These are harmless and do not indicate a fatal error. We added them to aid in debugging.
New HTTP endpoint lighthouse/add_peer to add trusted peer
This endpoint allows users to add a trusted peer to the peer database and dials it every heartbeat in case it gracefully disconnects. This is useful if the node struggles to find peers on the canonical chain. This can be used together with the --disable-discovery flag to limit the peers the node dials to speed up syncing to the right chain.
An example command to add a trusted peer:
curl -X POST -H "Content-Type: application/json" --data '{"enr": "enr:-Le4QLoE1wFHSlGcm48a9ZESb_MRLqPPu6G0vHqu4MaUcQNDHS69tsy-zkN0K6pglyzX8m24mkb-LtBcbjAYdP1uxm4BhGV0aDKQabfZdAQBcAAAAQAAAAAAAIJpZIJ2NIJpcIQ5gR6Wg2lwNpAgAUHQBwEQAAAAAAAAADR-iXNlY3AyNTZrMaEDPMSNdcL92uNIyCsS177Z6KTXlbZakQqxv3aQcWawNXeDdWRwgiMohHVkcDaCI4I"}' http://localhost:5052/lighthouse/add_peer
⚠️ Breaking Changes ⚠️
You should only upgrade to v7.0.0-beta.2 from v7.0.0-beta.{0,1} on Holesky. This release contains no breaking changes compared to v7.0.0-beta.1.
Please see the release notes for previous v7 releases for a summary of changes included in those releases:
- https://github.com/sigp/lighthouse/releases/tag/v7.0.0-beta.1
- https://github.com/sigp/lighthouse/releases/tag/v7.0.0-beta.0
Update Priority
This table provides priorities for which classes of users should update particular components.
| User Class | Beacon Node | Validator Client |
|---|---|---|
| Staking Users (testnet) | High | High |
| Non-Staking Users (testnet) | High | --- |
| Staking Users (mainnet) | DON'T UPDATE | DON'T UPDATE |
| Non-Staking Users (mainnet) | DON'T UPDATE | --- |
See Update Priorities for more information about this table.
Testnet users should update both the Lighthouse VC and BN to v7.0.0-beta.2 if using separate binaries. The execution layer client (e.g. Geth, Reth, Besu, Nethermind, Erigon) must also be updated prior to the Electra fork.
All Changes
- Bump version to v7.0.0-beta.2 (#7073)
- Manual compaction endpoint (#7072)
- Add CI fixes to
holesky-rescue(#7071) - Add http endpoint to add trusted peer (#7068)
- Add backtrace logging. (#7063)
- Prevent writing to state cache when migrating the database (#7067)
- Split block root lookups between fork choice and store on BBR response (#7066)
- Revert "Reuse milhouse subtrees to shrink inactivity_scores in memory (#7062)"
- Reuse milhouse subtrees to shrink inactivity_scores in memory (#7062)
- Manual finalization endpoint (#7059)
- Change state cache size default to 32. (#7055)
- Load block roots from fork choice where possible when serving
BlocksByRangerequests (#7058) - Optimise status processing for holesky-rescue (#7054)
Binaries
See pre-built binaries documentation.
The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0
| System | Architecture | Binary | PGP Signature |
|---|---|---|---|
| x86_64 | lighthouse-v7.0.0-beta.2-x86_64-apple-darwin.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v7.0.0-beta.2-x86_64-unknown-linux-gnu.tar.gz | PGP Signature | |
| aarch64 | lighthouse-v7.0.0-beta.2-aarch64-unknown-linux-gnu.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v7.0.0-beta.2-x86_64-windows.tar.gz | PGP Signature | |
| System | Option | - | Resource |
| Docker | v7.0.0-beta.2 | sigp/lighthouse |
Hemorrhage
Summary
This is a hot-fix release for the unsuccessful Electra upgrade on Holesky in February 2025. It contains patches to help users avoid the invalid justified Holesky block and handle the resulting network turmoil (#7041).
This release is only for Holeksy users. Do not use this version on mainnet.
Please see our informational issue for up-to-date advice on the Holesky situation: #7040
Update Priority
This table provides priorities for which classes of users should update particular components.
| User Class | Beacon Node | Validator Client |
|---|---|---|
| Staking Users | High (Holesky only) | High (Holesky only) |
| Non-Staking Users | High (Holesky only) | --- |
Do not use this version on mainnet
See Update Priorities more information about this table.
All Changes
- Bump version
- discard unused code
- take config value
- Bump sync-tolerance-epoch and make it a cli param
- Add vc --disable-attesting flag
- Ban peers with banned payloads
- Add log
- Blacklist invalid block root in block verification and blacklist invalid finalized epochs in sync.
- Blacklist invalid block root in block verification and blacklist invalid finalized epochs in sync.
- MORE
- Allow invalidation of "valid" nodes
- Implement invalidation API
- Remove more liveness risks
- lcli http-sync hacks
- Disable liveness risk
- Fix flag
- Add flag to disable attestation APIs
- Fix builder API headers (#7009)
- Rust 1.85 lints (#7019)
- Fix light client merkle proofs (#7007)
Binaries
See pre-built binaries documentation.
The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0
| System | Architecture | Binary | PGP Signature |
|---|---|---|---|
| x86_64 | lighthouse-v7.0.0-beta.1-x86_64-apple-darwin.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v7.0.0-beta.1-x86_64-unknown-linux-gnu.tar.gz | PGP Signature | |
| aarch64 | lighthouse-v7.0.0-beta.1-aarch64-unknown-linux-gnu.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v7.0.0-beta.1-x86_64-windows.tar.gz | PGP Signature | |
| System | Option | - | Resource |
| Docker | v7.0.0-beta.1 | sigp/lighthouse |
Beta-Seven
💡 Mainnet users can ignore this pre-release.
Summary
This beta release is high-priority for users on Holesky and Sepolia. It is a required upgrade for the upcoming Electra hard forks on these testnets.
- Electra on Holesky: Mon 24 Feb 2025 21:55:12 UTC. Slot 3710976.
- Electra on Sepolia: Wed 5 Mar 2025 07:29:36 UTC. Slot 7118848.
If you are running Lighthouse on a testnet, you must also upgrade your execution layer client to a Prague-Electra (Pectra) compatible release.
The Electra upgrade has not yet been scheduled on mainnet, so no action is required for mainnet users (do not upgrade).
Notable changes in Lighthouse v7.0.0-beta.0 include:
- IPv6 enabled by default when a globally-routable IPv6 address is configured.
- Light client server enabled by default.
- Bugfix for a regression in attestation subscription logic.
- Support for a new database backend,
redb.
⚠️ Breaking Changes ⚠️
You can upgrade to v7.0.0-beta.0 from any v5 or v6 release. If you are upgrading from v5 you should make sure to read the v6 release notes to account for breaking changes between v5 and v6.
Upgrading to Lighthouse v7.0.0-beta.0 should be automatic for all users, as there are no changes to CLI flags that will prevent the node from starting. Holesky and Sepolia users must upgrade before the Electra fork. Failure to upgrade in time will require a re-sync.
Once you upgrade to Lighthouse v7.0.0-beta.0, you can downgrade to v6, but only prior to the Electra fork.
⚠️ Deprecated CLI Flags ⚠️
The following beacon node flags have been deprecated. You should remove them, but the beacon node will still start if they are provided.
--light-client-server
IPv6 by Default
Lighthouse will now automatically listen on IPv6 if it detects a globally-routable address. We expect for the majority of users with IPv4-only setups that this change will have no effect, but that it will benefit users with correctly configured IPv6 stacks.
The default IPv6 listening port has been changed from port 9090 to port 9000 (same as IPv4) to make firewalling easier. The IPv6 port can be adjusted using the flag --port6.
You can opt-out of IPv6 by using the flag --listen-address 0.0.0.0 to only listen on IPv4.
For more information, see the implementation PR:
🦀 Minimum Supported Rust Version 🦀
We have updated the Minimum Supported Rust Version (MSRV) for this release from 1.80.0 to 1.83.0.
This is only relevant to users compiling Lighthouse from source.
You can update your Rust compiler using:
rustup update stable
⚡ Electra ⚡
The Electra hard fork, paired with the Prague hard fork on the execution layer – together known as Pectra – brings several new features to Ethereum.
The headline change is known as Max EB, and raises the maximum effective balance a single validator may wield from 32 ETH to 2048 ETH. Once adopted, this will allow the network to run more efficiently with a lower validator count, while retaining the same level of security. Max EB even removes some centralisation vectors from staking incentives so that solo validators are able to tap into the compounding rewards previously enjoyed exclusively by large operators.
The process of switching a validator's max effective balance is a consolidation, which transfers stake from one validator to another. Consolidations are triggered via a smart contract call, and are fully opt-in and voluntary. If you are a solo operator with a small number of validators, there is no need to consolidate, although you may choose to do so.
We expect the tooling and documentation for consolidations to mature as Electra on mainnet approaches. We have plans of our own to update the Lighthouse UI (Siren) with consolidation support, and will make an announcement when that is ready.
Light Client Server
We have enabled the Light Client Server by default 🎉. Our implementation has reached a stage of maturity and performance where we feel comfortable rolling it out by default. It should result in negligible changes to bandwidth, CPU usage, memory and disk I/O for users.
The light client protocol allows very lightweight clients and devices to interact with Ethereum, and we are excited to see how the ecosystem evolves with more widespread protocol support.
You can opt-out of the light client server using --disable-light-client-server.
Attestation Subscription Fix
A fix was made to the beacon node's attestation subscription logic which could lead to under-subscription and issues publishing aggregate attestations. This has been addressed in the following PR:
New Database Backends
A new database backend is available in the form of redb. This is a pure-Rust database, with nice ACID properties. We hope to modernise and optimise our database usage around redb in the coming months.
For now, redb is performing slightly worse than LevelDB for some operations, and is only recommended for expert users and tinkerers. You can opt-in using --beacon-node-backend redb. This won't make use of any existing LevelDB database, nor will it delete it, so you should delete your LevelDB database manually and then checkpoint sync if you would like to switch.
🐛 Known Issues 🐛
There are no known issues at the time of writing.
Update Priority
This table provides priorities for which classes of users should update particular components.
| User Class | Beacon Node | Validator Client |
|---|---|---|
| Staking Users (testnet) | High | High |
| Non-Staking Users (testnet) | High | --- |
| Staking Users (mainnet) | DON'T UPDATE | DON'T UPDATE |
| Non-Staking Users (mainnet) | DON'T UPDATE | --- |
See Update Priorities for more information about this table.
Testnet users should update both the Lighthouse VC and BN to v7.0.0-beta.0 if using separate binaries. The execution layer client (e.g. Geth, Reth, Besu, Nethermind, Erigon) must also be updated prior to the Electra fork.
All changes
- Release v7.0.0-beta.0 (#6962)
- Fix light client plumbing in beacon processor (#6993)
- Ensure
GET v2/validator/aggregate_attestationis backwards compatible (#6984) - Address cargo audit failure
RUSTSEC-2025-0006(#6972) - IPv6 By Default (#6808)
- Update EF tests to spec v1.5.0-beta.2 (#6958)
- Sync active request byrange ids logs (#6914)
- Enable Light Client server by default (#6950)
- Schedule Sepolia and Holesky Electra forks (#6949)
- Update attestation rewards API for Electra (#6819)
- Fix aggregate attestation v2 response (#6926)
- Remove duplicated fork_epoch and fork_version implementation (#6953)
- Optimise and refine
SingleAttestationconversion (#6934) - Fix fetch blobs in all-null case (#6940)
- Keep execution payload during historical backfill when prune-payloads set to false (#6766)
- Remove un-used batch sync error condition (#6917)
- Remove unused metrics (#6817)
- Reduce ForkName boilerplate in fork-context (#6933)
- Use old geth version due to breaking changes. (#6936)
- Fix attestation queue length metric (#6924)
- Update metrics.rs (#6863)
- Add individual by_range sync requests (#6497)
- Return error if getBlobs not supported (#6911)
- Add test to beacon node fallback feature (#6568)
- Add check to Lockbud CI job (#6898)
- UX Network Fixes (#6796)
- chore: update peerDAS KZG library to 0.5.3 (#6906)
- Migrate validator client to clap derive (#6300)
- Use data column batch verification consistently (#6851)
- Add builder SSZ flow (#6859)
- Subscribe to PeerDAS topics on Fulu fork (#6849)
- Fix subnet unsubscription time (#6890)
- Cargo update for openssl vuln (#6901)
- update libp2p to 0.55 (#6889)
- update MSRV (#6896)
- Compute columns in post-PeerDAS checkpoint sync (#6760)
- Fix mdbook build. (#6891)
POST /eth/v2/beacon/pool/attestationsbugfixes (#6867)- Cargo update without
rust_eth_kzg(#6848) - Implement PeerDAS Fulu fork activation (#6795)
- Make range sync chain Id sequential (#6868)
- Underflow and Typo (#6885)
- Increase jemalloc aarch64 page size limit (#5244) (#6831)
- Some sync/backfill format nits (#6861)
- Fork aware max values in rpc (#6847)
- More gossipsub metrics (#6873)
- Fix Redb implementation and add CI checks (#6856)
- Detect invalid proposer signature on RPC block processing (#6519)
- Add tests for ExecutionRequests decoding errors (#6832)
- Update to EF tests v1.5.0-beta.1 (#6871)
- Modularize beacon node backend (#4718)
- Electra minor refactorings (#6839)
- Update discv5 (#6836)
- Avoid computing columns from EL blobs if block has already been imported (#6816)
- Add MetaData V3 support to
node/identityAPI (#6827) - Refactor mock builder (#6735)
- Add EIP-7636 support (#6793)
- Fix custodial peer assumption on lookup custody requests (#6815)
- Do not send column requests if there is no blob for the block. (#6814)
SingleAttestationimplementation (#6488)- Misc. dependency cleanup (#6810)
- Remove ineffectual block RPC limits post merge (#6798)
- Implement PeerDAS subnet decoupling (aka custody groups) (#6736)
- Fix data columns not persisting for PeerDAS due to a
getBlobsrace condition (#6756) - Use existing peer count metrics loop to check for open_nat toggle (#6800)
- Implement changes for EIP 7691 (#6803)
- Execution requests with ...
Rick In A Vat In The Garage
Summary
This low-priority patch release fixes a few minor issues in the recent major release v6.0.0. If you have not yet upgraded to Lighthouse v6.0.0, we recommend skipping v6.0.0 and going straight to v6.0.1. These release notes provide a full description of the changes in v6.0.0 and v6.0.1, referred to collectively as v6.0.x.
Lighthouse v6.0.x includes new features and optimisations which are backwards-incompatible with Lighthouse v5.x.y.
After many months of testing, v6.0.x stabilises hierarchical state diffs, resulting in much more compact archive nodes! This long-awaited feature was also known as on-disk tree-states, and was available for pre-release testing in Lighthouse v5.1.222-exp.
Other notable changes in v6.0.x include:
- Removal and deprecation of several old CLI flags. Some flags will need to be removed in order for Lighthouse to start, see below for a full list.
- Improved beacon node failover and prioritisation in the validator client.
- Support for
engine_getBlobsV1to speed up import and propagation of blobs. - Optimised peer discovery and long-term subnet subscription logic.
- New commands for
lighthouse validator-manager. - Improved light client support. Enabled via
--light-client-server. - SSZ by default for blocks published by the VC.
Compared to v6.0.0, this release includes several bugfixes:
- Fix an issue with attestation subnet subscriptions not being tracked correctly (#6682).
- Fix the
/lighthouse/natAPI and NAT metrics (#6677). - Prevent sync from getting stuck on certain lookups (#6658).
- Quality-of-life improvements for archive node users (#6669, #6668).
⚠️ Breaking Changes ⚠️
Upgrading to Lighthouse v6.0.x should be automatic for most users, but you must:
- Remove any unsupported CLI flags (see below), and
- Be aware of the one-way database migration and the changes to archive nodes.
Once you upgrade a beacon node to Lighthouse v6.0.x, you cannot downgrade to v5.x.y without re-syncing.
⚠️ Database Migration ⚠️
The beacon node database migration for v6.0.x is applied automatically upon upgrading. No manual action is required to upgrade.
There is no database downgrade available. We did not take this choice lightly, but in order to deliver hierarchical state diffs, a one-way database migration was simplest. If you do find yourself wanting to downgrade, re-syncing using checkpoint sync is highly recommended as it will get the node back online in just a few minutes.
For Archive Nodes
The migration enables hierarchical state diffs which necessitates the deletion of previously stored historic states. If you are running an archive node, then all historic states will be deleted upon upgrading. If you would like to continue running an archive node, you should use the --reconstruct-historic-states flag so that state reconstruction can restart from slot 0.
If you would like to change the density of diffs, you can use the new flag --hierarchy-exponents which should be applied the first time you start after upgrading. We have found that the hierarchy-exponents configuration does not greatly impact query times which tend to be dominated by cache builds and affected more by query ordering. We still recommend avoiding parallel state queries at the same slot, and making use of sequential calls where possible (e.g. in indexing services). We plan to continue optimising parallel queries and cache builds in future releases, without requiring a re-sync.
For more information on configuring the hierarchy exponents see the updated documentation on Database Configuration in the Lighthouse book.
| Hierarchy Exponents | Storage requirement | Sequential slot query | Uncached query |
|---|---|---|---|
| 5,9,11,13,16,18,21 (default) | 418 GiB | 250-700 ms | up to 10 s |
| 5,7,11 (frequent snapshots) | 589 GiB | 250-700 ms | up to 6 s |
| 0,5,7,11 (per-slot diffs) | 1915 GiB+ | 250-700 ms | up to 2 s |
As part of the archive node changes the format of the "anchor" has also changed. For an archive node the anchor will no longer be null and will instead take the value:
"anchor": {
"anchor_slot": "0",
"oldest_block_slot": "0",
"oldest_block_parent": "0x0000000000000000000000000000000000000000000000000000000000000000",
"state_upper_limit": "0",
"state_lower_limit": "0"
}Don't be put off by the state_upper_limit being equal to 0: this indicates that all states with slots >= 0 are available, i.e. full state history.
NOTE: if you are upgrading from v5.1.222-exp you need to re-sync from scratch. The database upgrade will fail if attempted.
⚠️ Removed CLI Flags ⚠️
The following beacon node flags which were previously deprecated have been deleted. You must remove them from your beacon node arguments before updating to v6.0.x:
--self-limiter--http-spec-fork--http-allow-sync-stalled--disable-lock-timeouts--always-prefer-builder-payload--progressive-balances--disable-duplicate-warn-logs-l(env logger)
The following validator client flags have also been deleted and must be removed before starting up:
--latency-measurement-service--disable-run-on-all--produce-block-v3
In many cases the behaviour enabled by these flags has become the default and no replacement flag is necessary. If you would like to fine-tune some aspect of Lighthouse's behaviour the full list of CLI flags is available in the book:
⚠️ Deprecated CLI Flags ⚠️
The following beacon node flags have been deprecated. You should remove them, but the beacon node will still start if they are provided.
--eth1--dummy-eth1
The following global (BN and VC) flags have also been deprecated:
--terminal-total-difficulty-override--terminal-block-hash-override--terminal-block-hash-epoch-override--safe-slots-to-import-optimistically
⚠️ Modified CLI Flags ⚠️
The beacon node flag --purge-db will now only delete the database in interactive mode, and requires manual confirmation. If it is provided in a non-interactive context, e.g. under systemd or docker then it will have no effect. The beacon node will start without anything being deleted.
If you wish to use the old purge-db behaviour it is available via the flag --purge-db-force which never asks for confirmation.
Network Optimisations
This release includes optimisations to Lighthouse's subnet subscription logic, updates to peer discovery, and fine-tuning of IDONTWANT and rate limiting.
Users should see similar performance with reduced bandwidth. Users running a large number of validators (1000+) on a single beacon node may notice a reduction in the number of subscribed subnets, but can opt-in to subscribing to more subnets using --subscribe-all-subnets if desired (e.g. for marginally increasing block rewards from included attestations).
Validator Client Fallback Optimisations
The beacon node fallback feature in the validator client has been refactored for greater responsiveness. Validator clients running with multiple beacon nodes will now switch more aggressively to the "healthiest" looking beacon node, where health status is determined by:
- Sync distance (head distance from the current slot).
- Execution layer (EL) health (whether the EL is online and not erroring).
- Optimistic sync status (whether the EL is syncing).
The impact of this change should be less downtime during upgrades, and better resilience to faulty or broken beacon nodes.
Users running majority clients should be aware that in the case of a faulty majority client, the validator client may prefer the faulty chain due to it appearing healthier. The best defense against this problem is to run some (or all) validator clients without any connection to a beacon node running a majority CL client or majority EL client.
New Validator Manager Commands
The Lighthouse validator manager is the recommended way to manage validators from the CLI, without having to shut down the validator client.
In this release it has gained several new capabilities:
lighthouse vm import: now supports standard keystores generated by other tools likestaking-deposit-cli.lighthouse vm list: a new read-only command to list the validator keys that a VC has imported.lighthouse vm delete: a new command to remove a key from a validator client, e.g. after exiting.
For details on these commands and available flags, see the docs: https://lighthouse-book.sigmaprime.io/validator-manager.html
Future releases will continue to expand the number of available commands, with the goal of eventually deprecating the previous lighthouse account-manager CLI.
Fetch Blobs Optimisation
This release supports the new engine_getBlobsV1 API for accelerating the import of blocks with blobs. If the API is supported by your execution node, Lighthouse will use it to load blobs from the mempool without waiting for them to arrive on gossip. Our testing indicates that this will help Ethereum scale to a higher blob count, but we need more data from real network...
Tiny Rick
Summary
This low-priority major release introduces new features and optimisations which are backwards-incompatible with Lighthouse v5.x.y. If you are running Nethermind we recommend waiting for the release of Nethermind v1.30.0 before upgrading due to an incompatibility (see Known Issues).
After many months of testing, this releases stabilises hierarchical state diffs, resulting in much more compact archive nodes! This long-awaited feature was also known as on-disk tree-states, and was available for pre-release testing in Lighthouse v5.1.222-exp.
Other notable changes include:
- Removal and deprecation of several old CLI flags. Some flags will need to be removed in order for Lighthouse to start, see below for a full list.
- Improved beacon node failover and prioritisation in the validator client.
- Support for
engine_getBlobsV1to speed up import and propagation of blobs. - Optimised peer discovery and long-term subnet subscription logic.
- New commands for
lighthouse validator-manager. - Improved light client support. Enabled via
--light-client-server. - SSZ by default for blocks published by the VC.
⚠️ Breaking Changes ⚠️
Upgrading to Lighthouse v6.0.0 should be automatic for most users, but you must:
- Remove any unsupported CLI flags (see below), and
- Be aware of the one-way database migration and the changes to archive nodes.
Once you upgrade a beacon node to Lighthouse v6.0.0, you cannot downgrade to v5.x.y without re-syncing.
⚠️ Database Migration ⚠️
The beacon node database migration for v6.0.0 is applied automatically upon upgrading. No manual action is required to upgrade.
There is no database downgrade available. We did not take this choice lightly, but in order to deliver hierarchical state diffs, a one-way database migration was simplest. If you do find yourself wanting to downgrade, re-syncing using checkpoint sync is highly recommended as it will get the node back online in just a few minutes.
For Archive Nodes
The migration enables hierarchical state diffs which necessitates the deletion of previously stored historic states. If you are running an archive node, then all historic states will be deleted upon upgrading. If you would like to continue running an archive node, you should use the --reconstruct-historic-states flag so that state reconstruction can restart from slot 0.
If you would like to change the density of diffs, you can use the new flag --hierarchy-exponents which should be applied the first time you start after upgrading. We have found that the hierarchy-exponents configuration does not greatly impact query times which tend to be dominated by cache builds and affected more by query ordering. We still recommend avoiding parallel state queries at the same slot, and making use of sequential calls where possible (e.g. in indexing services). We plan to continue optimising parallel queries and cache builds in future releases, without requiring a re-sync.
For more information on configuring the hierarchy exponents see the updated documentation on Database Configuration in the Lighthouse book.
| Hierarchy Exponents | Storage requirement | Sequential slot query | Uncached query |
|---|---|---|---|
| 5,9,11,13,16,18,21 (default) | 418 GiB | 250-700 ms | up to 10 s |
| 5,7,11 (frequent snapshots) | 589 GiB | 250-700 ms | up to 6 s |
| 0,5,7,11 (per-slot diffs) | 1915 GiB+ | 250-700 ms | up to 2 s |
As part of the archive node changes the format of the "anchor" has also changed. For an archive node the anchor will no longer be null and will instead take the value:
"anchor": {
"anchor_slot": "0",
"oldest_block_slot": "0",
"oldest_block_parent": "0x0000000000000000000000000000000000000000000000000000000000000000",
"state_upper_limit": "0",
"state_lower_limit": "0"
}Don't be put off by the state_upper_limit being equal to 0: this indicates that all states with slots >= 0 are available, i.e. full state history.
NOTE: if you are upgrading from v5.1.222-exp you need to re-sync from scratch. The database upgrade will fail if attempted.
⚠️ Removed CLI Flags ⚠️
The following beacon node flags which were previously deprecated have been deleted. You must remove them from your beacon node arguments before updating to v6.0.0:
--self-limiter--http-spec-fork--http-allow-sync-stalled--disable-lock-timeouts--always-prefer-builder-payload--progressive-balances--disable-duplicate-warn-logs-l(env logger)
The following validator client flags have also been deleted and must be removed before starting up:
--latency-measurement-service--disable-run-on-all--produce-block-v3
In many cases the behaviour enabled by these flags has become the default and no replacement flag is necessary. If you would like to fine-tune some aspect of Lighthouse's behaviour the full list of CLI flags is available in the book:
⚠️ Deprecated CLI Flags ⚠️
The following beacon node flags have been deprecated. You should remove them, but the beacon node will still start if they are provided.
--eth1--dummy-eth1
The following global (BN and VC) flags have also been deprecated:
--terminal-total-difficulty-override--terminal-block-hash-override--terminal-block-hash-epoch-override--safe-slots-to-import-optimistically
⚠️ Modified CLI Flags ⚠️
The beacon node flag --purge-db will now only delete the database in interactive mode, and requires manual confirmation. If it is provided in a non-interactive context, e.g. under systemd or docker then it will have no effect. The beacon node will start without anything being deleted.
If you wish to use the old purge-db behaviour it is available via the flag --purge-db-force which never asks for confirmation.
Network Optimisations
This release includes optimisations to Lighthouse's subnet subscription logic, updates to peer discovery, and fine-tuning of IDONTWANT and rate limiting.
Users should see similar performance with reduced bandwidth. Users running a large number of validators (1000+) on a single beacon node may notice a reduction in the number of subscribed subnets, but can opt-in to subscribing to more subnets using --subscribe-all-subnets if desired (e.g. for marginally increasing block rewards from included attestations).
Validator Client Fallback Optimisations
The beacon node fallback feature in the validator client has been refactored for greater responsiveness. Validator clients running with multiple beacon nodes will now switch more aggressively to the "healthiest" looking beacon node, where health status is determined by:
- Sync distance (head distance from the current slot).
- Execution layer (EL) health (whether the EL is online and not erroring).
- Optimistic sync status (whether the EL is syncing).
The impact of this change should be less downtime during upgrades, and better resilience to faulty or broken beacon nodes.
Users running majority clients should be aware that in the case of a faulty majority client, the validator client may prefer the faulty chain due to it appearing healthier. The best defense against this problem is to run some (or all) validator clients without any connection to a beacon node running a majority CL client or majority EL client.
New Validator Manager Commands
The Lighthouse validator manager is the recommended way to manage validators from the CLI, without having to shut down the validator client.
In this release it has gained several new capabilities:
lighthouse vm import: now supports standard keystores generated by other tools likestaking-deposit-cli.lighthouse vm list: a new read-only command to list the validator keys that a VC has imported.lighthouse vm delete: a new command to remove a key from a validator client, e.g. after exiting.
For details on these commands and available flags, see the docs: https://lighthouse-book.sigmaprime.io/validator-manager.html
Future releases will continue to expand the number of available commands, with the goal of eventually deprecating the previous lighthouse account-manager CLI.
Fetch Blobs Optimisation
This release supports the new engine_getBlobsV1 API for accelerating the import of blocks with blobs. If the API is supported by your execution node, Lighthouse will use it to load blobs from the mempool without waiting for them to arrive on gossip. Our testing indicates that this will help Ethereum scale to a higher blob count, but we need more data from real networks before committing to a blob count increase.
There are several new Prometheus metrics to track the hit rate:
beacon_blobs_from_el_received_totalbeacon_blobs_from_el_expected_totalbeacon_blobs_from_el_hit_total
Logs at debug level also show the operation of this new feature (grep for fetch_blobs).
At the time of writing the follow execution clients support the API:
- Reth v1.0.7 or newer.
- Besu v24.9.1 or newer.
- Geth v1.14.12 or newer.
Unsupported:
- Nethermind v1.29.x (buggy, see below).
- Erigon.
🐛 Known Issues 🐛
Nethermind v1.29.x engine_getBlobsV1 bug
Nethermind versions v1.29.0 and v1.29.1 include ...
Risotto Groupon
Summary
This medium-priority release contains performance optimisations, bugfixes and new features. There are several breaking changes detailed below.
The main optimisations are:
- Faster beacon node start-up due to an optimisation to the database schema (#5897). The schema migration is a backwards-incompatible change which is described in more detail below.
- More reliable block proposals due to the use of the new
v3block production API (#5292). - Improvements to the validator client's broadcast mechanism (#5976, #6223).
- Support for gossipsub
IDONTWANT(#5422). This allows nodes to reduce their bandwidth usage by avoiding the download of duplicate messages.
The main bugfixes are:
- Fix parsing of the VC
proposer-nodesCLI flag (#6125). - Prevent connections from peers with a banned IP history (#6008).
- Prevent UPnP queries when UPnP is disabled (#6170).
- Corrections to
lighthouse accountCLI (#6153, #6091). - Update Sepolia bootnodes, fixing issues with 0 peers on Sepolia (#6037).
New features include:
- A new Rust slasher database backend,
redb(#4529). We still recommend running LMDB on mainnet, but encourage users to tryredbon testnets using the flag--slasher-backend redb. - The default number of log files retained (
logfile-max-number) has been increased from 5 to 10 to help with debugging issues (#6092). - Lots of code for the upcoming Electra and PeerDAS network upgrades (#5712, #5741, #5761, #5743).
Breaking Changes
🗑️ Removed CLI flags
The following CLI flags have been removed completely and will cause the BN to fail to start up if provided:
--builder-profit-threshold: This flag was already deprecated and has been incompatible since v5.2.0 (#6131).
You should remove any instances of these flags from your beacon node startup command.
👎 Deprecated CLI flags
Beacon node:
--disable-lock-timeouts: Lock timeouts are now disabled by default so this flag does nothing. It will be removed in a future release (#6048).--self-limiter: The self limiter is now enabled by default. This flag will be removed in a future release. The purpose of this is to prevent Lighthouse from sending excessive requests to peers in case of bugs or edge case scenarios. The rate limit quotas can be configured via the existing--self-limiter-protocolsflag (#6093).
Validator client:
--produce-block-v3: The produce block v3 API is now enabled by default and cannot be disabled, so this flag does nothing. It will be removed in a future release (#5292).
You should remove any instances of these flags from your BN/VC commands, although failing to do so will only result in a warning, not a failure to start.
🦀 Minimum Supported Rust Version 1.78.0 🦀
The minimum supported Rust version (MSRV) was updated to 1.78.0. Users who compile from source (i.e., not Docker or pre-built binary users) will receive the following error if they are using an earlier version of Rust:
lighthouse v5.3.0 (/home/sigp/lighthouse/lighthouse) cannot be built because it requires rustc 1.78 or newer
Users can typically obtain the latest version of Rust by running rustup update.
💾 Database Schema Upgrade
The beacon node database schema has been updated from v19 to v21. An automatic database schema upgrade will run when updating from a previous version. No manual intervention is required for upgrading.
The schema upgrade was required to prepare for the upcoming Electra fork, and to optimise how validator public keys are stored on disk. Public keys are now stored uncompressed, which uses twice as much space, but allows them to be loaded around 10x faster. This results in shorter start-up times after the initial start-up on v5.3.0.
If you decide to downgrade from v5.3.0, then a database schema downgrade is available. It must be run using the lighthouse db migrate command. For instructions on how to downgrade, see the Database Migrations section of the Lighthouse Book.
More details on the pubkey optimisation are found in the PR: #5897.
🐋 No more -portable tarballs or -modern images
There are no more -portable tarballs attached to this release, and the -modern Docker images on Docker Hub will no longer be updated. Both have been deprecated since v5.2.0 when portable builds became the default everywhere.
For portable binary users:
- Download the latest
.tar.gzfor your platform. It will continue to work exactly as before.
For modern Docker image users:
- Use the
:latesttag, or the:v5.3.0tag. It will be just as fast as the previously supplied-modernimage.
📝 HTTP API /eth/v1/beacon/deposit_snapshot to use JSON response by default
The /eth/v1/beacon/deposit_snapshot now returns JSON response by default (instead of SSZ) if a header is not specified, for consistency with other beacon APIs.
See: #5813.
🐧 Jemalloc enabled by default on non-Windows targets
jemalloc is now necessary on Linux with the introduction of tree-states for performance reasons. The jemalloc feature is now enabled by default on non-Windows targets.
See: #5995.
🔓 Removal of signed validator client responses
Signing of validator client HTTP responses has been removed. This feature had limited utility and adoption, and removing it has simplified VC API token management. Unless you were relying on the Signature header in VC responses, no action is required.
See: #5529.
💵 Fee recipient required with --always-prepare-payload
The --suggested-fee-recipient flag is now required when --always-prepare-payload is set. This feature is likely only used by block builders and relays.
See: #6079.
Update Priority
This table provides priorities for which classes of users should update particular components.
| User Class | Beacon Node | Validator Client |
|---|---|---|
| Staking Users | Medium | Medium |
| Non-Staking Users | Medium | --- |
See Update Priorities for more information about this table.
Lighthouse BNs and VCs from v5.0.0, v5.1.x, v5.2.x and v5.3.x are compatible. However, we recommend that users update both the VC and BN to v5.3.0 if upgrading.
All Changes
- Release v5.3.0 (#6194)
- Broadcast VC requests in parallel and fix subscription error (#6223)
- Don't expect DAS config in HTTP spec response (#6221)
- Downgrade re-org log to INFO (#6220)
- patch quick-protobuf (#6217)
- Work around UB in LMDB bindings (#6211)
- Avoid acquiring another read lock while holding one to avoid potential deadlock (#6200)
- Swap finalized chains based on processed batches (#6203)
- Use upgradable read lock for pubkey cache (#6190)
- default vc to block v3 endpoint and deprecate block-v3 flag (#5292)
- Remove timeout locks (#6048)
- limit dial concurrency on Swarm (#6184)
- Fix
lighthouse account validatorsubcommands help text (#6091) - Iterate expired components from the da_checker (#5895)
- Rust 1.80.0 lints (#6183)
- Fix unexpected
Marking peer disconnected in DHT(#6140) - Add
DataColumnSidecargossip topic and message handling (#6147) - Use
theninstead ofthen_somewhen checkingupnp_enabledto avoid useless UPnP query (#6170) - Remove backwards compatibility for
el_offlineandis_optimstic(#6168) - Show correct total validator count during validator recovery (#6153)
- Update msrv to 1.78.0 (#6159)
- Revert --image-download always flag to avoid CI getting rate limited (#6163)
- Siren Docs Upgrade (#5979)
- Remove use of ethers core (#5301)
- level down libp2p quic accept and close errors (#6148)
- Make proposer duties work pre-genesis (#4789)
- Update
is_availablecheck to support PeerDAS. (#6076) - Add PeerDAS presets and configs. (#6127)
- Upgrade
opensslto address cargo audit issue (#6143) - Attempt to fix flaky basic sim test (#6134)
- Delete
--builder-profit-thresholdflag (#6131) - Return syncing on HTTP when sync is stalled (#6129)
- Add an option to keep existing enclave when starting local testnet (#6065)
- Add requires
suggested-fee-recipientflag whenalways-prepare-payloadis set (#6079) - Disable slasher default features (#6115)
- Fix
proposer-nodescli flag name (#6125) - chore: update the trusted setup to match the consensus-specs version (#6117)
- Track store metrics by DB column (#6041)
- Ef tests electra (#5758)
- Remove VC response signing and fix HTTP error handling (#5529)
- Remove trace logging (#6103)
- Measure consensus verification time (#6089)
- Add
block_gossipBeacon API events (#5864) - Bump default
logfile-max-numberto 10 (#6092) - Beacon api + validator electra (#5744)
- Enable the o...
Kyle
Summary
This medium-priority release contains bugfixes for issues identified in v5.2.0. Users who have experienced sync issues with v5.2.0 are recommended to upgrade.
The list of bugfixes includes:
- Prevention of forwards sync stalls after restarting, through more robust handling of peer disconnects and errors (#5969, #5945, #5993).
- Prevention of backfill sync stalls (#5969).
- Removal of unnecessary warnings for stale blobs propagated on gossip by other clients (#5935). The log message
WARN Could not verify blob sidecar for gossip. Ignoring the blob sidecarshould no longer be seen. - Correct sizing of the slasher database for the growing validator sets on mainnet and Holesky (#5934).
Breaking Changes
There are no breaking changes between v5.2.0 and v5.2.1. Please see the release notes for v5.2.0 if you are upgrading from a prior 5.x.x release.
Known Issues
Teku 24.10 BN Incompatibility
Beacon nodes running Teku 24.10.x and newer are not compatible out of the box with the validator client from this release. If you want to run Lighthouse VC v5.2.x with Teku BN 24.10.x you must use the --produce-block-v3 flag with Lighthouse. Upgrading to Lighthouse v5.3.0 is preferable as it fixes the incompatibility without requiring any flags.
Nethermind v1.26.0 and Reth v1.0.0 engine_getClientVersionV1 bug
Nethermind v1.26.0 and Reth v1.0.0 include an implementation of engine_getClientVersionV1 which is not compliant with the API specification and does not work with Lighthouse. We recommend upgrading Nethermind to v1.27.0. At time of writing there is no compatible Reth release available.
The warning generated by a non-compliant implementation is:
WARN Execution engine call failed error: InvalidClientVersion("Input must be exactly 8 characters long (excluding any '0x' prefix)"), service: exec
This warning can be safely ignored, although it may cause a slight degradation to node performance as Lighthouse flip-flops the EL's state between healthy and unhealthy.
See NethermindEth/nethermind#7013 and paradigmxyz/reth#9137 for details.
Source builds without jemalloc will OOM on Linux
We've become aware of OOM issues for Lighthouse builds from source that disable the jemalloc feature on Linux. This is due to GNU malloc not handling the allocation of many tree nodes as well as other allocators (jemalloc, the Windows allocator). If you are building from source for Linux ensure that you include jemalloc in FEATURES (enabled by default). If you build the Dockerfile use --build-arg FEATURES=jemalloc (not enabled by default).
This will be addressed in v5.3.0 by an already-merged PR: #5995
Update Priority
This table provides priorities for which classes of users should update particular components.
| User Class | Beacon Node | Validator Client |
|---|---|---|
| Staking Users | Medium | Low |
| Non-Staking Users | Medium | --- |
See Update Priorities for more information about this table.
Lighthouse BNs and VCs from v5.0.0, v5.1.x and v5.2.x are compatible. However, we recommend that users update both the VC and BN to v5.2.1 if upgrading.
All changes
- Release v5.2.1 (testing branch) (#5989)
- Attempt to continue lookups after adding peers (#5993)
- Remove all batches related to a peer on disconnect (#5969)
- Update slasher DB size and Lighthouse book (#5934)
- Ensure all handler events are covered (#5945)
- report failed requests from
RPCafter being clear from rate limiter but not yet sent. (#5942) - Lower tolerance of stale blobs on gossip (#5935)
- chore: simplify method to generate a random valid blob (#5946)
- chore: change
impl Into<T> for Utoimpl From<U> for T(#5948) - Update CI nethermind and LLVM versions (#5933)
- Fix cargo audit: update curve25519-dalek (#5959)
- Update local testnet documentation (#5896)
- Fix SelfRateLimiter breaks the sequence of delayed_requests. (#5903)
- Verify inclusion proof should not be fallible (#5787)
- Remove
extern crate(#5922) - Add
FUNDING.jsonfor DRIPS (#5932) - Fix
CommandLineTestport conflicts on CI (#5908) - Remove some easy Electra TODOs (#5928)
- Rust 1.79 lints (#5927)
- Fix panic on startup in debug build (#5917)
- Fix skip-ci on
stable(#5909) - Beta compiler fixes (#5916)
See full changelog here.
Binaries
See pre-built binaries documentation.
The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0
| System | Architecture | Binary | PGP Signature |
|---|---|---|---|
| x86_64 | lighthouse-v5.2.1-x86_64-apple-darwin.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v5.2.1-x86_64-apple-darwin-portable.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v5.2.1-x86_64-unknown-linux-gnu.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v5.2.1-x86_64-unknown-linux-gnu-portable.tar.gz | PGP Signature | |
| aarch64 | lighthouse-v5.2.1-aarch64-unknown-linux-gnu.tar.gz | PGP Signature | |
| aarch64 | lighthouse-v5.2.1-aarch64-unknown-linux-gnu-portable.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v5.2.1-x86_64-windows.tar.gz | PGP Signature | |
| x86_64 | lighthouse-v5.2.1-x86_64-windows-portable.tar.gz | PGP Signature | |
| System | Option | - | Resource |
| Docker | v5.2.1 | sigp/lighthouse |
Tree People
Summary
This medium-priority release contains a major improvement to how states are managed in memory (tree-states) and an optimised epoch and block processing implementation. The beacon node now uses considerably less memory than the previous version thanks to more efficient state caching. It also includes other performance improvements, bug fixes and groundwork preparation for the upcoming Electra fork and PeerDAS upgrade.
Notable changes in this release:
- Epoch and block processing have been optimized, resulting in improved performance and security. The optimized epoch processing implementation is accompanied by a proof of correctness detailed in this post (#5279).
- Introduce in-memory
tree-states, providing more efficient state caching, improved overall performance and reduced memory usage (#5533). - Add server-sent events (SSE) for proposer slashing, attester slashing, and BLS to execution change (#5327, #5857).
- Encode execution engine client version in block graffiti to probe execution layer client diversity, more details below (#5290).
- Add new timing metrics for block availability to better reflect the actual delays caused by blob processing and other factors (#5510).
- Update the proposer re-orgs feature to align with the latest spec (#5417).
- Introduce new containers, presets, and configurations in preparation for the Electra fork (#5122, #5606, #5607, #5630, #5652, #5653).
- Refactor and simplifications in sync and block lookup logic in preparation for PeerDAS (#5561, #5562, #5563, #5564, #5583, #5655).
- Add a new
builder-header-timeoutflag, which allows for configuring timeouts for the get builder header API call (#5857) - Fix attestations not getting added to the aggregation pool. This improves the quality of aggregate attestations Lighthouse produces (#5863).
There are a few breaking changes. Please see the section on Breaking Changes below.
In-memory tree states (#5533)
This release introduces the memory-only portion of tree-states, marking a significant upgrade to state management. The beacon node can now hold ~64 times more states with approximately the same memory cost, significantly reducing memory usage and minimizing spikes. This enhancement also improves overall performance. Consequently, the default value for the --state-cache-size beacon node flag has been increased from 3 to 128. We recommend that users who had manually adjusted the state cache size try the new larger default.
This development brings us closer to the full implementation of tree-states (both in-memory and on-disk) in Lighthouse, an upgrade that has been in progress for over two years. Stay tuned for the remaining tree-states upgrades and their benefits in upcoming releases!
Encode execution engine client version in block graffiti (#5290)
To support better analysis of execution client diversity, Lighthouse now supports the engine_getClientVersionV1 API.
If no user graffiti is provided, and the method is supported by the execution node, then Lighthouse will use graffiti of the form:
GEab48LH5c2c
Here GE is the execution client shortcode corresponding to Geth, ab48 are the first 4 characters of Geth's commit hash, LH indicates Lighthouse, and 5c2c are the first 4 character's of Lighthouse's commit hash.
We expect execution clients to gradually gain support for this new API. Check your execution client's release notes for details. Note that the latest version of Nethermind (v1.26.0) includes a broken version of this API, so we recommend waiting until v1.27.0 is released before upgrading.
portable builds for Docker and binary releases (#5614, #5615)
All Lighthouse binaries and Docker images are now built using portable BLS. For backwards-compatibility *-modern Docker tags still exist as aliases. Similarly, binary tarballs for the -portable suffix are also attached to this release, but are identical to the unsuffixed binary tarballs. We recommend that automated update scripts switch over to using the :latest Docker image or the unsuffixed binary tarballs so we can remove the redundancy and streamline the available images and binaries in a future release.
The switch to portable is motivated by a change in the blst library to use runtime detection of hardware features. This puts portable builds on par for performance with modern builds, and simplifies the release update process for users and devs.
builder-header-timeout flag (#5857)
Lighthouse previously used a static timeout of 1.0s when requesting an execution payload header from an mev-boost relay. The new builder-header-timeout flag for the beacon node allows this value to be adjusted using a value in milliseconds, e.g. --builder-header-timeout 500 will set the timeout to 0.5s. This can be beneficial if communicating with a faraway relay which sometimes times out.
Breaking Changes
Several breaking changes have been made to CLI flags as part of a cleanup and modernisation effort. Please make sure to remove or update these flags as part of upgrading. The changes are described in detail below but the affected flags are:
- VC:
latency-measurement-service - BN:
progressive-balances,http-spec-fork,http-allow-sync-stalled,self-limiter,inbound-rate-limiter.
We have avoided making breaking changes to the database schema in this release so that users can upgrade and downgrade more simply. The next release will include a number of on-disk changes.
🌲 If you were already running the tree-states beta (v5.1.222-exp), then the database schema for v5.2.0 is not compatible with your existing database. You need to either stay on that release, or wait for a new version that incorporates the on-disk changes (maybe v5.3.0). Unfortunately a re-sync will be required eventually, as the stable form of the tree-states DB will be different from the beta.
🦀 Minimum Supported Rust Version 1.77 🦀
The minimum supported Rust version (MSRV) has been updated to 1.77.0. Users who compile from source (i.e., not Docker or pre-built binary users) will receive the following error if they are using an earlier version of Rust:
lighthouse v5.2.0 (/home/sigp/lighthouse/lighthouse) cannot be built because it requires rustc 1.77 or newer
Users can typically obtain the latest version of Rust by running rustup update.
Deprecated latency-measurement-service flag
The validator client --latency-measurement-service flag is now deprecated in favour of --disable-latency-measurement-service. By default the latency measurement service is enabled, and can be disabled by providing the new --disable-latency-measurement-service flag. Using the prior --latency-measurement-service flag will have no effect and will become a hard error in a future release. This change was made as part of an upgrade to a new version of our CLI parsing library, Clap, in #5273.
Deprecated http-spec-fork and http-allow-sync-stalled flags (#5500)
Lighthouse now ignores the http-spec-fork value and keeps the default spec fork, as this option was rarely used.
The http-allow-sync-stalled flag has been deprecated and is now the default behaviour as Lighthouse now prioritises liveness for the HTTP API. These flags will be removed in upcoming releases.
See #5430 and #5500 for more details.
Deprecated --progressive-balances flag (#5279)
This optimisation (--progressive-balances=fast) is now the default and cannot be disabled. See #5279 for more details. A warning will be logged if the flag is provided, and in a future version we will remove the flag entirely.
Removed block processing metrics (#5510)
The block import metrics have been reworked to more accurately measure import delays and provide more insights into block availability and readiness for attestation.
The following block processing metrics have been removed:
beacon_block_observed_slot_start_delay_timebeacon_block_imported_observed_delay_timebeacon_block_head_imported_delay_timebeacon_block_head_slot_start_delay_timebeacon_block_head_slot_start_delay_exceeded_totalbeacon_block_gossip_propagation_verification_delay_timebeacon_block_gossip_slot_start_delay_timebeacon_block_last_delaybeacon_block_gossip_arrived_late_totalbeacon_blob_gossip_propagation_verification_delay_timebeacon_blob_gossip_slot_start_delay_timebeacon_blob_last_delay
For more details and the updated metrics, see #5510.
MDBX excluded from the Docker build (#5614)
The slasher-mdbx feature is no longer included as part of our standard Docker images.
The default slasher backend was changed from MDBX to LMDB in Lighthouse v4.3.0 for more stability and has been the recommended option. Advanced users can still use the slasher-mdbx feature with a custom Docker build.
Modified self-limiter flag
_This flag is mostly useful for developers and probably shouldn't be part of y...
