Support Multi-Raft

[ariesdevil]

Is your feature request related to a problem? Please describe.
We would like to embed OpenRaft into a service that needs to host many independent Raft groups side-by-side (a “multi-raft” deployment). Today each node instance can participate in only one Raft group, so we have to spin up one OpenRaft stack per group, which multiplies the amount of boilerplate (transport layers, storage wiring, metrics) and complicates lifecycle management.

Describe the solution you'd like
Introduce first-class support for managing multiple, isolated Raft groups within the same codebase. Ideally this could live in a separate crate within the repository (e.g. openraft-multi) that reuses the core openraft crate but coordinates shared resources such as networking, async runtimes, and storage pools while keeping state machines and membership fully isolated per group.

Describe alternatives you've considered

• Running one process (or runtime) per Raft group, each embedding its own OpenRaft node. This works but wastes resources and makes orchestration harder.

• Hand-rolling a thin abstraction layer that multiplexes RPC/storage calls across groups. This becomes fragile quickly, especially when dealing with leader election, snapshotting, and metrics in a shared environment.

Additional context
A lightweight multi-raft harness would help build systems like distributed key/value stores or message queues where thousands of shards map to independent Raft groups. The jraft multi-raft design is a good reference I think: it keeps per-group metadata separate while sharing threads and transports. Even a minimal prototype demonstrating how to register multiple Raft instances against a shared runtime would be a big step forward.

[github-actions]

👋 Thanks for opening this issue!

Get help or engage by:

• /help : to print help messages.
• /assignme : to assign this issue to you.

[schreter]

Side note: we do the same - many small Raft consensus groups in one larger system. So far, we simply run separate tasks and separate connections between nodes. Regarding wasting resources, yes, this is a thing, and we want to add pooling of (especially) heartbeats, so instead of many channels between a random pair of nodes, we just have one. However, I'm not sure how this could be abstracted well (probably it can). Just note that multiple connections also have advantages: if you move a lot of data, single connection is typically quickly limited (we have 100G network).

Regarding threads, this is likely a non-issue with async tasks.

Anyway, looking forward to what you'll find out 🙂.

[ariesdevil]

I edited the original issue, added a link on how jraft makes abstraction, for reference.

FYI: jraft is a Java port of braft that was developed by Alibaba

[drmingdrmer]

Thanks for the detailed breakdown! I agree Storage and Network multiplexing are complex, but there are concrete examples (like jraft) to follow, so those seem manageable.

My main concern is thread affinity and task isolation in a multi-raft setup. With potentially hundreds of Raft groups, we'll have:

• Multiple storage I/O tasks per group (log writes, state machine applies)
• Multiple network tasks per group (replication, heartbeats)

Without proper isolation, tasks from different subsystems could starve each other. For example, heavy storage I/O from one group might block network tasks from other groups if they share the same executor.

Potential solution: The async runtime API might need to support task affinity:

runtime.spawn(future, TaskGroup::Storage(group_id))
runtime.spawn(future, TaskGroup::Network(group_id))

Or even simpler categorization:

runtime.spawn_storage(future)  // Pinned to storage thread pool
runtime.spawn_network(future)  // Pinned to network thread pool

This way we can:

• Dedicate thread pools for different task types
• Prevent cross-subsystem starvation
• Tune parallelism per subsystem (e.g., 4 storage threads, 8 network threads)

This seems like a fundamental requirement that differs from single-raft deployments where resource contention is less critical. Curious if others have thoughts on this approach vs just relying on tokio's work-stealing scheduler?

[ariesdevil]

Tokio has a budget mechanism that can prevent other tasks from being in a state of starvation. Every task has a budget and you can also manually tune the budget by using some apis, or control the yield point.