[fix / refactor] V2 replication mechanism

[lfbrehm]

Goal

The replication mechanism between two dataproxies is not stable in production. If a replication task fails, it is not retried after restart. To address both issues a refactor of the replication mechanism is needed with a more reliable mechanism (like bao-tree) and more readable and maintainable code are the main goals of this issue.

Workplan

1. Refactor up the replication handler into smaller functions
2. Replace the smaller parts with bao-tree
3. Change the dataproxy API
4. Trigger unfinished replication after a restart / failure
5. Scheduling logic with retries

Definition of Done

• Replication works reliably on instable connections
• Replication is retried after restarts and failures
• Tests are written for local testing

Test Concept

• Unit tests for replication handler components
• Dedicated tests for both single and multipart objects of different sizes
• Integration test for replication between two data proxies
• Tests should include also failing replications and retries

Risks / Blockers

• Bao-tree is not a viable alternative and we need to dive deep into the details of the current implementation
• V3 development is progressing faster than anticipated and we do not need to complete this