[WIP] ch cluster+replica+ha #7084
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kanwarujjaval
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds ClickHouse cluster support with database and table creation scripts. The changes focus on three main areas: configuring cluster-specific settings, cleaning up excessive debug logging in the API, and handling cluster mode for mutation validation.
Key Changes:
- Added comprehensive ClickHouse cluster configuration options including replication, distributed writes, and parallel replicas
- Modified mutation manager to detect cluster mode and validate against
_localtables when appropriate - Removed extensive console.log debug statements from API startup, replacing with proper log calls
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| api/jobs/mutationManagerJob.js | Added ClusterManager integration to handle validation table naming in cluster mode |
| api/configextender.js | Added documentation comment explaining ClickHouse configuration override mechanism |
| api/config.sample.js | Introduced cluster, replication, parallelReplicas, distributed, dictionary, and identity configuration sections |
| api/api.js | Cleaned up debug console.log statements and replaced with proper log calls |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| let validationTable = task.collection; | ||
| if (ClusterManager) { | ||
| try { | ||
| const cm = new ClusterManager(countlyConfig.clickhouse || {}); |
There was a problem hiding this comment.
Creating a new ClusterManager instance for every task in the loop is inefficient. Consider instantiating ClusterManager once before the loop and reusing it, or caching the cluster mode determination result.
feat: add database parameter to mutation manager job query execution
[fix] user profile tests
fix: db validation in mutation manager job
…development start scripts
…nto uj/ch-cluster
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
… support