starknet_patricia_storage: tune rocksDB config

[nimrod-starkware]

No description provided.

[reviewable-StarkWare]

This change is 

[nimrod-starkware]

• starknet_patricia_storage: use index DB for benchmarks #11877
• starknet_patricia_storage: rocksDB stats #11876
• starknet_patricia_storage: save options #11875
• starknet_patricia_storage: tune rocksDB config #11874 👈 (View in Graphite)
• starknet_patricia_storage: dont use wal for rocksdb #11869
• main-v0.14.1-committer

This stack of pull requests is managed by Graphite. Learn more about stacking.

[ArielElp]

@ArielElp reviewed 1 file and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @nimrod-starkware).

---

crates/starknet_patricia_storage/src/rocksdb_storage.rs line 60 at r1 (raw file):

        opts.set_write_buffer_size(WRITE_BUFFER_SIZE);
        opts.increase_parallelism(NUM_THREADS);
        opts.set_max_subcompactions(NUM_THREADS.try_into().unwrap());

(Non blocking) I thought you said this did not help much?

Also, are we sure about NUM_THREADS=MAX_BACKGROUND_JOBS=max_subcompactions? (worth verifying with cursor the numbers for 32 cores).

[dorimedini-starkware]

@dorimedini-starkware reviewed 1 file and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @nimrod-starkware).

---

crates/starknet_patricia_storage/src/rocksdb_storage.rs line 70 at r2 (raw file):

        block.set_partition_filters(true);

        block.set_block_size(DB_BLOCK_SIZE);

what was this and why is it removed?

Code quote:

block.set_block_size(DB_BLOCK_SIZE);

---

crates/starknet_patricia_storage/src/rocksdb_storage.rs line 31 at r2 (raw file):

// memory).
const BLOOM_FILTER_NUM_BITS: f64 = 10.0;
const KEY_PREFIX_BYTES_LENGTH: usize = 32;

what is a "key prefix"?
our DB keys are always 64 bytes

Code quote:

const KEY_PREFIX_BYTES_LENGTH: usize = 32;

[nimrod-starkware]

@nimrod-starkware made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @dorimedini-starkware).

---

crates/starknet_patricia_storage/src/rocksdb_storage.rs line 60 at r1 (raw file):

Previously, ArielElp wrote…

(Non blocking) I thought you said this did not help much?

Also, are we sure about NUM_THREADS=MAX_BACKGROUND_JOBS=max_subcompactions? (worth verifying with cursor the numbers for 32 cores).

I expected it to improve the 1k-constant flavor, but it didn't make it better or worst.
Cursor suggested to use it and set it to the number of cores the machine has.
Maybe the constant NUM_THREADS should be given from the committer config, @dorimedini-starkware wdyt?

[dorimedini-starkware]

@dorimedini-starkware made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @ArielElp and @nimrod-starkware).

---

crates/starknet_patricia_storage/src/rocksdb_storage.rs line 60 at r1 (raw file):

Previously, nimrod-starkware wrote…

I expected it to improve the 1k-constant flavor, but it didn't make it better or worst.
Cursor suggested to use it and set it to the number of cores the machine has.
Maybe the constant NUM_THREADS should be given from the committer config, @dorimedini-starkware wdyt?

should be configurable, yes, as should (many?) other fields.
can one of you take this (in a separate PR)?

[nimrod-starkware]

@nimrod-starkware made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @ArielElp and @dorimedini-starkware).

---

crates/starknet_patricia_storage/src/rocksdb_storage.rs line 60 at r1 (raw file):

Previously, dorimedini-starkware wrote…

should be configurable, yes, as should (many?) other fields.
can one of you take this (in a separate PR)?

I can take it, added a todo.