Adding support to custom thread pools

[filipe-cantarelli]

Proposal implementation support for custom threadpools by exposting the PoolBuilder.

I did move the build method outside the builder, not sure if that's the best place. IMO, now the builder looks more like a a helper for building instead of controlling the building process.

Feel free to request changes. I'll add documentation and tests once we are ok on the path forward.

[filipe-cantarelli]

@RReverser any feedback on this?

[RReverser]

Ah sorry. I remember I looked at this, but admittedly I don't see what this does / how it ties into the original feature request, which was about building custom rayon threadpools instead of global one.

The changes here mostly expose implementation details and private methods to JS rather than the high-level Rust API for creating a rayon ThreadPool.

[filipe-cantarelli]

Ah sorry. I remember I looked at this, but admittedly I don't see what this does / how it ties into the original feature request, which was about building custom rayon threadpools instead of global one.

The changes here mostly expose implementation details and private methods to JS rather than the high-level Rust API for creating a rayon ThreadPool.

What I did was expose the builder so people could take control of the creation on their own. e.g:

import init, { rayon_test, initThreadPool, initWorkerPool, MyPool } from "/pkg/wasm_parallel.js";
// Initialize the WASM module
await init();
// Global TP
await initThreadPool(5);

// local TP
let wp = await initWorkerPool(1);
let pool1 = new MyPool(wp);
console.log("Running with Global pool:");
rayon_test();
console.log("Done");
console.log("Running with Local threads:");
pool1.rayon_test();
console.log("Done");

Then the pool can be something like:

#[wasm_bindgen]
impl MyPool {
    #[wasm_bindgen(constructor)]
    pub fn new(pb: &wbg_rayon_PoolBuilder) -> Self {
        let tp = ThreadPoolBuilder::new()
            .num_threads(pb.num_threads())
            .spawn_handler(|v| {
                pb.sender()
                    .send(v)
                    .map_err(|_| Error::other("Failed to send thread to pool"))
            })
            .build()
            .unwrap();

        MyPool { pool: tp }
    }

    pub fn rayon_test(&self) {
        self.pool.install(|| {
            (0..10).into_par_iter().for_each(|_| {
                sleep(Duration::from_secs(1));
            });
        });
    }

you rather have the created ThreadPool itself being returned instead?

[RReverser]

you rather have the created ThreadPool itself being returned instead?

Either ThreadPool or maybe ThreadPoolBuilder for more flexibility, yeah. The main point is that, unlike for global init, it doesn't make sense to expose custom builders to JS - Rayon threadpools are only usable from Rust, so I'd rather keep it on this side and not expose any private implementation details to JS as public API.