Skip to content

The multicore feature of future::plan has a delay in Radian #522

@crazyxiaoj

Description

@crazyxiaoj

Here's the R code snippet I was working with:

library(biomaRt)
library(Seurat)
library(ggplot2)
library(harmony)
library(magrittr)
library(dplyr)
library(future)

load("17_scRNA_base/scRNA_after_PCA.Rdata")
RhpcBLASctl::blas_set_num_threads(1)
options(future.globals.maxSize = 100 * 1024 ^ 3)
future::plan("multicore", workers = 16)

scRNA <- JackStraw(scRNA, num.replicate = 100, dims = 50)

When executing this code in base R, monitoring with htop -u shows that the multicore functionality from the future package engages smoothly right away.

Image

However, in radian, it initially appears as though only a single core is active. After waiting approximately 3 minutes (tested with an Rdata file containing around 40,000 cells and about 500 MB in size), the multicore feature does start to work as expected.

Image Image

Thank you for your time and for maintaining such a useful tool! I'm happy to provide more details or test any potential fixes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions