Using one marimo installation with multiple environments 🐶

[liquidcarbon]

marimo + puppy = 💚

This is a recipe for structuring your marimo projects for multiple environments using a package manager I developed called puppy.

The same approach will work for Jupyter.

Puppy is a CLI wrapper for pixi and uv.

The TLDR of puppy:

1. create a folder, install python + whatever toys you will use across all your projects, e.g. uv, marimo, jupyter, pytest, etc. (also conda and non-python stuff)
2. use pup add commands to build virtual environments defined by pyproject.toml files with uv
3. use pup.fetch() to "activate" the venvs from any interactive shell (python, marimo, ipython, jupyter)

Let's build

Suppose you're trying out some cool new anywidget tools that you plan to use with both marimo and jupyter.

1. Start in an empty folder

mkdir puppy-marimo && cd puppy-marimo

2. Install puppy

curl -fsSL https://raw.githubusercontent.com/liquidcarbon/puppy/main/pup.sh | bash

output:

✨ pixi 0.35.0 found
✔ Created /home/a/puppy-marimo/pixi.toml
✔ Added python=3.12
✔ Added uv>=0
🟣 uv 0.5.4
✔ Added click>=8

3. Install marimo, jupyter, and anything else that you'll use from root environment.

pixi add jupyter marimo quarto

output:

✔ Added jupyter >=1.1.1,<2
✔ Added marimo >=0.9.26,<0.10
✔ Added quarto >=1.5.57,<2

4. Create new virtual environment called newenv and install tldraw:

pup add newenv tldraw

output:

[2024-11-27 15:18:05] 🐶 heard: pup new newenv
[2024-11-27 15:18:05] 🐶 said: pixi run uv init /home/a/puppy-marimo/newenv -p /home/a/puppy-marimo/.pixi/envs/default/bin/python --no-workspace
Initialized project `newenv` at `/home/a/puppy-marimo/newenv`
[2024-11-27 15:18:05] 🐶 said: pixi run uv venv /home/a/puppy-marimo/newenv/.venv -p /home/a/puppy-marimo/.pixi/envs/default/bin/python
Using CPython 3.12.7 interpreter at: .pixi/envs/default/bin/python
Creating virtual environment at: newenv/.venv
Activate with: source newenv/.venv/bin/activate
[2024-11-27 15:18:06] 🐶 heard: pup add newenv tldraw
[2024-11-27 15:18:06] 🐶 said: pixi run uv add tldraw --project /home/a/puppy-marimo/newenv
Resolved 47 packages in 31ms
Installed 41 packages in 45ms
 + anywidget==0.9.13
 + asttokens==2.4.1
...

5. Instead of installing a whole new marimo into each new environment, place a copy of marimo's executable into newenv, patching it with pup.fetch

VENV=newenv
head -5 .pixi/envs/default/bin/marimo > "$VENV/.venv/bin/marimo"
echo -e '\nimport pup\npup.fetch("'$VENV'")\n' >> "$VENV/.venv/bin/marimo"
tail -5 .pixi/envs/default/bin/marimo >> "$VENV/.venv/bin/marimo"
chmod +x "$VENV/.venv/bin/marimo"

Alternatively, just run main marimo pixi run marimo and pup.fetch inside the notebook. This will work the same in marimo and jupyter - no need to mess with kernels. You can fetch more than one venv or build reproducible environments on the fly:
pup.fetch("anotherenv", "awswrangler", "duckdb")

6. Start marimo

newenv/.venv/bin/marimo --edit --no-token

Output:

[2024-11-27 15:24:40] fetched packages from 'newenv': /home/a/puppy-marimo/newenv/.venv/lib/python3.12/site-packages added to `sys.path`
[2024-11-27 15:24:40] 🐶 heard: pup list newenv
{
  "newenv": [
    "tldraw>=3.0.0"
  ]
}
Update available 0.9.26 → 0.9.27
Run pip install --upgrade marimo to upgrade.


        Create or edit notebooks in your browser 📝

        URL: http://localhost:2718

7. Marimo runs and knows how to find tldraw:

Done drawing? try another one

8. Create another environment with another widget:

VENV=newenv2
pup add $VENV drawdata pandas
head -5 .pixi/envs/default/bin/marimo > "$VENV/.venv/bin/marimo"
echo -e '\nimport pup\npup.fetch("'$VENV'")\n' >> "$VENV/.venv/bin/marimo"
tail -5 .pixi/envs/default/bin/marimo >> "$VENV/.venv/bin/marimo"
chmod +x "$VENV/.venv/bin/marimo"

9. Run marimo:

newenv2/.venv/bin/marimo --edit --no-token

Now drawdata venv is active, and, as expected, tldraw is not there.

[liquidcarbon]

Patching Jupyter executable in this manner is not that simple, so it's much easier to use pup.fetch inside the notebook.

Start using pixi run jupyter lab or pixi run jupyter notebook.

[liquidcarbon]

Here's some more hackery - using runpy, we inject custom code prior to launching marimo - in this case, we pup.fetch another environment. This works in Dockerfiles (example on HuggingFace)

pixi run python -c \
    'import pup, runpy; pup.fetch("sql_and_plots"); runpy.run_path(".pixi/envs/default/bin/marimo", run_name="__main__")' \
    tutorial sql --host 0.0.0.0 --port 7860 --no-token

[yharby]

I like it very cool