Switching kernels

[ahulist]

Is it possible to switch conda kernels in the same/similar fashion as in Jupyter? I'm looking for separation between project requirements and runtime - I'd like to have marimo installed in one env and use modules from another. Is that possible?

[akshayka]

That's not currently possible. It's kind of tricky because marimo is itself a library -- so it needs to be installed in each environment you use it in. We've tried to keep the dependencies quite slim, so hopefully it's not too much of an issue.

[dmadisetti]

Not directly, but you can use the new /// script support for python requirements, and couple them with other solutions like nix for runtime support. See https://docs.marimo.io/guides/editor_features/package_management.html#running-marimo-in-a-sandbox-environment-uv-only

[liquidcarbon]

@akshayka do you have an easy way to inject code prior to starting the starlette server?

maybe another click option to marimo edit that would add a parameter to mo._server.start : https://github.com/marimo-team/marimo/blob/main/marimo/_server/start.py#L67

you could do something like

marimo run notebook.py --with myenv/.venv which would sys.path.append the packages of that venv

[liquidcarbon]

In Jupyter, when you create a kernel, you are writing a JSON file that points to a specific python executable. The server environment kind of doesn't matter, as long as the target env has ipykernel installed. Most people just install another notebook, which is unnecessary.

python -m ipykernel install --name mykernel means "create a pointer to whatever python stands for", as determined by order in PATH, env vars in subshells etc.; that pointer lives in kernel.json, which is in subfolders known to base jupyter. So once you head over to your localhost:8888, you can open any notebook with any kernel.

How can we match this Jupyter UX in marimo? Some thoughts:

1. Splitting marimo into server and kernels sounds painful.

2. Installing it every time into every venv is ok but not great (about 90 MB and 3600 files). But to know which marimo .py file goes with which kernel, you'd have to 1) start a new app with this env's python; and 2) have a way to point to it, probably through a special directive in the python file

3. messing with sys.path may have some funky edge cases, but it's pretty much exactly how venvs already operate! The python in .venv/bin/python is a symlink to outer python, and .venv/lib/python3.12/site-packages/ is last on sys.path, after all the folders inherited from outer python. I'm surprised it's last and not first; I thought you'd want to prepend rather than append, so that potentially overlapping packages are first read from the folder that you most care about.

4. I imagine if you made UI for package manager, it would be straightforward to make an interactive sys.path editor, where you can add and remove paths?

5. why not do that transparently in a notebook? You could have the sys.path editor write a cell, but I think it's not in the spirit of reducing boilerplate. Returning to necessity of a pointer inside the file - maybe simply as an undecorated (no @app.cell) function, or even right above app.run(); the latter is the easiest.

Regarding different python versions. My approach to organizing python work is that you have a folder defined by one and only one python executable; inside, you have project folders with venvs (symlinks to outer python) and project files and VCS. Like so:

├── py312
│   ├── .path/to/bin/python
│   ├── notebooks
│   ├── env1
│   │   ├── .venv
│   │   └── pyproject.toml
│   ├── env2
│   │   ├── .venv
│   │   └── pyproject.toml
├── py311torch
│   ├── .path/to/bin/python
│   ├── notebooks
│   ├── env3
│   │   ├── .venv
│   │   └── pyproject.toml
│   ├── env4
│   │   ├── .venv
│   │   └── pyproject.toml

In puppy, the outer layer is managed by pixi, the inner layers by uv.

So the place to install marimo is into py312 and py311torch, with the idea that you would not mix notebooks from py312 with notebooks from py311torch. We could allow that with a big warning, or disallow completely.

TLDR: py312/bin/marimo py312/notebooks/example.py --with env1 will inject sys.path.append(Path(sys.prefix).parent / "env1" / ".venv/lib/python3.12/site-packages") into either the notebook file itself, or its temp copy, or somewhere inside app.run()

[dmadisetti]

the server couples pretty tightly, I don't think it'd be possible to easily cross python versions between server and notebook

I think /// script gets us part way there, the only thing that's missing is sandbox from the general marimo edit home UI, without the need for an extra flag

[liquidcarbon]

I wouldn't settle for that, I prefer a version of kernels; PEP 723 scripts won't help you, say, build a package.

[dmadisetti]

You can use build tools on top of marimo- for instance, I use bazel to compile my code with bindings, and expose the result to marimo. Not opposed, just wondering the scope

For a more general solution, I've proposed allowing top level imports:

# /// script
# ...

import marimo

import top_level_dep # e.g. numpy
import marimo_plugin # Point to hook in and hack on sys.path or trigger compilation or a per session basis

@app.cell
def _(...):
    ...

...

Which gives some flexibility to the user to scope out 3rd party solutions. However, I think in general marimo wants to be "batteries included", and reduce dependency on external tweaks to the notebook experience. I do think there's a balance though, because I'm not certain marimo should be a build tool either

[liquidcarbon]

"build a package" in a sense of develop, not build for distribution
what I mean by that is I like developing in an interactive notebook environment, with hot reload from module files that I edit in an IDE

The marimo_plugin option you suggest is a great option to develop a prototype; hope to share in a little bit.

[liquidcarbon]

Well there was a possibility of not needing any changes at all to marimo code, but maybe not.

So if you add one line at the top
import sys; sys.path.append("bla")
to a marimo file, then
.\.venv\Scripts\marimo.exe edit .\.venv\Lib\site-packages\marimo\_tutorials\layout.py
and then look at sys.path in marimo, you get that line added three times; how does that happen?

But then the edits disappear when notebook is auto-saved.
If you add it at the end before app.run(), sys.path doesn't seem to change, and the sys.path edits disappears on saving as well.

Can sys.path-containig strings maybe get excluded from whatever stripping that happens on save?

[dmadisetti]

Save is file agnostic to what ever it is replacing, it's a dump of what's in memory. Modifying a marimo file out of a @app.cell tag is only useful as a script- since there's no easy way to easily keep track of global modifications.

The 3 additions is curious- sort of a bug - but not really consequential since it doesn't effect script mode. But you can see that marimo is actually importing the file x3 times:

import traceback
import marimo

__generated_with = "0.8.22"
app = marimo.App(width="medium")

@app.cell
def __():
  ...

traceback.print_stack()

if __name__ == "__main__":
    app.run()

1st time, standard module load: cli:edit line 324, in edit -> /marimo/_ast/codegen.py, line 230, in get_app
2nd time session setup: app_manager. -> file_manager", line 141, in _load_app
3rd time session connecting to session (You'll see if you refresh the page, it will run again)

[liquidcarbon]

I'm personally satisfied with the sys.path route: https://www.linkedin.com/feed/update/urn:li:activity:7250602323402502144/