Skip to content

Technologicat/raven

BranchesTags

Repository files navigation

Logo of Raven

Table of Contents

The Raven constellation

Raven is a constellation of apps loosely related to natural language processing, with a focus on scientific use cases.

The vision is to help you absorb information from large volumes of text.

Raven is 100% local, 100% private, 100% open source.

Raven-visualizer: Visualize research literature

Screenshot of Raven-visualizer's main window Screenshot of Raven-visualizer's wordcloud window

  • Documentation: Visualizer user manual
  • Goal: Take 10k+ studies, find the most relevant ones.
    • Status: ✅ Fully operational. Could still use more features; we plan to add some later.
  • Features:
    • GUI app for analyzing BibTeX databases
    • Semantic clustering
    • Automatic per-cluster keyword detection
    • Command-line converters for Web of Science (WOS), arXiv, conference abstract PDFs
    • 100% local, maximum privacy, no cloud services
  • This was the original Raven.

Raven-librarian: Scientific LLM frontend

Screenshot of Raven-librarian

  • Documentation: under development
  • Goal: Efficiently interrogate a stack of 2k scientific papers. Talk with a local LLM for synthesis, clarifications, speculation, ...
    • Status: 🚧 An initial prototype of GUI app raven-librarian is available; as well as a command-line mini-prototype raven-minichat (note that the GUI app has more features).
      • The GUI app is under development.
      • For the GUI app raven-librarian, raven-server must be running.
      • For the command-line raven-minichat, we recommend having raven-server running; this allows the LLM to search the web.
  • Features:
    • 100% local when using a locally hosted LLM
    • Natively nonlinear branching chat history - think Loom (original; obsidian) or SillyTavern-Timelines.
      • Chat messages are stored as nodes in a tree.
      • Branching is cheap. A chat branch is just its HEAD pointer.
      • The chain of parent nodes uniquely determines the linear history for that branch, up to and including the system prompt.
    • RAG (retrieval-augmented generation) with hybrid (semantic + keyword) search.
    • Tool-calling (a.k.a. tool use).
      • Currently, a websearch tool is provided. This will be expanded later.
    • Anime avatar for the LLM, see Raven-avatar below.
      • Speech synthesizer with lipsynced animation.
      • Subtitles with machine translation.
      • Speech recognition. Use your mic to talk to the LLM.
  • Uses oobabooga/text-generation-webui as the LLM backend through its OpenAI-compatible API.

Audio recording is 100% private:

  • Audio is only recorded when the mic button is clicked.
  • The audio recording is only used for locally hosted STT (speech recognition), then discarded.
    • The speech recognizer is hosted by the stt module of your local Raven-server.
  • The audio recording is never saved to disk.
  • The text transcript of the audio is shown, for your own information, in the Librarian client log.
    • This is the same text that goes into Librarian's chat view (and, like any chat message, is saved in the chat datastore).
    • The log is only shown in the terminal window that started raven-librarian, and not saved.
  • If you want to verify these claims, see raven.server.modules.stt, the function api_stt_transcribe in raven.server.app, and stt_* functions in raven.client.api (and their use sites).

Raven-avatar: AI-animated anime avatar

Screenshot of Raven-avatar-settings-editor Screenshot of Raven-avatar-pose-editor

  • Documentation: Avatar user manual
  • Goal: Visually represent your LLM as a custom anime character, for PR stunts and for fun.
    • Status: ✅ Fully operational standalone tech demo, and Python bindings to integrate the avatar to Python-based GUI apps.
    • JS bindings possible, but not implemented yet. See #2.
  • Features:
    • One static input image into realtime video (THA3 engine).
    • Talking custom anime character with 28 emotional expressions.
    • Lipsync to Raven-server's TTS. Record TTS-generated speech with lipsync (audio + image sequence).
    • Realtime Anime4K upscaler.
    • Realtime video postprocessor with visual effects such as bloom, chromatic aberration, or scanlines.
    • Web API to receive avatar video stream and to control the avatar.

Raven-server: Web API server

Screenshot of Raven-server

  • Documentation: Server user manual
  • Goal: Run all GPU processing on the server, anywhere on the local network.
    • Status: ✅ Fully operational.
  • Features:
    • AI components for natural language processing (NLP).
    • Speech synthesizer (TTS), using Kokoro-82M.
    • Speech recognition (STT), using whisper-large-v3-turbo.
    • Server side of Raven-avatar.
  • Partially compatible with SillyTavern. Originally developed as a continuation of SillyTavern-extras.
  • Python bindings (client for web API) provided.
    • JS bindings possible, but not implemented yet. See #2.

Install & run

The Raven constellation consists traditional desktop apps. It needs to be installed.

Currently, this takes the form of installing the app and dependencies into a venv (virtual environment). At least at this stage of development, app packaging into a single executable is not a priority.

Raven is developed and tested on Linux Mint. It should work in any environment that has bash and pdm.

It has been reported to work on Mac OS X, as well as on Windows (with Miniconda to provide Python).

From source

Raven has the following requirements:

  • A Python environment for running the PDM installer. Linux OSs have one built-in; on other OSs it is possible to use tools such as Miniconda to install one.
  • An NVIDIA GPU for running AI models via CUDA. (This is subject to change in the future.)

❗ Help wanted! If you have an AMD GPU and would be willing to collaborate to get Raven working on it, please chime in. Raven does not directly depend on CUDA, but only on PyTorch and on various AI libraries in the Python ecosystem. ❗

Install PDM in your Python environment

Raven uses PDM to manage its dependencies. This allows easy installation of the app and its dependencies into a venv (virtual environment) that is local to this one app, so that installing Raven will not break your other apps that use machine-learning libraries (which tend to be very version-sensitive).

Note that in contrast to many AI/ML apps, which use conda to manage the venv for the app, Raven instead uses PDM. The venv creation and management for the app is automatic, but you need a Python environment to run PDM in. That Python environment is used for running PDM only. Raven itself will run in the venv created automatically by PDM, which may even have a Python version different from that of the environment where PDM runs.

If your Python environment does not have PDM, you will need to install it first:

python -m pip install pdm

Don't worry; it won't break pip, poetry, uv, or other similar tools.

Install Raven via PDM

Then, to install Raven, in a terminal that sees your Python environment, navigate to the Raven folder.

We will next initialize the new venv, installing the required Python version into it. This Python will be available for PDM venvs, and is independent of Python that PDM itself runs on.

Raven is currently developed against the minimum supported Python version, so we recommend to install that version, like this:

pdm python install --min

The venv will be installed in the .venv hidden subfolder of the Raven folder.

Then, install Raven's dependencies as follows. (If you are a seasoned pythonista, note that there is no requirements.txt; the dependency list lives in pyproject.toml.)

Basic install without GPU compute support

pdm install

This may take a while (several minutes).

Now the installation should be complete.

Install with GPU compute support

❗ Currently this requires an NVIDIA GPU and CUDA. ❗

❗ Using CUDA requires the proprietary NVIDIA drivers, also on Linux. ❗

❗ Currently Raven uses CUDA 12.x. Make sure your NVIDIA drivers support this version. ❗

pdm install --prod -G cuda

If you want to add GPU compute support later, you can run this install command on top of an already installed Raven.

Installing dependencies may take a long time (up to 15-30 minutes, depending on your internet connection), because torch and the NVIDIA packages are rather large (my .venv shows 11.1 GB in total).

Now the installation should be complete.

Install on an Intel Mac with MacOSX 10.x

Installing Raven may fail, if Torch cannot be installed.

On MacOSX, installing torch 2.3.0 or later requires an ARM64 processor and MacOSX 11.0 or later.

If you have an Intel Mac (x86_64) with MacOSX 10.x, to work around this, you can use Torch 2.2.x.

To do this, modify Raven's pyproject.toml in a text editor, so that the lines

    "torch>=2.4.0",
    "torchvision>=0.22.0",

become

    "torch>=2.2.0,<2.3.0",
    "torchvision>=0.17.2",

Also, ChromaDB requires onnxruntime, which doesn't seem to be installable on this version of OS X. This means Raven-librarian and Raven-server won't work (as the RAG backend and the server's embeddings module require ChromaDB), but you can still get Raven-visualizer to work, by removing ChromaDB. Run this command in the terminal:

pdm remove chromadb

Then run pdm install again.

❗ In general, if a package fails to install, but is not explicitly listed in the dependencies, you can try to find out which package pulls it in, by issuing the command pdm list --tree. This shows a tree-structured summary of the dependencies. ❗

Install on Windows (if Windows Defender gets angry)

Installing Raven does not need admin rights.

  • Raven can be installed as a regular user.
    • We recommend Miniconda as the Python environment.
  • The only exception, that does need admin rights, is installing espeak-ng, so the TTS (speech synthesizer) can use that as its fallback phonemizer.
    • Raven only ever calls espeak-ng from Raven-server's tts module, and only for those inputs for which the TTS's built-in Misaki phonemizer fails.
    • In practice, that is for out-of-dictionary words in English, as well as for some non-English languages.

Using Raven does not need admin rights.

  • All the apps are regular userspace apps that you can run as a regular user.

If you get a permission error when trying to run pdm, try replacing "pdm" with "python -m pdm".

For example, instead of:

pdm install

run the command:

python -m pdm install

This works because PDM is just a Python module. This will be allowed to run if python is allowed to run.

Similarly, Raven apps are just Python modules, and can be run via Python, as follows. Full list as of Raven v0.2.4:

Command                                Replacement

raven-visualizer                  →    python -m raven.visualizer.app
raven-importer                    →    python -m raven.visualizer.importer
raven-librarian                   →    python -m raven.librarian.app
raven-arxiv2id                    →    python -m raven.tools.arxiv2id
raven-burstbib                    →    python -m raven.tools.burstbib
raven-dehyphenate                 →    python -m raven.tools.dehyphenate
raven-wos2bib                     →    python -m raven.tools.wos2bib
raven-pdf2bib                     →    python -m raven.tools.pdf2bib
raven-server                      →    python -m raven.server.app
raven-avatar-settings-editor      →    python -m raven.avatar.settings_editor.app
raven-avatar-pose-editor          →    python -m raven.avatar.pose_editor.app
raven-check-cuda                  →    python -m raven.tools.check_cuda
raven-check-audio-devices         →    python -m raven.tools.check_audio_devices
raven-minichat                    →    python -m raven.librarian.minichat

Check that CUDA works (optional)

Once you have installed Raven with GPU compute support, you can check if Raven detects your CUDA installation:

raven-check-cuda

This command will print some system info into the terminal, saying whether it found CUDA, and if it did, which device CUDA is running on.

It will also check whether the cupy library loads successfully. This library is needed by the spaCy natural language analyzer (so that too can run on GPU).

Example output:

INFO:raven.tools.check_cuda:Raven-check-cuda version 0.2.3
Checking dependencies...
1. PyTorch availability check [SUCCESS] ✅
2. CUDA device availability check [SUCCESS] ✅ (Using NVIDIA GeForce RTX 3070 Ti Laptop GPU)
3. CuPy & CuPyX (for spaCy NLP) [SUCCESS] ✅

System information:
   Python version: 3.10.12
   OS: Linux 6.8.0-109049-tuxedo
   PyTorch version: 2.7.0+cu126

Activate the Raven venv (to run Raven commands such as raven-visualizer or raven-server)

In a terminal that sees your Python environment, navigate to the Raven folder.

Then, activate Raven's venv with the command:

$(pdm venv activate)

Note the Bash exec syntax $(...); the command pdm venv activate just prints the actual internal activation command.

❗ Windows users note: The command $(pdm venv activate) needs the bash shell, and will not work in most Windows command prompts. ❗

Alternatively, you can run the venv activation script directly. You can find the script in .venv/bin/.

❗ For Linux and Mac OS X, the script is typically named .venv/bin/activate; for Windows, typically .venv/bin/activate.ps1 or ./venv/bin/activate.bat. ❗

Whenever Raven's venv is active, you can use Raven commands, such as raven-visualizer.

Activate GPU compute support (optional)

If CUDA support is installed but not working, you can try enabling CUDA (for the current command prompt session) as follows.

With the venv activated, and the terminal in the Raven folder, run the following bash command:

source env.sh

This sets up the library paths and $PATH so that Raven finds the CUDA libraries. This script is coded to look for them in Raven's .venv subfolder.

Choose which GPU to use (optional)

If your machine has multiple GPUs, there are two ways to tell Raven which GPU to use.

If your system permanently has several GPUs connected, and you want to use a different GPU permanently, you can adjust the device settings in raven.server.config, raven.visualizer.config, and raven.librarian.config.

If you switch GPUs only occasionally (e.g. a laptop that sometimes has an eGPU connected and sometimes doesn't), you can use the CUDA_VISIBLE_DEVICES environment variable to choose the GPU temporarily, for the duration of a command prompt session.

We provide an example script run-on-internal-gpu.sh, meant for a laptop with a Thunderbolt eGPU (external GPU), which forces Raven to run on the internal GPU when the external is connected (which is useful e.g. if your eGPU is dedicated for a self-hosted LLM). On the machine where the script was tested, PyTorch sees the eGPU as GPU 0 when available, pushing the internal GPU to become GPU 1. When the eGPU is not connected, the internal is GPU 0.

With the venv activated, and the terminal in the Raven folder, run the following bash command:

source run-on-internal-gpu.sh

Then for the rest of the command prompt session, any Raven commands (such as raven-visualizer) will only see the internal GPU, and "cuda:0" in the device settings will point to the only visible GPU.

Exit from the Raven venv (optional, to end the session)

❗ There is usually no need to do this. You can just close the terminal window. ❗

If you want to exit from the Raven venv without exiting your terminal session, you can deactivate the venv like this:

deactivate

After this command completes, python again points to the Python in your Python environment (where e.g. PDM runs), not to Raven's app-local Python.

If you want to also exit your terminal session, you can just close the terminal window as usual; there is no need to deactivate the venv unless you want to continue working in the same terminal session.

Uninstall

python -m pip uninstall raven-visualizer

Or just delete the venv, located in the .venv subfolder of the Raven folder.

AI models auto-install themselves elsewhere:

  • The THA3 AI animator (of Raven-avatar) is auto-installed in the raven/vendor/tha3/models/ subdirectory of your top-level raven directory.

  • The dehyphenator AI model (of Raven-server's sanitize module) is auto-installed in ~/.flair/embeddings/.

  • All other AI models are auto-installed from HuggingFace Hub.

    • These live at the default models cache location of the huggingface_hub Python package, which is usually ~/.cache/huggingface/hub.
    • Note that this models cache is shared between many different Python-based AI apps, so removing everything is not recommended.

Technologies

Raven builds upon several AI, NLP, statistical, numerical and software engineering technologies:

Note that installing Raven will auto-install dependencies into the same venv (virtual environment). This list is here just to provide a flavor of the kinds of parts needed to build a constellation like this.

License

2-clause BSD.

Acknowledgements

This work was financially supported by the gH2ADDVA (Adding Value by Clean Hydrogen production) project at JAMK, co-funded by the EU and the Regional Council of Central Finland.

JAMK Institute of New Industry

About

Absorb scientific literature

Resources

Readme

License

BSD-2-Clause license
Activity

Stars

4 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

4 tags

Packages

No packages published

Languages