[Bug]: OpenRouter OpenAI API causes embeddings_maintainer to crash

[russkinz]

Checklist

• I have updated to the latest available Frigate version.
• I have cleared the cache of my browser.
• I have tried a different browser to see if it is related to my browser.
• I have tried reproducing the issue in incognito mode to rule out problems with any third party extensions or plugins I have installed.

Describe the problem you are having

Around 60% of Frigate's API calls to OpenRouter via their OpenAI API endpoint causes the embeddings_maintainer python app to crash.

The balance of the time it works without issues.

Standard OpenAI works, gemini works, ollama works. It just seems to be the OpenRouter OpenAI API endpoint which causes this.

Steps to reproduce

1. Switch OPENAI_BASE_URL to https://openrouter.ai/api/v1
2. Send requests as usual until a failure occurs (usually within 2-4 requests)
3. Recovery seems to require a restart of Frigate (I have yet to see it self-recover)

Version

0.15.0-cea210d

In which browser(s) are you experiencing the issue with?

No response

Frigate config file

# yaml-language-server: $schema=http://*:5000/api/config/schema.json
mqtt:
  enabled: true
  host: *
  port: 1883
  user: *
  password: *
auth:
  enabled: false
birdseye:
  enabled: true
detectors:
  coral:
    type: edgetpu
    device: usb
model:
  path: plus://*
ffmpeg:
  hwaccel_args: preset-intel-qsv-h264
go2rtc:
  streams:
    Path:
      - rtsp://*:554/Preview_01_main
      - ffmpeg:Path#video=h264#audio=aac#hardware
    Path_sub:
      - rtsp://*/Preview_01_sub
    Driveway:
      - rtsp://*/Preview_01_main
      - ffmpeg:Driveway#video=h264#audio=aac#hardware
    Driveway_sub:
      - rtsp://*/Preview_01_sub
    Backdoor:
      - rtsp://*:554/Preview_01_main
      - ffmpeg:Backdoor#video=h264#audio=aac#hardware
    Backdoor_sub:
      - rtsp://*:554/Preview_01_sub
#    Frontdoor:
#      - rtsp://*:554/Preview_02_main
#      - "ffmpeg:Frontdoor#video=h264#audio=aac#hardware"
#    Frontdoor_sub:
#      - rtsp://*:554/Preview_02_sub
    Shitcam:
      - rtsp://*:554/stream1
      - ffmpeg:Shitcam#audio=aac
    Shitcam_sub:
      - rtsp://*:554/stream2
cameras:
  Shitcam:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/Shitcam
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Shitcam_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
    onvif:
      host: 192.168.1.173
      port: 2020
      user: tapoadmin
      password: ade35r
  Path:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/Path
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Path_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
    detect:
      width: 640
      height: 360
      fps: 4
    zones:
      Path_Zone:
        coordinates: 
          0,0,0,0.385,0.283,0.328,0.319,0.625,0.202,1,0.359,1,0.43,1,0.46,0.846,0.475,0.751,0.602,0.75,0.608,0.869,0.644,0.879,0.628,1,0.803,1,1,1,1,0,0.731,0,0.34,0
        inertia: 3
        loitering_time: 0
      Path_Front_Door_Light_Detection_Zone:
        coordinates: 0.275,0.259,0.317,0.62,0.198,0.999,0.832,0.998,0.856,0.318,0.577,0.241
        loitering_time: 0
        objects: person
        inertia: 3
    motion:
      mask: 0.508,0.014,0.508,0.071,0.974,0.075,0.975,0.012
  Driveway:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/Driveway
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Driveway_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
    detect:
      width: 640
      height: 360
      fps: 4
  Backdoor:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/Backdoor
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Backdoor_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
    detect:
      width: 896
      height: 512
      fps: 5
objects:
  track:
    - person
    - car
    - cat
    - dog
    - package
    - bus
    - nzpost
    - motorcycle
    - bicycle
    - face
snapshots:
  enabled: true
  timestamp: false
  bounding_box: true
  retain:
    default: 14
record:
  enabled: true
  sync_recordings: true
  retain:
    days: 7
    mode: all
  alerts:
    retain:
      days: 14
  detections:
    retain:
      days: 14
semantic_search:
  enabled: true
  reindex: false
  model_size: small
genai:
  enabled: true
  provider: openai
  api_key: *
  model: google/gemini-2.5-pro-exp-03-25:free
  prompt: "Analyze the sequence of images containing the {label}, taken from a residential security camera, {camera}. Focus on the likely intent or behavior of the {label} based on its actions and movement, rather than describing its appearance or the surroundings. Consider what the {label} is doing, why, and what it might do next. Assume any date or time stamps are in the NZ regional format."
  object_prompts:
    cat: "Analyze the sequence of images taken from a residential security camera, {camera}, containing cat(s). Focus on the likely intent or behavior of the cat(s) based on its actions and movement. Observe whether the cat jumps over a fence in the sequence of images. Describe the appearance of the cat, and consider what it is doing, why, and what it might do next. Assume any date or time stamps are in the NZ regional format."
version: 0.15-1
camera_groups:
  Front_of_House:
    order: 1
    icon: LuArrowBigUp
    cameras:
      - Path
      - Driveway
  Birdseye:
    order: 2
    icon: LuEye
    cameras: birdseye

docker-compose file or Docker CLI command

docker create --name='frigate' --net='bridge' --pids-limit 2048 -e TZ="Pacific/Auckland" -e HOST_OS="Unraid" -e HOST_HOSTNAME="babel" -e HOST_CONTAINERNAME="frigate" -e 'PLUS_API_KEY'='10c9422d-3250-4be7-a745-150609b39436:da3d994ad08866a97e5fc9692c45cc947b1199fa' -e 'PUID'='99' -e 'PGID'='100' -e 'UMASK'='022' -e 'NVIDIA_VISIBLE_DEVICES'='none' -e 'NVIDIA_DRIVER_CAPABILITIES'='none' -e 'OPENAI_BASE_URL'='https://openrouter.ai/api/v1' -l net.unraid.docker.managed=dockerman -l net.unraid.docker.webui='https://[IP]:[PORT:5000]' -l net.unraid.docker.icon='https://raw.githubusercontent.com/yayitazale/unraid-templates/main/frigate.png' -p '8971:8971/tcp' -p '8554:8554/tcp' -p '5000:5000/tcp' -p '8555:8555/tcp' -p '8555:8555/udp' -v '/mnt/user/appdata/frigate/':'/config':'rw' -v '/mnt/user/nvr':'/media/frigate':'rw' -v '/etc/localtime':'/etc/localtime':'rw' --device='/dev/bus/usb' --device='/dev/dri/renderD129' --shm-size=1024m --mount type=tmpfs,target=/tmp/cache,tmpfs-size=1000000000 --restart=unless-stopped --cap-add=CAP_PERFMON 'ghcr.io/blakeblackshear/frigate:stable'

Relevant Frigate log output

Request: POST https://openrouter.ai/api/v1/chat/completions "HTTP/1.1 200 OK"
2025-04-07 15:30:26.318817668  Exception in thread embeddings_maintainer:
2025-04-07 15:30:26.318823212  Traceback (most recent call last):
2025-04-07 15:30:26.318824525    File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
2025-04-07 15:30:26.318825423      self.run()
2025-04-07 15:30:26.318826404    File "/opt/frigate/frigate/embeddings/maintainer.py", line 75, in run
2025-04-07 15:30:26.318827200      self._process_event_metadata()
2025-04-07 15:30:26.318828049    File "/opt/frigate/frigate/embeddings/maintainer.py", line 287, in _process_event_metadata
2025-04-07 15:30:26.318828927      self.handle_regenerate_description(event_id, source)
2025-04-07 15:30:26.318829918    File "/opt/frigate/frigate/embeddings/maintainer.py", line 398, in handle_regenerate_description
2025-04-07 15:30:26.318830789      self._embed_description(event, embed_image)
2025-04-07 15:30:26.318831760    File "/opt/frigate/frigate/embeddings/maintainer.py", line 313, in _embed_description
2025-04-07 15:30:26.318832712      description = self.genai_client.generate_description(
2025-04-07 15:30:26.318833730    File "/opt/frigate/frigate/genai/__init__.py", line 48, in generate_description
2025-04-07 15:30:26.318834574      return self._send(prompt, thumbnails)
2025-04-07 15:30:26.318835356    File "/opt/frigate/frigate/genai/openai.py", line 53, in _send
2025-04-07 15:30:26.318836078      if len(result.choices) > 0:
2025-04-07 15:30:26.318836844  TypeError: object of type 'NoneType' has no len()

Relevant go2rtc log output

N/A

Operating system

UNRAID

Install method

Docker CLI

Network connection

Wired

Camera make and model

reolink

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[hawkeye217]

This must be something particular with the way that the OpenRouter endpoint returns results. The Frigate code could probably be more robust as to not crash, but it seems likely that if the other endpoints are working, especially the real OpenAI endpoint, that OpenRouter is not completely OpenAI compatible.

[russkinz]

This must be something particular with the way that the OpenRouter endpoint returns results. The Frigate code could probably be more robust as to not crash, but it seems likely that if the other endpoints are working, especially the real OpenAI endpoint, that OpenRouter is not completely OpenAI compatible.

They claim they're completely compatible with the OpenAI API, but yes that seems probable. OpenRouter is becoming pretty popular and supports a very wide array of different LLMs so perhaps this should become a feature request not a bug request.

In any case, I've started a regen of the embeddings (because I couldn't really think of anything more useful to do right now) which seems to be going fine. After that's done I'll reboot and turn on full debug logging and see if there's any more info I can provide.

[hawkeye217]

Thanks. I'll put up a small PR to ensure we return None and not crash in that spot when an invalid result is returned.

[hawkeye217]

#17572

This PR will get merged to dev (0.16), not master (0.15), so you'll have to manually apply it to your instance.

[russkinz]

Awesome, thank you - will go ahead and manually apply.

[russkinz]

Quick update - debug logs show nothing new of value, so have applied the fix manually and will restart the container when I'm home.