torch.OutOfMemoryError: CUDA out of memory.

[The-Darumonkey]

I think it's salient to say that it used to work. I was on Kubuntu 24.04.3 and was happily generating away with no issues.

I reinstalled Kubuntu 24.04.3 today, and while ComfyUI worked fine after reinstalling it, I've been completely unable to get FP-Studio, or even the original FramePack, to work. The generation goes ahead for the first sampling run, then breaks at Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete.

I booted to Windows 11, installed FramePack Studio there, and it works just fine. I installed Mint (also Debian-derived) and tried installing FP-Studio on that. No dice.

I've tried running it with all three attention packages installed (individually) to no avail. I've made sure that there is enough swap space (50GB) to handle the models when they're unloaded, in the event that it was memory capacity that was the issue.

I followed the Python venv-based guidance on the wiki, which previously worked. I have CUDA 12.8 installed. I am running the proprietary Nvidia driver. Secure boot is on and I have enrolled my MOK, so I don't think it's a CUDA/Nvidia driver issue (although I honestly wouldn't really know if it were).

I'm completely out of ideas (and, to be frank, my depth) here.

Pertinent specs (that I can think of):

• CPU: AMD Ryzen7 7800X3D
• RAM: 64GB DDR5-6000
• GPU: Nvidia RTX 4070 Super (12GB VRAM)
• GPU driver: Nvidia-580-open
• Python version: 3.12.3
• 50GB swap partition on a Gen4 NVME drive

Advice would be much appreciated. I've learnt in my years that these kinds of issues are usually down to me doing stupid.

[pahnin]

I am trying to change the code to use lesser VRAM, I have 3050 8GB, like you mentioned, it goes past the first sampling and then it runs out of VRAM, but with the approach I am doing, it is swallowing frames, I want to create a PR if my approach works, I can share a draft PR may be you could have a look

[colinurbs]

@The-Darumonkey can you please post your logs in the 'help' section of the discord? Would be happy to try to figure this out. Discord is generally better for support.

@pahnin sounds great. I'll leave this issue open so you can update us both if you end up submitting the PR.

[The-Darumonkey]

@The-Darumonkey can you please post your logs in the 'help' section of the discord? Would be happy to try to figure this out. Discord is generally better for support.

@pahnin sounds great. I'll leave this issue open so you can update us both if you end up submitting the PR.

Hi.

I 'fixed' the issue with Stability Matrix (a launcher for AI tools). I suspect the issue was caused by my new install using a later CUDA toolkit (now at 13.0).

I'd prefer to avoid Discord for support, as one must be a member of the Discord server to see the discussion and results - I don't think open source software should obfuscate solutions to problems in the name of convenience.

[pahnin]

@The-Darumonkey
that makes sense, it stopped working for me after updating cuda toolkit as well

@colinurbs I will create a PR soon, what I was trying to do was trying to implement tiled vae encoding, I think it will help in performance, thanks

[pahnin]

ok, good news, I was able to get it working, multiple iterations and no OOM error, I am doing a bit more testing, will create a PR today.

[pahnin]

[phanindra@phanindra-ms7e26 FramePack-Studio]$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Wed_Aug_20_01:58:59_PM_PDT_2025
Cuda compilation tools, release 13.0, V13.0.88
Build cuda_13.0.r13.0/compiler.36424714_0
[phanindra@phanindra-ms7e26 FramePack-Studio]$ nvidia-smi
Thu Nov  6 11:32:44 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05              Driver Version: 580.95.05      CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3050        Off |   00000000:01:00.0  On |                  N/A |
| 33%   60C    P2            N/A  /  115W |    4925MiB /   8192MiB |    100%      Default |
|                                         |                        |                  N/A |