What's the most recent Mistral/EU LLM <9B @ 4 bits for Unsloth fine-tuning?

[mags0ft]

Hey to y'all,

the title is pretty much my question. I am constrained (due to policy requirements for a project, please don't question why 😮‍💨) to using European, open-weights LLMs for fine-tuning with Unsloth. The model in question should not be bigger than 9B parameters, ideally smaller. And recent enough.

I've found out about Ministral 8B, which looks awesome, but Unsloth hasn't released 4-bit quants for the base model, because the base model isn't even open-weights as far as I'm concerned.

Is Mistral 7b v0.3 @ 4 bits still a good choice in 2025?

Any help is greatly appreciated!

Best greets.

[mags0ft]

Okay, after some research, Apertus-8B seems to look good! I'll check if it works and hopefully remember to tell you guys and gals over here. :)

[shimmyshimmer]

Mistral currently only released 24B parameter models now so you will have to look at other providers. I think OLMO models might be good but they're international/American I'm not exactly sure

[mags0ft]

That sounds interesting, thank you! I'll take a look.