mldsa-native integration

[mkannwischer]

This pull request imports mldsa-native - a high-speed high-assurance ML-DSA implementation deveoped as a part of the post-quantum code package (PQCP) and maintained by @hanno-becker, @jakemas, and myself. It includes a portable C implementation, a x86_64 implementation, and an AArch64 implementation. This integration replaces the existing pqcrystals implementations.

This pull request also removes the patches that were required for the pqcrystals implementation. For mldsa-native, we require no patches.
Furthermore, I have removed the constant-time exceptions in tests/constant_time/sig/passes/ml_dsa*. For mldsa-native, we don't require exceptions and instead use the excplicit declassifications (MLD_CT_TESTING_DECLASSIFY)

Performance We have measured the impact of this integration on liboqs performance on various x86_64 CPUs in #546. For the portable implementation we see a solid boost in performance of 9-25% despite conservative constant-time hardening (see #371, #392, #391). For the x86_64-optimized implementation, there is a small performance regression of up to 13% in signing. We primarily account this regression to our x86_64 implementation so far not including all AVX2 intrinsics from the pqcrystals implementation. The reason for being selective in pulling in native code is that we strive to formally verify the correctness of each native function using HOL-Light later on (this is still work on progress). We will revisit that x86_64 performance gap in #545 and hope to close this gap in a follow-up release. Likewise, for AArch64 we are expecting some small performance improvements over time.

Formal verifcation All C code in the portable C implementation is proven type-safe and memory-safe using the C Bounded Model Checker (CBMC) in the same way that mlkem-native is. We plan to formally verify the correctness of the native code using HOL-Light and the s2n-bignum infrastructure - so far this has been completed for the x86_64 NTT (see #640). Both HOL-Light and CBMC proofs run in the mldsa-native CI and should be easy to reproduce.

We are about to release the v1.0.0-alpha version, and I will switch to the named release once available

This PR uses the recently released v1.0.0-alpha version of mldsa-native

• Resolves Update ML-DSA implementation to use mldsa-native once available #2272

[mkannwischer]

I think all failures are unrelated. Feel free to close this PR if it is bothering you, I can open a new one once we are ready.

I added the OQS import to the mldsa-native CI in pq-code-package/mldsa-native#480 which should keep everything working.

[mkannwischer]

We are nearing the v1.0.0-alpha release and I don't expect any more code changes until then. I'll update to the named release once it's out. Feel free to review already.

[mkannwischer]

mldsa-native v1.0.0-alpha just landed. I have updated this PR to use that version.

[bhess]

Thank you for this work and the integration, @mkannwischer, looks really good!
I really like how smoothly the integration works without needing any patches. The new arm64 optimization and the formally proven memory safety are a big plus. The AVX2 performance penalty seems acceptable to me, given your explanation and the plan to improve it in the next release.

I have two extra questions (besides the inline ones):

• Are HashML-DSA variants available in mldsa-native, or are they planned for the future?
• Could the externalMu variant be used if liboqs decides to support it?

[mkannwischer]

Thanks for the review @bhess.

I have two extra questions (besides the inline ones):

• Are HashML-DSA variants available in mldsa-native, or are they planned for the future?
• Could the externalMu variant be used if liboqs decides to support it?

Yes, we support both HashML-DSA and externalMu mode. Both are exposed in the public API.
See https://github.com/pq-code-package/mldsa-native/blob/main/mldsa/mldsa_native.h.
We match all testvectors that are in the ACVP testsuite - so we have full feature coverage.
We should be able to add those to liboqs relatively easily.
Could you take a look at that in a follow-up PR @bhess?

[bhess]

Yes, we support both HashML-DSA and externalMu mode. Both are exposed in the public API.
See https://github.com/pq-code-package/mldsa-native/blob/main/mldsa/mldsa_native.h.
We match all testvectors that are in the ACVP testsuite - so we have full feature coverage.
We should be able to add those to liboqs relatively easily.
Could you take a look at that in a follow-up PR @bhess?

Great to see these features available! As mentioned in yesterday’s OQS call, @xuganyu96 will likely take a closer look at pre-hash in OQS (including for SLH-DSA) as a follow-up.

[bhess]

Looks good to me. Thank you very much @mkannwischer for the work on mldsa-native, and the integration to liboqs!

Asking @open-quantum-safe/liboqs-committers for another look.