Add support for fp16 iGEMM with SME2

[JonathanC-ARM]

Initial prototype for FP16 Igemm support for SME2
continuing work from ##8687

[dsharlet]

This isn't building for us:

test/gemm-microkernel-tester.cc:2455:40: error: no viable overloaded '+='
 2455 |         c_ref[m_index * n() + n_index] +=
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
 2456 |             xnn_float16_to_float(input_f16[m_index * k() + k_index]) *
      |             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 2457 |             xnn_float16_to_float(weights[n_index * k() + k_index]);
      |             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test/gemm-microkernel-tester.cc:2459:38: error: no viable overloaded '+='
 2459 |       c_ref[m_index * n() + n_index] += xnn_float16_to_float(bias[n_index]);
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 errors generated.

[JonathanC-ARM]

Hi @dsharlet could you give a bit more context of the error, particularly around the build command used. The strange thing on my end is that I cant see this.

I am compiling on an M4 however, but I'm going to try on an x86_64 machine shortly and cross compile

bazel build -c opt --enable_bzlmod --define xnn_enable_arm_sme=true --define xnn_enable_arm_sme2=true //test:gemm_microkernel_tester

Tried a few variations on the command, cleaned my environment etc. Also synced my fork with master in case anything since.

[JonathanC-ARM]

@dsharlet thanks for telling me about the build problem, seemed to only show up on Linux machines. I was able to fix the build issue in the latest commit.

Will be resolving the conflicts with Master shortly.

[gonnet]

This is still failing to build for the CI workflows, e.g. https://github.com/google/XNNPACK/actions/runs/18713631848/job/53367695764.

[JonathanC-ARM]

@gonnet I have resolved the build failures and tested it both on a ubuntu machine and mac with clean environments. So should be building now.

[JonathanC-ARM]

Hi @dsharlet thanks for the approval, I seen there was a few failures in the workflows. Two of them appear to be avx related tests, so not sure that is necessarily related to this change. But the other was an unused variable warning on one gcc version, so i've gone ahead and removed that now, so think it should be ok next time around.

[dsharlet]

The failures on this branch look real, a lot of tests that run SME code are crashing.

[JonathanC-ARM]

@dsharlet yeah that seems to have been the case, spent a good bit of today sorting through things. I should have a new push shortly. The crashes appear to be missing sme1 or 2 variants of pack functions for example, in the crashing batch matmul test case it seemed that some gemm config pointers were null which led me to missing packs etc..

Also a quick question which has come from this maybe yourself or @gonnet might be able to answer, the tools/update_microkernels.py file autogenerates bazel and cmake files for various microkernels. But one thing I've encountered is that for example most sme2 kernels their pack functions are sme, the naming convention of the file dictates what is included in those bazel or cmake files. The only way without modifying the logic of the python script is to basically duplicate files across neonsme and neonsme2, it seems that has been the case, but I just wanted to ask your thoughts on it.

[JonathanC-ARM]

Hi @dsharlet I made some additional changes, and ran all of //test/... with sme2 on/off and vice versa. Everything seemed to pass testing, I was able to replicate the original failures and work through them. So I think it should be all good now.

Thanks

[dsharlet]

There were some issues with build timeouts earlier. I re-ran the failed builds, there is a remaining real build issue:

C:\Users\runneradmin\.cargo\bin\ccache.exe C:\PROGRA~1\MICROS~2\2022\ENTERP~1\VC\Tools\MSVC\1444~1.352\bin\Hostx64\arm64\cl.exe  /nologo /TP -DNOMINMAX -DPTHREADPOOL_NO_DEPRECATED_API=1 -DXNN_ENABLE_ARM_BF16=0 -DXNN_ENABLE_ARM_DOTPROD=1 -DXNN_ENABLE_ARM_FP16_SCALAR=0 -DXNN_ENABLE_ARM_FP16_VECTOR=1 -DXNN_ENABLE_ARM_I8MM=1 -DXNN_ENABLE_ARM_SME2=0 -DXNN_ENABLE_ARM_SME=1 -DXNN_ENABLE_ASSEMBLY=0 -DXNN_ENABLE_AVX256SKX=1 -DXNN_ENABLE_AVX256VNNI=1 -DXNN_ENABLE_AVX256VNNIGFNI=1 -DXNN_ENABLE_AVX2=1 -DXNN_ENABLE_AVX512AMX=1 -DXNN_ENABLE_AVX512BF16=0 -DXNN_ENABLE_AVX512F=1 -DXNN_ENABLE_AVX512FP16=0 -DXNN_ENABLE_AVX512SKX=1 -DXNN_ENABLE_AVX512VBMI=1 -DXNN_ENABLE_AVX512VNNI=1 -DXNN_ENABLE_AVX512VNNIGFNI=1 -DXNN_ENABLE_AVX=1 -DXNN_ENABLE_AVXVNNI=1 -DXNN_ENABLE_AVXVNNIINT8=0 -DXNN_ENABLE_CPUINFO=1 -DXNN_ENABLE_F16C=1 -DXNN_ENABLE_FMA3=1 -DXNN_ENABLE_HVX=1 -DXNN_ENABLE_KLEIDIAI=0 -DXNN_ENABLE_RISCV_VECTOR=1 -DXNN_ENABLE_SPARSE=1 -DXNN_ENABLE_SSE2=1 -DXNN_ENABLE_SSE41=1 -DXNN_ENABLE_SSE=1 -DXNN_ENABLE_SSSE3=1 -DXNN_ENABLE_VSX=1 -DXNN_ENABLE_WASM_REVECTORIZE=0 -DXNN_LOG_LEVEL=0 -IC:\a\XNNPACK\XNNPACK\include -IC:\a\XNNPACK\XNNPACK\build\windows\arm64\pthreadpool-source\include -external:IC:\a\XNNPACK\XNNPACK\. -external:IC:\a\XNNPACK\XNNPACK\build\windows\arm64\googletest-source\googlemock\include -external:IC:\a\XNNPACK\XNNPACK\build\windows\arm64\googletest-source\googlemock -external:IC:\a\XNNPACK\XNNPACK\build\windows\arm64\googletest-source\googletest\include -external:IC:\a\XNNPACK\XNNPACK\build\windows\arm64\googletest-source\googletest -external:W0 /UNDEBUG  /DWIN32 /D_WINDOWS /GR /EHsc /O2 /Ob2 /DNDEBUG -std:c++14 -MD /wd4146 /bigobj /wd4190 /O2 /DEBUG:FASTLINK /Zi /showIncludes /Fotest\CMakeFiles\gemm-microkernel-tester.dir\gemm-microkernel-tester.cc.obj /Fdtest\CMakeFiles\gemm-microkernel-tester.dir\gemm-microkernel-tester.pdb /FS -c C:\a\XNNPACK\XNNPACK\test\gemm-microkernel-tester.cc
C:\a\XNNPACK\XNNPACK\test\gemm-microkernel-tester.cc(2864): error C3861: 'xnn_packed_size_kai_f16_conv_goki_w': identifier not found
C:\a\XNNPACK\XNNPACK\test\gemm-microkernel-tester.cc(2870): error C3861: 'xnn_pack_kai_f16_conv_goki_w_sme': identifier not found

[JonathanC-ARM]

I just made some small tweaks for ifdef's which meant this stuff was getting into non kleidi builds.
bazel test --compilation_mode=opt --define xnn_enable_assembly=false --define xnn_enable_arm_fp16_scalar=false --define xnn_enable_arm_bf16=false --define xnn_enable_kleidiai=false //test/... Was able to see the failure resolved it and from what I was able to test on my end it should be working now.