Is dynamic weight update / fine-tuning supported in QNN / XNNPACK backends?

[qqqqqqqwy]

🚀 The feature, motivation and pitch

I’m working on a research project to fine-tune a model on Android devices. I am exploring using ExecuTorch + QNN or XNNPACK backend for inference acceleration, but need to ensure that the backend can support dynamic modification of weights (i.e., after initialization, allow updating weights / biases, and then run forward again).

What I found

• In executorch/extension/training/examples/XOR/train.cpp, training code based on executorch is provided, but it does not mention the supported backends.
• The official XNNPACK backend documentation describes that, during runtime initialization, weights / biases are “packed” (i.e. weight packing) into XNNPACK’s internal data structures, and the original preprocessed blob’s data is freed. This seems to imply that weights become static / immutable from the perspective of the backend’s execute graph.
• I did not find description in the docs or runtime API of any mechanism to “unlock” or “update” those packed weights at runtime.
• There is an existing issue (PT2E Dynamic Quantization with XNNPACK Backend for Android fails Loading of method forward in runtime #11355) reporting that even dynamic quantization + XNNPACK + Android may fail to load “forward” method, which suggests that non-static quantization / dynamic behavior is fragile or unsupported.
• For QNN backend, I saw open / triaged issues about compilation or binary loading, but none that explicitly mention support for runtime weight update.

My questions

1. Does ExecuTorch (any of its backends: QNN, XNNPACK, Vulkan, etc.) currently support runtime in-place weight updates (i.e. treat model weights as mutable parameters, allow updating them between forward calls, as required in fine-tuning / training / zeroth-order optimization)?
2. If not supported, is there a recommended workflow / workaround for on-device fine-tuning with ExecuTorch? Or is this explicitly out of scope?
3. If it’s not currently supported, would the maintainers be open to considering such a feature in future (e.g. a “mutable weight” delegate, or mechanism to reload new weights into backend graph)?

Thank you for your time and for developing ExecuTorch — it is a great tool for on-device inference / deployment, and I hope it can support on-device fine-tuning in the future.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

cc @JacobSzwejbka

[cccclai]

ExecuTorch on XNN might be working a bit, not on QNN at the moment. @JacobSzwejbka probably knows the most recent progress there.

[qqqqqqqwy]

Hi @JacobSzwejbka. Would it be possible for us to continue the conversation via email? My email is 8208231603@csu.edu.cn. Thank you very much for your time, and I look forward to hearing from you.

[JacobSzwejbka]

Only XNNPack has support (and even then only really for linear layers). XNNPacks support though is using a slower kernel which does not use packed weights.

There is work around XNNPack and Vulkan to support loading external weights in an ATen compliant serialization keyed by fqn which would then allow you to ship 2 graphs.

One training graph which uses the slower kernel but supports dynamic weight update and the other is an inference only graph which packs the weights at init.

[JacobSzwejbka]

Oh you have to set a flag on the XNNPack partitioner to have it use the proper mode. https://github.com/pytorch/executorch/blob/main/backends/xnnpack/partition/config/xnnpack_config.py#L51 I can update the example to show that.