[Tracing] Copy HFTracer to support latest transformers

[kylesayrs]

Background

• 🚨 Remove torch.fx support huggingface/transformers#41683 remove some fx tracing utilities used by LLM Compressor's sequential pipeline capabilities

Purpose

• Allow LLM Compressor to support latest transformers

Changes

• Copied src/transformers/utils/fx.py (with copyright)
• Use trust_remote_code for trace testing (the old deepseek model definitions are not supported with the latest transformers

Testing

• Used test_models to test that all models can still trace with transformers main
• Quantized llama 8B e2e and confirmed sane outputs with transformers main

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

[gemini-code-assist]

Summary of Changes

Hello @kylesayrs, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request updates the LLM Compressor to ensure compatibility with recent changes in the transformers library. Specifically, it addresses the removal of certain FX tracing utilities from transformers by internalizing the necessary HFTracer components directly into the llmcompressor codebase. This allows the LLM Compressor's sequential pipeline capabilities to continue functioning correctly with the latest transformers versions, while also adjusting test configurations to accommodate these updates.

Highlights

• Internalized Tracing Utility: Copied the HFTracer utility from transformers.utils.fx into llmcompressor to maintain compatibility with the latest transformers library.
• Updated Import Paths: Modified src/llmcompressor/pipelines/sequential/helpers.py to import the newly internalized HFTracer.
• Adjusted Test Configuration: Changed trust_remote_code to False in test_models.py for trace testing, addressing compatibility issues with older model definitions and newer transformers versions.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request addresses a breaking change in the transformers library by vendoring the necessary fx utility file. This is a good pragmatic solution to unblock development and ensure compatibility with the latest transformers version.

However, vendoring a large file like this introduces a maintenance burden. The copied code will not receive updates or bug fixes from upstream. I've identified a few critical bugs in the copied file that were likely present in the original transformers code. Please see my detailed comments.

For the long term, it might be beneficial to:

1. Contribute fixes for these bugs back to the transformers library if they still exist in some form there.
2. Investigate if a smaller, more focused utility could be extracted from this file to reduce the maintenance surface.

Overall, the approach is sound for the immediate problem, and the changes in the test files look appropriate.

[HDCharles]

looks good

[dsikka]

LGTM. Should we add any of their existing unit tests for the helpers?

[kylesayrs]

@dsikka I think we're good for now. Tracing is already tested e2e