-
Notifications
You must be signed in to change notification settings - Fork 92
test(e2e): basic E2Es #440
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
test(e2e): basic E2Es #440
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good overall. It is fairly self-contained, so it shouldn't cause any issues as long as the tests aren't flaky. Just one blocker as discussed in the call.
.github/workflows/e2e.yml
Outdated
| uv self update | ||
| uvx --version | ||
| echo python | ||
| uv python install 3.12 --default | ||
| python --version |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
During our call, we concluded that it would likely be a better practice to use the pylock to set dependencies in the E2E tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jaredoconnell Unfortunately it ended up as a change to tox usage all across the board - since we never actually used the pylock.toml in our tests.
I added a tox plugin called tox-pdm which uses our lock file, then I integrated that modification to all tox related workflows.
From now on - every single tox based test environment will use pdm and pylock.toml by default.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few small nits
|
@sjmonson I just straight up applied all of your suggestions. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One more nit. Also needs signoff, E.g. git rebase --signoff main.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Revert this. It resolves to python>=3.10,<4.0, python>=3.11,<4.0, python>=3.12,<4.0; overlapping ranges do not make sense to lock separately. The previous version was python>=3.12,<4.0, python>=3.10,<3.12. Having a split is not strictly necessary anymore since we dropped 3.9 but it does allow us to lock newer packages for the modern python version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. I can't really fully revert this, because the original generated pylock.toml does not work for 3.10 (for example torchcodec revolves only 3.12 and 3.13), and the CI fails.
I am working on modifying my version to generate non-overlapping ranges that still work for both 3.10 and 3.13 to pass the CI. Will update
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK I see the issue - I was using the generate_pylock.sh script without -f which generated an invalid pylock.toml (not sure exactly why, seems like a strange interaction between --append and --update-reuse).
In any case, I am applying a full revert of my changes
279e62e to
2d83c3d
Compare
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Samuel Monson <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Samuel Monson <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Samuel Monson <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Samuel Monson <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Samuel Monson <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Samuel Monson <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Samuel Monson <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Samuel Monson <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: guangli.bao <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: guangli.bao <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: guangli.bao <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: guangli.bao <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: guangli.bao <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: guangli.bao <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: dalthecow <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: dalthecow <[email protected]> Signed-off-by: Alon Kellner <[email protected]>
Signed-off-by: Alon Kellner <[email protected]>
2d83c3d to
8de814d
Compare
Signed-off-by: Alon Kellner <[email protected]>
Summary
E2E tests which check basic GuideLLM functionality, using vLLM simulator.
Details
Test Plan
Use of AI
## WRITTEN BY AI ##)