Incompatible dependencies causing error with Snowflake runs #5520
Description
Need someone smart to figure out what is happening. I think it is an upstream problem, but I don't now how to confirm that.
We use Snowflake as our database. This morning our regular sqlmesh run jobs just stopped working (we run them every hour).
The logs showed errors like this:
File "C:\Program Files\venv\uv\cache\archive-v0\UpKntVeOkWW3Jxcd1j723\Lib\site-packages\snowflake\snowpark\_internal\analyzer\expression.py", line 14, in <module>
from snowflake.snowpark._internal.type_utils import (
File "C:\Program Files\venv\uv\cache\archive-v0\UpKntVeOkWW3Jxcd1j723\Lib\site-packages\snowflake\snowpark\_internal\type_utils.py", line 32, in <module>
import snowflake.snowpark.types # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\venv\uv\cache\archive-v0\UpKntVeOkWW3Jxcd1j723\Lib\site-packages\snowflake\snowpark\types.py", line 337, in <module>
class PandasDataFrame(pandas.DataFrame, Generic[_TT]):
~~~~~~~^^^^^
File "C:\Program Files\Scoop\apps\python312\current\Lib\typing.py", line 398, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Scoop\apps\python312\current\Lib\typing.py", line 1098, in _generic_class_getitem
raise TypeError(
TypeError: Parameters to Generic[...] must all be type variables or parameter specification variables.
We use uv to run the job e.g.
uvx --python 3.12 --from "sqlmesh[snowflake,postgres]==0.224.0" sqlmesh run prod
After doing some investigation, it appears that the python environment uvx generated had snowflake-snowpark-python==0.12.0 installed which was from 2022 or sometime.
Also had installed snowflake-connector-python==4.0.0. Turns out 4.0.0 was released this morning, just before we started seeing these problems.
https://github.com/snowflakedb/snowflake-connector-python/releases/tag/v4.0.0
Was able to circumvent the problem by amending our uvx call
uvx --python 3.12 --from "sqlmesh[snowflake,postgres]==0.224.0" --with "snowflake-connector-python==3.18.0" sqlmesh run prod
This resulted in uvx using snowflake-snowpark-python==1.40.0 instead -- the most recent release -- which is what you would expect.
Just for fun I tried this in another environment
uvx --python 3.12 --from "sqlmesh[snowflake]==0.225.0" --with "snowflake-connector-python==4.0.0" --with "snowflake-snowpark-python>0.12.0" sqlmesh --version
and got this as the result
× No solution found when resolving tool dependencies:
╰─▶ Because only the following versions of snowflake-snowpark-python are available:
snowflake-snowpark-python<0.12.0
snowflake-snowpark-python==1.0.0
snowflake-snowpark-python==1.1.0
snowflake-snowpark-python==1.2.0
snowflake-snowpark-python==1.3.0
snowflake-snowpark-python==1.4.0
snowflake-snowpark-python==1.5.0
snowflake-snowpark-python==1.5.1
snowflake-snowpark-python==1.6.0
snowflake-snowpark-python==1.6.1
snowflake-snowpark-python==1.7.0
snowflake-snowpark-python==1.8.0
snowflake-snowpark-python==1.9.0
snowflake-snowpark-python==1.10.0
snowflake-snowpark-python==1.11.0
snowflake-snowpark-python==1.11.1
snowflake-snowpark-python==1.12.0
snowflake-snowpark-python==1.12.1
snowflake-snowpark-python==1.13.0
snowflake-snowpark-python==1.14.0
snowflake-snowpark-python==1.15.0
snowflake-snowpark-python==1.16.0
snowflake-snowpark-python==1.17.0
snowflake-snowpark-python==1.18.0
snowflake-snowpark-python==1.19.0
snowflake-snowpark-python==1.20.0
snowflake-snowpark-python==1.21.0
snowflake-snowpark-python==1.21.1
snowflake-snowpark-python==1.22.0
snowflake-snowpark-python==1.22.1
snowflake-snowpark-python==1.23.0
snowflake-snowpark-python==1.24.0
snowflake-snowpark-python==1.25.0
snowflake-snowpark-python==1.26.0
snowflake-snowpark-python==1.27.0
snowflake-snowpark-python==1.28.0
snowflake-snowpark-python==1.29.0
snowflake-snowpark-python==1.29.1
snowflake-snowpark-python==1.30.0
snowflake-snowpark-python==1.31.0
snowflake-snowpark-python==1.31.1
snowflake-snowpark-python==1.32.0
snowflake-snowpark-python==1.33.0
snowflake-snowpark-python==1.34.0
snowflake-snowpark-python==1.35.0
snowflake-snowpark-python==1.36.0
snowflake-snowpark-python==1.37.0
snowflake-snowpark-python==1.38.0
snowflake-snowpark-python==1.39.0
snowflake-snowpark-python==1.39.1
snowflake-snowpark-python==1.40.0
and snowflake-snowpark-python==1.0.0
depends on snowflake-connector-python>=2.7.12,<3.0.0,
we can conclude that snowflake-snowpark-python>0.12.0,<1.1.0 depends on snowflake-connector-python>=2.7.12,<3.0.0.
And because snowflake-snowpark-python>=1.1.0,<=1.5.1
depends on snowflake-connector-python>=2.7.12,<4.0.0 and snowflake-snowpark-python==1.6.0 was yanked,
we can conclude that snowflake-snowpark-python>0.12.0,<1.6.1 depends on snowflake-connector-python>=2.7.12,<4.0.0.
And because snowflake-snowpark-python>=1.6.1,<=1.7.0
depends on snowflake-connector-python>=3.0.4,<4.0.0 and snowflake-connector-python>=3.2.0,<4.0.0,
we can conclude that snowflake-snowpark-python>0.12.0,<1.11.0 depends on snowflake-connector-python>=2.7.12,<4.0.0.
And because snowflake-snowpark-python==1.11.0 was yanked and snowflake-snowpark-python==1.11.1 depends on snowflake-connector-python>=3.4.0,<4.0.0,
we can conclude that snowflake-snowpark-python>0.12.0,<1.12.0 depends on snowflake-connector-python>=2.7.12,<4.0.0.
And because snowflake-snowpark-python>=1.12.0,<=1.15.0
depends on snowflake-connector-python>=3.6.0,<4.0.0 and snowflake-connector-python>=3.10.0,<4.0.0,
we can conclude that all of:
snowflake-snowpark-python>0.12.0,<1.22.0
snowflake-snowpark-python>1.22.0,<1.25.0
depend on snowflake-connector-python>=2.7.12,<4.0.0.
And because snowflake-snowpark-python==1.22.0 was yanked and snowflake-snowpark-python>=1.25.0,<=1.28.0 depends on snowflake-connector-python>=3.12.0,<4.0.0,
we can conclude that snowflake-snowpark-python>0.12.0,<1.29.0 depends on snowflake-connector-python>=2.7.12,<4.0.0.
And because snowflake-snowpark-python>=1.29.0,<=1.38.0 depends on snowflake-connector-python>=3.14.0,<4.0.0 and snowflake-connector-python>=3.17.0,<4.0.0,
we can conclude that snowflake-snowpark-python>0.12.0 depends on snowflake-connector-python>=2.7.12,<4.0.0.
And because you require snowflake-connector-python==4.0.0 and snowflake-snowpark-python>0.12.0, we can conclude that your requirements are unsatisfiable.
You probably won't see this error unless you do a fresh install after today and you are using Snowflake.