This stems from an old issue where [`jinja2` imports were deprecated](https://github.com/sphinx-doc/sphinx/issues/10291). [PR #47](https://github.com/ammaraskar/sphinx-action/pull/47), which updates the Docker image to use `sphinx 5.0.2` should resolve this for at least `python <3.10` (see [`astropy/astropy` Issue #11725](https://github.com/astropy/astropy/issues/11725)).