[BUG] ModuleNotFoundError: No module named 'vlmeval.utils'

[SnowCharmQ]

Hi authors, thanks for the insightful work, and I am very interested in your benchmark. However, when I was trying to reproduce the experiments following the script in the repo, I encountered the following error:

It seems the vlmeval.utils module is not properly included in the project/repo and I find the item in the .gitignore file, which may cause the issue:

Could you please upload the missing files to the project so that we can reproduce the experiments successfully?

Thanks and regards.

[SnowCharmQ]

Hi authors, I saw you just updated the project a few days ago, and the previous bug was resolved. However, I still encountered a similar issue below:

Traceback (most recent call last):
  File "/mnt/ali-sh-1/dataset/zeus/ylqiu/codes/demo_exp/MMPB/VLMEvalKit/run.py", line 10, in <module>
    from vlmeval.config import supported_VLM
  File "/mnt/ali-sh-1/dataset/zeus/ylqiu/codes/demo_exp/MMPB/VLMEvalKit/vlmeval/__init__.py", line 7, in <module>
    from .api import *
  File "/mnt/ali-sh-1/dataset/zeus/ylqiu/codes/demo_exp/MMPB/VLMEvalKit/vlmeval/api/__init__.py", line 4, in <module>
    from .qwen_vl_api import QwenVLWrapper, QwenVLAPI, Qwen2VLAPI
  File "/mnt/ali-sh-1/dataset/zeus/ylqiu/codes/demo_exp/MMPB/VLMEvalKit/vlmeval/api/qwen_vl_api.py", line 8, in <module>
    from vlmeval.vlm.qwen2_vl.prompt import Qwen2VLPromptMixin
  File "/mnt/ali-sh-1/dataset/zeus/ylqiu/codes/demo_exp/MMPB/VLMEvalKit/vlmeval/vlm/__init__.py", line 5, in <module>
    from .aria import Aria
  File "/mnt/ali-sh-1/dataset/zeus/ylqiu/codes/demo_exp/MMPB/VLMEvalKit/vlmeval/vlm/aria.py", line 8, in <module>
    from .base import BaseModel
  File "/mnt/ali-sh-1/dataset/zeus/ylqiu/codes/demo_exp/MMPB/VLMEvalKit/vlmeval/vlm/base.py", line 2, in <module>
    from ..dataset import img_root_map, DATASET_TYPE
  File "/mnt/ali-sh-1/dataset/zeus/ylqiu/codes/demo_exp/MMPB/VLMEvalKit/vlmeval/dataset/__init__.py", line 5, in <module>
    from .image_yorn import ImageYORNDataset
  File "/mnt/ali-sh-1/dataset/zeus/ylqiu/codes/demo_exp/MMPB/VLMEvalKit/vlmeval/dataset/image_yorn.py", line 4, in <module>
    from .utils import build_judge, DEBUG_MESSAGE
ModuleNotFoundError: No module named 'vlmeval.dataset.utils'

It seems there should be a utils.py file in the folder 'vlmeval.dataset.utils'.

Thanks for your work!

[SnowCharmQ]

Hi @KIM-JAKE, any update on this issue?

[KIM-JAKE]

Hi, I just added vlmeval/dataset/utils a moment ago. Could you please try again and see if the issue is resolved?

If you run into any other problems, feel free to let me know right away and I’ll fix them promptly.