Skip to content

Fail to run CotQA_context.ipynb in hotpotQA. parse_action(string) in agents.py returns None causing TypeError #47

@ennka

Description

@ennka

Hi,

I encountered an issue when running the code. Specifically, the parse_action(string) function in agents.py appears to return None, which leads to a TypeError during execution. The traceback is as follows:

File "/my_own_path/reflexion-main/hotpotqa_runs/agents.py", line 91, in step
action_type, argument = parse_action(action)
TypeError: cannot unpack non-iterable NoneType object

From my understanding, this issue arises because the LLM sometimes outputs an Action when it is only supposed to output a Thought. This situation disrupts the expected format and results in parse_action failing to parse the output properly. While this behavior of LLMs is not uncommon, I find it surprising that this issue was not accounted for during the experiments, as it seems like a significant edge case.

Could you please advise on how to handle this situation? Are there any recommended strategies to mitigate this issue, such as improving the model's output alignment or adding a fallback mechanism in the code?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions