Using Ground Truth in Evaluator for HotpotQA

[Skevinci]

Hi there,

When I was inspecting the code, I found that it was using ground truth in the Evaluator part for HotpotQA. I'm wondering is that correct?

Thank you!

[YangningLi1]

same, it is really strange

[becklabs]

Yes, this is how the task is defined in the paper (see Sec. 4.2). @Skevinci