Fix bug with GRPO trainer when tokenizer outputs token_type_ids

[d0rbu]

When training with a tokenizer that has return_token_type_ids=True as default, we get an error from _validate_model_kwargs in the transformers library:

The following `model_kwargs` are not used by the model: ['token_type_ids'] (note: typos in the generate arguments will also show up in this list)

This sets return_token_type_ids to False to prevent this from happening in the GRPO trainer. PleIAs/Baguettotron and PleIAs/Monad are examples of models that I ran into this issue with.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a GitHub issue? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[qgallouedec]

I'm not sure about this one. The issue probably comes from the fact that that the tokenizer configuration is wrong. A patch for this could be

trainer = GRPOTrainer(
    model="PleIAs/Monad",
    ...
)
trainer.processing_class.model_input_names.pop("token_type_ids")
trainer.train()

[d0rbu]

Got it, thanks for the feedback!