How do I include a stop sequence?

[amrezo]

Hello, I am creating a chatbot with this API. I was wondering how I can add a stop sequence so it doesn't keep on generating text after the response.

Thank you very much!

[TheProtaganist]

In a way the the "user" parameter is the stop token. I had to create it manually because the server I'm using for it provided by vicgalle doesn't have a stop token. What you can do is define the user and bot parameters. The user is the name of who the bot is speaking to, the bot is the imaginary person or thing you are planning to speak to. I highly recommend you don't use periods in the examples. It causes the model to continue when it wasn't supposed to... Just make sure in the examples the user prompts are on the left and the bot examples are on the right

[TheProtaganist]

The server sometimes goes down. If the API doesn't seem to respond it's probably because the website got updated or something. Is ok they will have it fixed soon.

[TheProtaganist]

The website is up again. If you don't mind can you tell me what kinda chatbot your gonna build 🙂. Is it for business, personal, or something else?

[TheProtaganist]

Good news stop token was just added to the server! I will implement it and let you know.

[Simplicitylinux]

I'd be interested in being able to define a stop sequence too. If you've implemented it, could you let me know?

[bensonchow123]

Me too, i wish stop sequences are implemented! It can finally become an alternative to gpt-3, if stop sequence are implemented!

[vicgalle]

@Simplicitylinux @bensonchow123 If it is of interest, I have already implemented custom stop sequences at https://github.com/vicgalle/gpt-j-api