Logit Bias

🐶 What is it?

Would you like to exclude certain words or filter specific tokens from your generated text?

Logit biases can be used to promote or suppress the generation of specific tokens. This is accomplished by adding a bias term to each token's respective logits. Where a positive bias increases the generation probability, a negative bias decreases it.

Note that logit bias operates at the token level, so you must refer to valid tokens in the Jurassic-2 vocabulary, otherwise the API returns an error. Watch out for whitespaces, which are replaced with a special underscore character in our string representation of tokens (see here).