Logit Bias

🐶 What is it?

Logit biases can be used to promote or suppress the generation of specific tokens. This is accomplished by adding a bias term to each token's respective logits. Where a positive bias increases generation probability, a negative bias decreases it.

Note that logit bias operates at the token level, so you must refer to valid tokens in the Jurassic-1 vocabulary, otherwise the API returns an error. Watch out for whitespaces, which are replaced with a special underscore character in our string representation of tokens (see here).


The following example introduces a large negative bias to avoid generating the expected continuation "a box of":