Logit Bias
🐶 What is it?
Logit biases can be used to promote or suppress the generation of specific tokens. This is accomplished by adding a bias term to each token's respective logits. Where a positive bias increases generation probability, a negative bias decreases it.
Note that logit bias operates at the token level, so you must refer to valid tokens in the Jurassic-1 vocabulary, otherwise the API returns an error. Watch out for whitespaces, which are replaced with a special underscore character in our string representation of tokens (see here).
Example
The following example introduces a large negative bias to avoid generating the expected continuation "a box of":
🐶
Logit bias example
Open Recipe
Updated 16 days ago