Private AI
Troubleshooting & Performance Optimization
Resolve common issues and optimize performance for AI21’s Jamba model deployments
Overview
This guide helps you troubleshoot common deployment issues and optimize performance for AI21’s Jamba models across different deployment scenarios.
Before troubleshooting, ensure you’re using the recommended vLLM version v0.6.5
to v0.8.5.post1
.
Troubleshooting
Memory Issues
Model Loading Issues
Performance Issues
Getting Help
Need Support?
If you need support, please contact our team at support@ai21.com with the following information:
Environment Details:
- Hardware specifications (GPU model, memory, CPU)
- Software versions (vLLM, CUDA, drivers)
- Full vLLM command
Diagnostics:
- Full error messages and stack traces
- GPU utilization logs (
nvidia-smi
output)