Handling of too many queries at once
Limit from Azure: 150K tokens per minute ~ 900 requests per minute (estimate from Azure)
Queue for API - prioritized based on rate limit restrictions.
Limit from Azure: 150K tokens per minute ~ 900 requests per minute (estimate from Azure)
Queue for API - prioritized based on rate limit restrictions.