Bug: Retrying inappropriate errors

[nissa-seru]

A rate limit error from Roo typically results from:

• A huge tool call output; or
• Provided API key lacks sufficient capacity to operate Roo.

In either branch, we shouldn't retry - I would recommend only retrying for connection errors.

Example of the failure mode:

API request:

429 {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}

Retrying now...

Originally posted by @veerababumanyam in #554

[mrubens]

Makes a lot of sense. Any ideas on the best way to differentiate?

[mrubens]

I think some 429 errors tell you when to try again - that would be great to consider as well

[nissa-seru]

export class BadRequestError extends APIError<400, Headers> {}

export class AuthenticationError extends APIError<401, Headers> {}

export class PermissionDeniedError extends APIError<403, Headers> {}

export class NotFoundError extends APIError<404, Headers> {}

export class ConflictError extends APIError<409, Headers> {}

export class UnprocessableEntityError extends APIError<422, Headers> {}

export class RateLimitError extends APIError<429, Headers> {}

Above is example from Anthropic - ideally other providers would follow similar conventions regarding error codes. 80/20 is likely to base retry strategy on error code and iterate from there. Can use exponential retry + jitter to attempt retry of rate limit, but tbh, I think that may be worse UX (in case of rate limit specifically) than just not retrying - rate limit issue with Roo is usually bc API key lacks capacity that Roo needs, or something is going wrong (like a massive tool call result or similar) that should not be retried.