Fail Loudly on a Missing Credential

That throw deserves its own section, because skipping it costs a genuinely miserable afternoon.

If the credential is missing and nothing checks, the request goes out with an empty key. The provider rejects it with an authentication error whose message says, in effect, you did not provide an API key.

Which is true, and completely misleading, because the code that reads the key ran fine and the row exists. You will go looking for a revoked key, a wrong environment, a proxy stripping headers. The actual answer is that a credential resolved to an empty string and nobody checked.

So the exception names the tenant and the place to look:

namespace App\Exceptions;

use App\Models\Workspace;
use RuntimeException;

final class MissingAiCredentials extends RuntimeException
{
    public static function for(Workspace $workspace): self
    {
        return new self(sprintf(
            'Workspace %s has no AI credential. '
                .'Set one with `php artisan '
                .'credentials:set %s ai`.',
            $workspace->id,
            $workspace->id,
        ));
    }
}

And the API renders it once, in bootstrap/app.php, like every other failure that is not the caller’s fault:

// bootstrap/app.php, inside withExceptions()
$exceptions->render(function (MissingAiCredentials $e) {
    report($e);

    return response()->json([
        'message' => __('errors.ai_not_configured'),
        'code' => ErrorCode::AiNotConfigured,
    ], Response::HTTP_FAILED_DEPENDENCY);
});

424 Failed Dependency says what is true: the request was fine, and something this service depends on is not set up. The message the caller sees is a sentence you wrote. The detailed one, with the workspace id and the command to fix it, goes to the log, because that is where the next person will look, and the next person is you in four months.

This generalises. A misconfiguration should fail where it is configured, not where it is used. The further those two points drift apart, the more expensive the bug.

Retry What Is Worth Retrying

Retries are where good intentions produce bad systems. The default instinct, “retry three times on failure”, quietly turns one malformed request into three, and one billing error into three.

The policy that survives contact with a real provider distinguishes between failures that might resolve themselves and failures that definitely will not, and a model adds one category that ordinary APIs do not have.

Rate limits with a short Retry-After are worth one wait. The provider has told you exactly when to come back. The SDK does this for you: it retries a 429 or an overloaded response once, if the header asks for ten seconds or less.

Rate limits without a Retry-After are not. Some providers return a 429 with no header when you have hit a spending cap, and it keeps failing until somebody raises the cap. Retrying it is a loop that waits for a human.

4xx errors are never retried. A malformed request will be malformed on the second attempt too. Retrying it wastes latency and rate limit, and hides the bug: an error you retry three times is an error you notice three times more slowly.

A timeout is never retried. This is the one that is specific to models, and we learned it from the invoice. When a request times out on your side, the provider has usually not stopped. It is still generating the answer, and it will bill you for it. Retry, and you pay for a second answer while the first is still being written. Our first policy retried every connection failure, timeouts included. We changed it in October so that one request can bill one completion, and no more. Now the only connection failures worth retrying are the ones where the request provably never left the machine: a DNS failure, a refused connection.

The SDK’s built-in behaviour already follows these rules. The mistake to avoid is wrapping it in a retry of your own:

// Don't: this retries timeouts, and every retry is a
// second bill.
$response = retry(
    3,
    fn () => (new SiteCopywriter($workspace))
        ->prompt($brief),
);

If you need resilience beyond one short wait, the SDK’s answer is failover, and it is the right one:

$response = (new SiteCopywriter($workspace))
    ->prompt($brief, provider: [
        $providers->for($workspace, Lane::Copy),
        $providers->fallbackFor($workspace, Lane::Copy),
    ]);

Failover moves to the next provider on a rate limit, an overloaded provider, exhausted credit or a failed connection. Ordinary errors, like a bad request, fail where they happened. That is mostly the same distinction as above, applied across vendors instead of across attempts, with one catch worth knowing: to the SDK, a timeout is a failed connection. So a request that timed out on the first provider, which may still be generating and billing, is sent again to the second. For a cheap lane that is an acceptable price for staying up. For an expensive one, leave failover off and let the timeout fail. Whether you want a second provider at all is a product decision with a cost, and Chapter 18 weighs it against simpler options.