A Time Budget for Every Request

Caching makes the list fast. It does nothing for a license being created, and nothing for the moment the cache is empty. In those cases the request waits for the provider, and how long it is allowed to wait is a decision you have to make, because the default is far too long.

A PHP application serves requests from a fixed pool of workers. Say you have twenty. Each request holds one until it finishes. Now the provider slows down and every call takes thirty seconds. Within moments all twenty workers are sitting in a call to Statamic, and the twenty-first request, a consumer asking for nothing more than the cached list, waits in a line that isn’t moving.

That is how a slow dependency becomes a total outage. The provider being slow didn’t take your API down. Your willingness to wait did.

That is why Chapter 3’s client set two limits and didn’t explain them:

->connectTimeout(3)
->timeout(10)

connectTimeout is how long to wait for the connection to open. A healthy server accepts in milliseconds, so three seconds is generous, and a provider that is down fails fast. timeout is the whole exchange. Ten seconds is long for an API call. Choose it from the measurements: if the provider’s slowest normal answer is two seconds, a ten-second ceiling only ever cuts off calls that were already lost.

Then add up the worst case, from the outside in. Three attempts of ten seconds, plus two waits of up to five seconds between them, is forty seconds for one provider call. Creating a license can make two, the list and the create, which is eighty. Your web server has to allow a request longer than that, or it kills the request while the last attempt is still running and the consumer gets a bare gateway error where your clean 504 should have been. Chapter 8’s idempotency lock has to outlive it too, which is why that lock is held for two minutes. And the consumer’s own timeout has to be longer than all of it. Every layer waits a little longer than the one inside it.

Stop Calling What Is Down

Timeouts limit how long each request waits. They don’t stop the next request from waiting just as long. While the provider is down, every create request still spends its full budget finding that out again.

A circuit breaker fixes that. After a number of failures in a row, stop calling for a while and fail immediately. Laravel’s rate limiter, which Chapter 6 used to count requests, counts failures just as well:

// app/Services/License/StatamicDriver.php
private const FAILURES = 'provider-failures:statamic';

private function guarded(Closure $call): ClientResponse
{
    if (RateLimiter::tooManyAttempts(self::FAILURES, 5)) {
        throw new ProviderUnavailable;
    }

    try {
        $response = $call();
    } catch (ConnectionException $e) {
        RateLimiter::hit(self::FAILURES, decaySeconds: 60);

        throw $e;
    }

    RateLimiter::clear(self::FAILURES);

    return $response;
}

ClientResponse is the HTTP client’s response class, imported under that name to keep it apart from the response a controller returns. Each provider call in the driver is wrapped in guarded(). This is delete() as it ends up:

public function delete(string $key): void
{
    $response = $this->guarded(fn () => $this->http()
        ->withUrlParameters(['key' => $key])
        ->delete('/sites/{key}'));

    Cache::forget($this->cacheKey());

    if (! $response->notFound()) {
        $response->throw();
    }
}

fetchAll() wraps its get() the same way. create() needs a little more care, because the wrapped call sits inside its try:

public function create(
    string $name,
    string $domain,
): License {
    try {
        $data = $this->guarded(fn () => $this->http()
            ->post('/sites', [
                'name' => $name,
                'domain' => $domain,
            ]))->throw()->json('data');
    } finally {
        Cache::forget($this->cacheKey());
    }

    return License::fromProvider((array) $data);
}

Five connection failures within a minute, and for the rest of that minute the driver throws without touching the network. ProviderUnavailable is an exception of the application’s own that renders itself as the same 504, with the same code, that Chapter 7 gave a timeout. A consumer gets that answer in a millisecond, your workers stay free, and the provider isn’t hammered while it tries to recover. When the minute is up the counter has expired, the next call goes through, and one success clears the slate.

Only ConnectionException counts. A 422 from the provider means the provider is up and disagreed with you, which is no reason to stop calling it.

This is twenty lines of protection, not a resilience framework. It doesn’t track half-open states or success ratios. For one provider behind one API it is enough, and you will know when it isn’t, because Chapter 13’s dashboard will show you.

The driver has now been built across three chapters. This is all of it:

StatamicDriver implements LicenseContract
  __construct(string $token) Ch. 3
  for(Consumer): self        Ch. 4   one driver per consumer
  all(): Collection          Ch. 9   cached, then typed
  create(name, domain)       Ch. 9   always forgets the list
  delete(key)                Ch. 9   a 404 is success
  fetchAll(): array          Ch. 9   the GET, checked
  http(): PendingRequest     Ch. 3   the macro, retrying a 429
  backoff(), wasThrottled()  Ch. 3   when to retry
  guarded(Closure)           Ch. 9   the circuit breaker
  cacheKey(): string         Ch. 9   one list per account

Knowing the Optimization Worked

This closes the loop the chapter opened. You measured before you touched anything. Ship the fix, then look at the same number.

If it didn’t move, one of three things is true: you optimized the wrong thing, the bottleneck was never where you thought it was, or the fix didn’t actually ship. All three are better to know than “I added caching, it should be faster now.”

For queries, you can pin the result in a test so that it can’t regress unnoticed:

it('lists consumers in a fixed query count', function () {
    Consumer::factory()->count(20)->create();

    Sanctum::actingAs(
        Consumer::factory()->create(),
        ['consumers:manage'],
    );

    $this->expectsDatabaseQueryCount(3);

    $this->getJson('/api/consumers')->assertOk();
});

Twenty consumers or two thousand, the count stays at three: one to count the rows, one to fetch the page, and one to record the request. The day someone adds a lazy relationship to the Resource, this test tells them. (Laravel documentation: Database Testing.)

Chapter 9 Summary

Before you touch anything:

• Confirm the slow request in your measurements. Don’t work from a feeling.
• Identify whether the cost is the database, an external call, or the payload. Return the fields a consumer uses, and no others.

Queries:

• Batch or eager load any query that would run once per row of a loop.
• Turn on shouldBeStrict() outside production.
• Bound the page size, on both sides, on any endpoint where a client chooses it. Use cursorPaginate() when nobody needs the total.
• Every index matches a real WHERE clause, in the column order the query filters on.

Caching:

• Cache what is expensive, usually network calls, not lookups that are already fast.
• Cache plain arrays, and build objects after the read.
• Every cache key includes whatever makes the data different for different callers.
• Every write forgets the key it invalidates, by name, where the write happens, whether or not the write succeeded.

Waiting on a provider:

• Give every outgoing call a connect timeout and a total timeout, and add up the worst case across retries.
• Stop calling a provider that is down. A counter in the rate limiter is enough.

After you ship:

• Confirm the number moved. If it didn’t, find out why before you move on.