Building a Delphi API Client That Doesn’t Fall Apart When the API Misbehaves

The first version of almost every API client I write in Delphi looks embarrassingly simple.

Send a request. Parse the JSON. Return the result.

Something like this:

uses
  Dext.Net.RestClient;

type
  TOrder = record
    Id: Integer;
    Customer: string;
    Total: Currency;
  end;

function GetOrder(Id: Integer): TOrder;
begin
  Result := RestClient('https://api.store.com')
    .Get<TOrder>('/orders/' + Id.ToString)
    .Await;
end;

For a quick test or a quick proof of concept, that is usually enough.

Then I deploy it and leave it running in a background service, a multi-tier worker, or a busy desktop app. Eventually, the remote connection hangs indefinitely, the API starts returning 500s or 503s, or I get hit with a 429 Too Many Requests because polling was a little too enthusiastic.

That is usually the point where the “simple client” stops being simple.

The interesting part of consuming APIs is not making the HTTP request. It is deciding which failures are worth retrying, which ones must fail immediately, and how to protect both your application and the remote service. That distinction matters far more than throwing a naive try..except retry loop around everything.

---

1. The First Thing I Add Is an Intentional Timeout

I used to treat timeouts as an afterthought or an optional configuration detail.

I do not anymore.

A request without an explicit, well-calibrated timeout can block a thread much longer than expected when the remote endpoint becomes unresponsive or gets stuck behind a saturated gateway. In a background task or high-throughput worker pool, threads accumulate, connection pools starve, and the entire application gradually grinds to a halt.

So even before thinking about retries or pipelines, I always ensure an explicit timeout is configured:

var
  Client: TRestClient;
begin
  Client := RestClient('https://api.store.com')
    .Timeout(10000); // 10,000 ms (10 seconds)

Ten seconds is not a magic universal constant. It depends on what the API is doing. For a lightweight market-data or quote endpoint, two seconds might already be plenty. For a bulky batch export or slow legacy ERP, thirty seconds might be required.

The important lesson is that the timeout must be deliberate and intentional, never left to operating system defaults.

---

2. Not Every Error Should Be Retried (Resilience ≠ Stubbornness)

This was probably the mistake I made most often when I first started writing API integrations in Delphi.

The naive version looks like this:

// DON'T DO THIS
for Attempt := 1 to 5 do
begin
  try
    Exit(MakeRequest());
  except
    Sleep(2000);
  end;
end;

It gives an illusion of robustness because the code “keeps trying.” In reality, it often makes bad situations much worse:

• If the server returns 401 Unauthorized or 403 Forbidden, retrying five times will not magically fix invalid credentials or an expired token.
• If the endpoint returns 404 Not Found, waiting two seconds and asking for the missing resource again is pointless.
• If the server returns 422 Unprocessable Entity or 400 Bad Request, the request itself is invalid. Retrying just spams the server with the exact same bad request.

A client should never confuse persistence with resilience.

The failures we usually consider temporary and eligible for retry are:

1. Network connectivity glitches and socket timeouts.
2. HTTP 429 (Too Many Requests).
3. Transient 5xx Server Errors (502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout).

Everything else deserves fail-fast behavior.

---

3. Why Jitter Matters (Avoiding the Thundering Herd)

When adding exponential backoff, calculating the delay as Delay := BaseDelay * Power(2, Attempt - 1) seems solid on paper.

However, imagine you have 30 background workers or hundreds of distributed POS terminals querying the same remote server. If a network hiccup or restart causes all of them to fail at roughly the same second, and all of them calculate identical mathematical backoff intervals:

• 1 second
• 2 seconds
• 4 seconds
• 8 seconds

They stay synchronized! Instead of easing pressure on the recovering server, they repeatedly hit it together in synchronized waves. This is the classic Thundering Herd Problem.

By introducing Jitter—a small, randomized variance—the retry spikes get smoothed out:

function CalculateDelayWithJitter(Attempt: Integer; BaseDelayMs: Integer): Integer;
var
  ExponentialDelay: Integer;
  Jitter: Integer;
begin
  ExponentialDelay := Trunc(BaseDelayMs * System.Math.Power(2, Attempt - 1));
  // Add a random jitter between 0 and 500ms
  Jitter := Random(500);
  Result := ExponentialDelay + Jitter;
end;

For one desktop app running on a single PC, this barely registers. For multiple threads, background workers, or distributed instances, it is the difference between a self-healing system and an unintentional self-inflicted denial of service.

---

4. Rate Limits (429) Need Respect

When an API responds with HTTP 429 Too Many Requests, it is explicitly asking you to slow down: “You are overwhelming me”. Sending the request again after a hardcoded 500ms sleep is not resilience; it is ignoring the server’s contract.

Well-designed APIs accompany a 429 status with a Retry-After header indicating how many seconds you must wait:

var
  RetryAfterSec: Integer;
  HeaderVal: string;
begin
  if Response.StatusCode = 429 then
  begin
    HeaderVal := Response.GetHeader('Retry-After');
    if (HeaderVal <> '') and TryStrToInt(HeaderVal, RetryAfterSec) then
      Sleep(RetryAfterSec * 1000)
    else
      Sleep(CalculateDelayWithJitter(Attempt, 1000));
  end;
end;

Respecting the server’s guidance prevents accounts from being banned, protects access tokens, and keeps traffic compliant with external SLAs.

---

5. Retrying Writes Is Dangerous: Respect Idempotency

Retrying GET queries is usually straightforward because safe reads have no side effects.

POST requests demand extreme caution. Suppose your client submits a credit card payment or creates a purchase order. The server processes the payment successfully, but the network drops right before the acknowledgment reaches your client.

From your client’s perspective, the request timed out and “failed.” If you blindly retry that POST, the customer may be charged twice.

Whenever retrying non-idempotent operations (POST, PATCH), employ Idempotency Keys (Idempotency-Key):

var
  IdempotencyKey: string;
  Response: IRestResponse;
begin
  // Keep the SAME key across retries of the same transaction
  IdempotencyKey := TGUID.NewGuid.ToString;

  Response := RestClient('https://api.payments.com')
    .Timeout(15000)
    .Header('Idempotency-Key', IdempotencyKey)
    .PostJson('/v1/charges', '{"amount": 150.00, "currency": "USD"}')
    .Await;

If the remote API supports idempotency keys, it recognizes that the second request is a duplicate attempt and safely returns the original response without executing the transaction twice.

---

6. Real-World Resilience with Dext: Pipelines and Circuit Breakers

In the Dext Framework, resilience is treated as a first-class citizen rather than an afterthought. Instead of writing boilerplate retry loops across your repositories and services, you can attach a Resilience Pipeline directly to TRestClient:

uses
  System.SysUtils,
  Dext.Net.RestClient,
  Dext.Resilience;

procedure PlaceSafeOrder;
var
  Pipeline: TResiliencePipeline;
  Client: TRestClient;
  Response: IRestResponse;
begin
  // Define an enterprise-grade resilience policy:
  // 1. Retry up to 3 times with exponential backoff
  // 2. Circuit Breaker: open circuit if 5 consecutive failures occur, resting for 30s
  Pipeline := TResiliencePipeline.Create
    .AddRetry(3, 200)               // Max 3 retries, base delay 200ms
    .AddCircuitBreaker(5, 30000);   // Stop calling if 5 failures occur in a row

  Client := RestClient('https://api.store.com')
    .Timeout(5000)
    .ResiliencePipeline(Pipeline.Instance);

  try
    Response := Client.Get('/health').Await;
    Writeln('Status: ', Response.StatusCode);
  except
    on E: ECircuitBrokenException do
      Writeln('Circuit is OPEN! Remote service is unhealthy. Fast-failing to preserve resources.');
    on E: Exception do
      Writeln('Operation permanently failed: ', E.Message);
  end;
end;

What Happens Here?

1. Connection Pooling: Dext automatically reuses sockets via its internal pooled engine, avoiding socket exhaustion under heavy load.
2. Exponential Backoff: Transient errors trigger retries with progressive backoff.
3. Circuit Breaker: If the remote service completely crashes or enters an outage, after 5 consecutive failures Dext’s Circuit Breaker trips into cbsOpen state. Any subsequent call fails instantly in memory without tying up threads or network sockets, giving the downstream service room to breathe and recover.

---

7. Useful Logging vs. Guesswork

When an automated script or multi-threaded service fails at 03:00 AM, a generic exception like Socket Error # 10054 or HTTP request failed leads to tedious guesswork.

At minimum, when inspecting failures, you need to know:

• The exact endpoint and HTTP verb.
• The HTTP status code received (if any).
• Which retry attempt failed.
• The total elapsed duration.
• Distributed tracing identifiers (Trace-Id / Span-Id).

Dext natively incorporates distributed tracing (OpenTelemetry) and structured logging into outbound requests:

// Outbound calls automatically create OpenTelemetry-compatible spans
// registering http.url, http.method, and http.status_code.
Response := RestClient('https://api.partner.com')
  .Header('X-Correlation-ID', CorrelationId)
  .Get('/catalog')
  .Await;

If the call fails, the tracer records the failure status and error message into your logging sink (Console, File, Seq, Prometheus, or OpenTelemetry APM).

---

8. Summary Checklist for Resilient Delphi Clients

Before shipping an API client into production, verify:

Aspect	Bad Habit	Resilient Approach
Timeouts	Unset / Default OS timeout	Explicit, intentional timeouts per use-case (.Timeout(ms))
Retry Strategy	Retry all exceptions in a loop	Retry only transient codes (429, 502, 503, 504) and timeouts
Backoff Timing	Fixed Sleep(1000)	Exponential backoff with random Jitter
Rate Limits	Immediate retry on 429	Parse and respect the Retry-After header
Write Operations	Blindly retrying POST requests	Use Idempotency-Key or restrict automatic retry on writes
Cascading Outages	Hammering a dead service	Use a Circuit Breaker to fail fast
Resource Usage	Creating new clients per call	Use connection pooling and reusable pipelines

---

Final Thoughts

Getting an HTTP 200 OK is the easy part of software development.

The real engineering begins when the remote service is unreachable, misconfigured, or struggling under load. The most dependable clients are not the ones that retry the most times. They are the ones that have a clear, disciplined strategy for failure:

They know when to wait. They know when to try again. And, just as importantly, they know when to stop.

---

Want to Learn More?

• Official Dext Repository: Explore the full ecosystem and contribute on GitHub (dotpas/dext).
• Official Dext Web Book: Master modern web API development, concurrency, microservices, and resilience in Professional Web Development with Delphi and Dext Framework.