# Request-Reply Resilience So far this chapter has hardened the connection underneath one-way traffic: `order-svc` publishes orders, and the subscribers receive them. But `order-svc` also asks questions. Before it confirms an order it sends a request on `orders.inventory.check` and waits for the `inventory` responder to answer. A fault on that call has the biggest impact, because the application is blocked on the reply. Core NATS already taught you the mechanics of request-reply: the client sends a request, the responder answers on a private `_INBOX` subject, and the reply finds its way back. This page assumes that machinery and covers what happens when the reply doesn't come. A request can fail in two different ways, and the two mean different things, so the client must tell them apart before it retries. This page introduces two new ideas: the request timeout and the no-responders signal, and retry with backoff plus idempotency. We'll define each before we use it. ## A request has three outcomes A bare `request` from Core NATS takes a subject, a payload, and a **timeout**: the deadline by which a reply must arrive. Wait on it and exactly one of three things happens. The first is the happy path: the `inventory` responder answers, and the reply comes back inside the deadline. The application reads it and moves on. The second is a timeout. The deadline passes with no reply. A timeout doesn't mean the responder is gone, only that no answer arrived *in time*. The `inventory` responder may be up but slow, for example because of a long database query, a GC pause, or a burst of load. The request was delivered to a live listener; the reply just didn't come back fast enough. The third is **no responders**. The moment the client sends the request, the server already knows whether any subscription is listening on `orders.inventory.check`. If none is, the server sends back an immediate no-responders signal (a 503 status with no body) and the request call returns at once instead of waiting out the timeout. No responder means the `inventory` service isn't running, isn't registered, or its account can't see the subject. Rather than a slow answer, this means there is no one to answer. Here's `order-svc` asking the inventory question with a timeout set. The CLI prints the reply on success, a timeout message if the deadline passes, and the 503 immediately if nobody is listening: #### CLI ``` #!/bin/bash # order-svc asks the inventory question and waits for a reply, with a # timeout that bounds how long it blocks. # # A request has exactly three outcomes: # - a reply arrives inside the deadline -> printed below # - the deadline passes with no reply -> a timeout # - no subscription is listening -> an immediate 503 # (no responders) # # --timeout sets the deadline. Keep it at two or three times the # responder's p99 latency so a slow-but-healthy answer is not mistaken # for a fault. The client libraries surface the same three outcomes as a # reply, a timeout error, and a no-responders error -- see the other tabs. nats request orders.inventory.check \ '{"order_id":"ord_8w2k","customer":"acme-co","total_cents":4200,"ts":"2026-05-22T10:14:22Z"}' \ --timeout 2s ``` The request carries the same canonical order shape used everywhere in this chapter: ``` {"order_id":"ord_8w2k","customer":"acme-co","total_cents":4200,"ts":"2026-05-22T10:14:22Z"} ``` The full set of request options is documented in [Reference](/reference/.md). Here we cover only the ones that change how a request behaves under fault. ## Why no-responders is useful You might treat a missing answer as just a missing answer and stop there. But the no-responders signal is the most useful failure NATS gives you, because it's fast and certain. Without it, a request to a subject nobody listens on would sit until the timeout expired: two seconds of the application blocked, without learning anything it didn't already know. With it, the client finds out in a single round trip that there's no `inventory` responder at all. That lets the two failures drive different behavior. A timeout means someone is there, so the client should try again soon. No responders means nobody is there yet, so the client should give them a moment to start. The retry strategy in the next section turns that distinction into code. The animation below shows the difference. One request times out, waits, retries, and finally gets a reply; the no-responders path returns the instant it's sent: **Message flow — requestRetryAnimated:** Interactive NATS flow diagram. * order-svc → server * server → inventory * inventory → server (subject: reply) * server → order-svc * order-svc → server (subject: timeout — no reply) * server → order-svc (subject: 503 no responders) * order-svc → order-svc (subject: backoff…) The no-responders signal needs a server new enough to send the 503 and a client that advertised support for it during the connect handshake. Both have been the default for years; you get it for free on any current setup. You can see it yourself by requesting a subject nobody answers: #### CLI ``` #!/bin/bash # See the no-responders signal for yourself. # # Request a subject that nobody is listening on. With no inventory # responder running, the server knows immediately that no subscription # exists on orders.inventory.check, and returns a 503 no-responders # signal at once -- the request does NOT wait out the --timeout. # # Run this with no `nats reply orders.inventory.check ...` running, and # the CLI prints the no-responders error right away. Start a responder # and the same request gets an answer instead. The client libraries # surface this as a distinct no-responders error you can branch on, # separate from a timeout -- see the other tabs. nats request orders.inventory.check \ '{"order_id":"ord_8w2k","customer":"acme-co","total_cents":4200,"ts":"2026-05-22T10:14:22Z"}' \ --timeout 2s ``` ## Retry differently per failure Knowing the two failures apart is only useful if the client acts differently on each. That's **retry with backoff**: re-sending a failed request, with a growing wait between attempts so an overloaded responder gets time to recover instead of receiving more requests. The two failures call for different timing. A timeout means the responder is up but slow, so a fast retry is reasonable; it may answer on the second try. No responders means nothing is listening, so an immediate retry is wasted; the responder is likely starting up, and the client should back off with a growing wait to give it time to register. Use a short, bounded retry for a timeout and an exponential backoff for no responders. Either way the retry loop must be **bounded**. An unbounded retry against a responder that never comes back is a busy loop that never ends. Cap the attempts (five is a reasonable default) and add jitter to the wait so a fleet of requesters doesn't retry in lockstep and overwhelm the responder the instant it returns. There's one more thing a retry needs to be safe, and it's the second concept of this page: **idempotency**. A retried request is a *duplicate* request. If the first attempt actually reached the `inventory` responder and only the reply was lost, the retry asks the same question a second time. If asking twice causes two effects (two stock reservations, two charges), the retry has corrupted state. The fix is to make a duplicate a no-op. `order-svc` keys every inventory check by its `order_id` (`ord_8w2k` here), and the `inventory` responder remembers recently seen IDs. A request whose `order_id` it has already answered returns the cached reply instead of reserving stock again. The ID is already in the payload, so making the call idempotent costs nothing on the wire; it's a discipline on both ends, not a new message. A request in flight when the connection drops is lost outright; NATS doesn't persist it. The inbox re-subscribes automatically on reconnect (the reconnection page covered that), so a retry after the link returns works normally. And because the retry is idempotent, replaying it is safe even if the original had reached the responder before the drop. ## Pitfalls A few traps turn request resilience into either a hang or corrupted state. Each is scoped to this page's two ideas: the two failures, and safe retry. **Treating a timeout and no-responders as the same failure.** They are not the same. A blind "retry on any error" backs off the same way for both, which is wrong in both directions: it wastes a fast-retry opportunity on a slow responder and it floods a subject that has no responder at all with repeated requests. Branch on the error (fast-retry a timeout, exponential-backoff a no-responders) so the client matches its behavior to what actually went wrong. **A timeout shorter than the responder's real latency.** If you set the timeout below what the `inventory` responder needs at its p99, every busy moment looks like a failure and the client retries answers that were already on their way back. Measure the responder's latency and set the timeout to two or three times its p99, so a slow-but-healthy answer isn't mistaken for a fault. **An unbounded retry loop.** Retrying forever against a responder that never returns is a busy loop that pins a CPU and never reports the problem. Always cap the attempts, add jitter, and stop with a clear signal (log the failure and let the caller decide) rather than retrying silently. **A non-idempotent retry that double-acts.** If the first attempt reached the `inventory` responder and only the reply was lost, a naive retry reserves stock twice. Key every request by `order_id` and have the responder de-dupe on it, so replaying a request is always safe. Here's the safe pattern: branch on the failure, bound the retries, and key the request by `order_id` so a duplicate is a no-op on the responder side: #### CLI ``` #!/bin/bash # The safe retry pattern: branch on the failure, bound the retries, and # key the request by order_id so a duplicate is a no-op on the responder. # # The CLI does not expose a per-failure backoff strategy the way the # client libraries do (see the other tabs for fast-retry-on-timeout vs # exponential-backoff-on-no-responders). Here we show the same shape with # a bounded loop and growing waits, and we keep the payload -- including # its order_id -- byte-identical on every attempt so the inventory # responder can de-dupe a re-sent request. PAYLOAD='{"order_id":"ord_8w2k","customer":"acme-co","total_cents":4200,"ts":"2026-05-22T10:14:22Z"}' MAX_ATTEMPTS=5 attempt=1 while [ "$attempt" -le "$MAX_ATTEMPTS" ]; do # Same order_id on every attempt -> the responder treats a retry as the # same question and returns the cached answer instead of acting twice. if nats request orders.inventory.check "$PAYLOAD" --timeout 2s; then echo "inventory check answered on attempt $attempt" exit 0 fi # Grow the wait, with a little jitter, so a fleet of requesters does not # retry in lockstep and stampede the responder the instant it returns. wait=$(( attempt * attempt )) jitter=$(( RANDOM % 2 )) echo "attempt $attempt failed; backing off ${wait}s before retry" >&2 sleep "$(( wait + jitter ))" attempt=$(( attempt + 1 )) done echo "inventory check failed after $MAX_ATTEMPTS attempts; giving up" >&2 exit 1 ``` ## Where you are `order-svc`'s inventory check now distinguishes "responder absent" from "responder slow" and retries each correctly. You have: * A request call that tells its three outcomes apart: a reply, a timeout, or an immediate no-responders signal. * A retry that fast-retries a timeout, backs off on no-responders, and is bounded with jitter so it never busy-loops or overwhelms the responder. * Idempotent requests keyed by `order_id`, so a retry after a lost reply or a reconnect never double-acts. The connection now survives faults in every direction it sends and receives traffic. But it still does everything in the clear: the bytes on the wire are readable, and the server takes whatever name the client offers. ## What's next The last mechanism in this chapter is **TLS and auth**: pointing the client at the `order-svc` credentials file so it authenticates, and at the cluster CA so it validates the server over an encrypted link. Continue to [TLS & Auth](/learn/resilient-clients/tls-and-auth.md). ## See also * [Core NATS → Request-Reply](/learn/core-nats/request-reply.md) — the `_INBOX` mechanism this page assumes. * [Services](/learn/services/.md) — the framework that builds discovery, endpoints, and metrics on top of raw request-reply. * [Reference](/reference/.md) — the full set of request and timeout options.