# JetStream health

The previous page showed you the monitoring port and named `/jsz` as the JetStream endpoint. That endpoint counts streams and consumers, but a count doesn't tell you whether the `shipping` consumer is keeping up. To answer that you read *state*, the live numbers JetStream keeps for the stream and for each consumer, and you turn a few of them into one number: lag.

This page introduces two ideas. The first is **stream and consumer state**: what the server reports about an `ORDERS` stream and its `shipping` consumer right now. The second is **lag**: a single number, computed from two of those fields, that says how far behind the consumer has fallen.

This page reads state. It doesn't re-explain what a stream, a consumer, or an ack is; the [JetStream deep dive](/learn/jetstream/.md) owns those definitions, and this page assumes them.

**Entering:** the `east` cluster is up, the `ORDERS` stream has been taking orders, and the `shipping` consumer has fallen behind. You can hit `/jsz` on any node. Now you want the numbers underneath it.

## Stream state: what is stored right now

Every stream keeps a small block of **state** separate from its configuration. The configuration is what you asked for when you created the stream. The state is what's actually in it at this moment, and it changes with every published order.

Read it with `nats stream info`, or fetch the same fields from any client:

#### CLI

```
#!/bin/bash

# Read the live state of the ORDERS stream.

# LastSeq is the sequence of the newest stored order; you need it to

# compute how far behind any consumer is.

nats stream info ORDERS



# The same state as JSON, narrowed to the fields that matter for lag.

nats stream info ORDERS --json | \

  jq '.state | {messages, last_seq, first_seq, num_subjects}'
```

The output has a configuration half and a state half. Only the state half moves. For the `ORDERS` stream it looks like this:

```
State:



             Messages: 1,000

                Bytes: 412 KiB

        First Sequence: 1

         Last Sequence: 1,000

          Num Subjects: 4

        Active Consumers: 2
```

Three of these fields matter most here.

**Messages** (`messages`) is how many orders the stream holds. **Last Sequence** (`last_seq`) is the sequence number assigned to the newest order, `1000` in our snapshot. Every published order gets the next sequence, so `last_seq` only ever climbs. You'll use it in a moment to measure how far any consumer is behind. **Num Subjects** (`num_subjects`) is how many distinct subjects the stream has seen: `orders.created`, `orders.shipped`, `orders.cancelled`, and one regional subject make four.

The full set of stream state fields is documented in [Reference → jsz](/reference/system/monitor/jsz.md) and [Reference → stream API](/reference/jetstream/api/stream/.md). We only need `last_seq` here.

## Consumer state: position and gap

A consumer keeps its own state. Where the stream state describes what's *stored*, the consumer state describes how far one reader has *gotten*. Read it with `nats consumer info`, or with `consumer_info` from any client:

#### CLI

```
#!/bin/bash

# Read the live state of the shipping consumer on the ORDERS stream.

# The state numbers tell you lag, in-flight, and redelivery at a glance.

nats consumer info ORDERS shipping



# For a machine-readable copy of the same fields, ask for JSON and pick

# out the ones that describe how far behind the consumer is.

nats consumer info ORDERS shipping --json | \

  jq '{num_pending, num_ack_pending, num_redelivered, delivered, ack_floor}'
```

For the `shipping` consumer the state half reads:

```
State:



  Last Delivered Message: Consumer sequence: 980  Stream sequence: 980

        Acknowledgment Floor: Consumer sequence: 975  Stream sequence: 975

        Outstanding Acks: 5 out of maximum 1,000

            Redelivered Messages: 3

            Unprocessed Messages: 20

                  Waiting Pulls: 0 of maximum 512
```

Four fields describe the consumer's progress, and each one has a precise meaning.

**Last Delivered Message** carries `delivered.stream_seq`: the stream sequence of the last order the server handed to a `shipping` worker. In the snapshot it's `980`. The stream's `last_seq` is `1000`. The gap between them is what this page is about.

**Acknowledgment Floor** (`ack_floor.stream_seq`) is the sequence below which every order has been acked. Orders `981` through `1000` haven't been delivered yet; orders `976` through `980` were delivered but aren't all acked. That second group is the next field.

**Outstanding Acks** counts orders delivered but not yet acked, also called **in-flight** (`num_ack_pending`). Five `shipping` workers are holding five orders, working on them, and haven't confirmed them back. In-flight measures work in progress, which is a different thing from lag.

**Redelivered Messages** (`num_redelivered`) counts how many orders the server has had to deliver more than once since the consumer was created: three so far. A redelivery happens when a worker takes an order and never acks it, so the server hands it to someone else.

The full set of consumer state fields is documented in [Reference → consumer API](/reference/jetstream/api/consumer/.md). We only need the four above.

## Lag is a number you compute

The most important health signal for a consumer is **lag**: the number of orders the stream holds that haven't been delivered to this consumer yet. It's the gap you saw a moment ago, written as one number:

```
lag = stream.last_seq − consumer.delivered.stream_seq
```

For the pinned snapshot:

```
lag = 1000 − 980 = 20
```

Twenty orders are waiting for a `shipping` worker to fetch them. That's what "the shipping consumer is behind" means as a number. A pull consumer also reports this same value directly as **Unprocessed Messages** (`num_pending`), so most of the time you read it off `nats consumer info` rather than subtracting by hand. You compute it yourself when you want to cross-check that the reported number is fresh, a point the Pitfalls return to.

Three numbers now describe the full state of the `shipping` consumer:

* **lag = 20**: orders waiting, never delivered (`num_pending`)
* **in-flight = 5**: orders delivered, not yet acked (`num_ack_pending`)
* **redelivered = 3**: orders the server had to hand out again (`num_redelivered`)

A healthy consumer keeps lag near zero and redelivery flat. Lag climbing while `last_seq` climbs means orders arrive faster than `shipping` processes them. Redelivery climbing means workers are taking orders and failing to ack them. Each symptom has a different cause, and naming which number moved is the first step every time.

This animation shows the three numbers as positions on the stream log: orders append and push `last_seq` up, the consumer cursor advances as workers fetch and ack, the gap between them is lag, and a failed order pulses back as a redelivery.

**Message flow — consumerLagAnimated:** Interactive NATS flow diagram.

* order-svc → ORDERS LastSeq ${lastSeq}
* ORDERS LastSeq ${lastSeq} → shipping Delivered ${delivered}
* ORDERS LastSeq ${lastSeq} → worker

Watching lag is the task here. Deciding what to do about a lag that won't come down (adding `shipping` workers, or sizing the deployment) belongs to the [worker pool](/learn/jetstream/worker-pool.md) page and the [deployment](/learn/deployment/.md) chapter. This page names the symptom, and the fix is covered there.

## Pitfalls

These traps are scoped to this page's two concepts: reading consumer state, and reading lag from it. Each one is a number that misleads if you read it without context.

**Pull-consumer lag is only fresh when a client is fetching.** A pull consumer reports `num_pending` based on what it knew the last time a worker asked for messages. If every `shipping` worker has crashed, no one is fetching, and the number can sit stale at its last value while orders keep arriving. Don't trust a flat `num_pending` as proof the consumer is healthy; cross-check it. Compute lag yourself from the two source fields and compare:

#### CLI

```
#!/bin/bash

# Read the live state of the shipping consumer on the ORDERS stream.

# The state numbers tell you lag, in-flight, and redelivery at a glance.

nats consumer info ORDERS shipping



# For a machine-readable copy of the same fields, ask for JSON and pick

# out the ones that describe how far behind the consumer is.

nats consumer info ORDERS shipping --json | \

  jq '{num_pending, num_ack_pending, num_redelivered, delivered, ack_floor}'
```

Read `stream.last_seq` and `consumer.delivered.stream_seq`, subtract, and if your computed lag is large while `num_pending` looks small, the reported number is stale and the workers are gone.

**In-flight is not lag.** `num_ack_pending` counts orders a worker already holds; `num_pending` counts orders no worker has taken yet. Reading one for the other hides a real problem: a handler that fetches orders and then hangs shows a steady, non-zero in-flight count while lag quietly grows behind it. Don't collapse the two into one "behind" number. Read both, every time: rising in-flight means a stuck handler, rising lag means not enough handlers.

**A filtered consumer's lag counts only its subjects.** The `analytics` consumer filters `orders.shipped`. Its `num_pending` counts only the shipped orders it hasn't seen, not the whole stream. An empty pending on a filtered consumer doesn't mean the `ORDERS` stream is empty; it means nothing on *that filter* is waiting. Don't read a filtered consumer's lag as a stream-wide number. Compare it against the stream's per-subject counts, not against `last_seq`.

## Where you are

You can now read the live state of the `ORDERS` stream and its `shipping` consumer, and you can turn it into the three numbers that matter:

* **lag = 20**: orders waiting, from `last_seq − delivered.stream_seq`, also reported as `num_pending`
* **in-flight = 5**: orders delivered but not acked (`num_ack_pending`)
* **redelivered = 3**: orders handed out more than once (`num_redelivered`)

You know which field is which, and you know lag and in-flight are different measurements that move for different reasons.

## What's next

State is something you have to ask for: you poll `/jsz` or run `nats consumer info` and read the answer. Some events never show up in a poll. When a poison order exhausts its deliveries, the server announces it once, on a subject, and if you're not listening you never learn it happened.

The next page covers subscribing to those announcements.

Continue to [Advisories & events](/learn/monitoring/advisories-and-events.md).

## See also

* [Reference → consumer API](/reference/jetstream/api/consumer/.md) — every consumer state field and its type
* [Worker pool](/learn/jetstream/worker-pool.md) — how a pool of `shipping` workers shares the lag
* [Monitoring endpoints](/learn/monitoring/monitoring-endpoints.md) — the `/jsz` endpoint these numbers come from