← Learn / Notebooks

Models & toolsPractitionerStudy guide

Model Serving: Health Checks and Timeouts

Separate startup, readiness and liveness decisions, then inspect what a request timeout can and cannot guarantee.

Before you start: Basic services and requests. Python async familiarity helps with the optional timeout note.

AI-assisted study notebook · Published 14 Nov 2026. Source-support checks and synthetic examples are recorded; automated review is not independent expert fact verification.

In [1]: Architecture

1. Architecture and scope

A model process can exist before its weights and service dependencies are ready for requests. Kubernetes gives startup, readiness and liveness probes different roles. A configured startup probe gates the other two until it succeeds. Readiness affects service traffic; liveness can cause a container restart after the configured failures. [1] [2]

A request deadline answers a different question: how long the caller waits for an operation. Python’s asyncio.wait_for cancels the awaited task on timeout and waits for cancellation to complete, so elapsed time can exceed the timeout. A cancellation request is not proof that all underlying computation or external side effects have stopped. [3]

Scope. A toy control decision model based on Kubernetes probe roles. It does not operate a cluster or contact a model endpoint.

Workflow and scope notes
  1. Initialize service. Allow the application to initialize under its configured startup policy. The toy holds the other checks while startup has not succeeded.
  2. Evaluate signals. After startup, evaluate readiness and liveness independently. Readiness does not gate liveness in the actual probe design.
  3. Apply controls. Readiness controls eligibility for normal Service traffic; liveness failures can trigger restart. The toy uses a liveness failure threshold of three.
  4. Observe requests. Record timeout, cancellation completion and request outcome separately. Request timeout handling is separate from the probe state shown below.
In [2]: Experiment

2. Probe role decision model

Startup is configured. Before it succeeds, readiness and liveness are not evaluated. Afterward, vary readiness and a consecutive liveness-failure streak. The fictional restart threshold is three.

Probe role decision model

Original synthetic experiment · Change an input to update the figure.

This is an instantaneous decision model. It omits probe periods, startup failure budget, termination grace periods, restart policy and transient endpoint propagation. A restart decision does not mean traffic removal or restart has already completed.

Startup gate and independent probe lanesNot ready · Traffic eligible no · Restart no. Readiness and liveness are separate checks after startup succeeds.StartupSucceeded — other probes activeREADINESSSignal failsNormal Service traffic signalLIVENESS1 / 3 failuresRestart threshold in this toyDecision: Not readyTraffic eligible now: No · Restart decision: No
Not ready · Traffic eligible no · Restart no
Calculation record
{
  "inputs": {
    "started": "yes",
    "ready": "no",
    "failures": "1"
  },
  "route": false,
  "restart": false,
  "outcome": "Not ready"
}
CaseResultInterpretation
Startup has not succeededOther probes gatedThis model does not simulate exhausting the startup failure budget.
Started, readiness fails, liveness passesNot ready; no liveness restartA readiness failure alone is not the liveness restart condition.
Started, 3 liveness failuresRestart decisionThree is this example’s chosen threshold, not a universal default.
In [3]: Interpretation

3. Interpretation and limitations

The readiness result and liveness failure counter are drawn on separate lanes. At readiness No with zero liveness failures, the model is not eligible for normal Service traffic but has no liveness-triggered restart. At threshold, the restart decision takes priority in the summary; the diagram still preserves the separate readiness signal. [1] [2]

An HTTP success from a health endpoint says only what that endpoint checks. Editorial design questions include whether readiness should include successful model initialization and required dependencies, and whether a liveness check can mistake slow useful work for a deadlock. This notebook does not prescribe a single probe for every serving system. [1]

For Python 3.13, a wait_for timeout normally raises TimeoutError after requesting task cancellation. Cleanup can extend the wait. When an inference call delegates work to another process or remote service, confirm that system’s own cancellation behavior rather than assuming a client timeout stops it. [3]

Practice exercise

Set startup to No and change every other control. Explain why the model continues to gate those probes. Then enable startup, fail readiness and set zero liveness failures: describe the two independent outputs.

In [4]: Python

4. Worked Python example

The core calculation below uses only the Python standard library. The full download adds the verification assertions and input fixtures. It runs locally on synthetic data; it does not load a model or contact a service.

lab.py · Core example · Python 3.13
from itertools import product


def probes(started, ready, failures):
    if not started:
        return {"route": False, "restart": False,
                "outcome": "Startup gate"}
    restart = failures >= 3  # Chosen toy threshold, not a universal default.
    return {"route": ready and not restart, "restart": restart,
            "outcome": "Restart decision" if restart else
                       "Ready for traffic" if ready else "Not ready"}

Download full Python example and checks ↓

Run: python -I lab.py. Recorded run: Python 3.13; 20 fixture states plus scenario assertions passed. The JSON output is a reproducibility record, not a real-system benchmark.

In [5]: Knowledge check

5. Questions and explanations

Predict before opening the explanation. Answers refer to the stated scope and synthetic inputs.

Q1 · StartupWhen a configured startup probe has not succeeded, what happens to the other probes?
When a configured startup probe has not succeeded, what happens to the other probes?
Answer and explanation

Answer: They are gated. Kubernetes holds readiness and liveness checks until startup succeeds. [1]

Q2 · ReadinessDoes failing readiness alone mean a liveness restart?
Does failing readiness alone mean a liveness restart?
Answer and explanation

Answer: No. Readiness and liveness have different purposes and actions. [1] [2]

Q3 · DeadlineCan asyncio.wait_for take longer than its timeout?
Can asyncio.wait_for take longer than its timeout?
Answer and explanation

Answer: Yes, while cancellation completes. The documented function waits for the future to be cancelled. [3]

Q4 · ScopeDoes this page test a live model server?
Does this page test a live model server?
Answer and explanation

Answer: No. All signals are local teaching inputs; no endpoint is contacted.

In [6]: Sources

References and source links

  1. Kubernetes contributors. (Documentation accessed 2026). Liveness, Readiness, and Startup Probes | Kubernetes.https://kubernetes.io/docs/concepts/workloads/pods/probes/
  2. Kubernetes contributors. (Documentation accessed 2026). Configure Liveness, Readiness and Startup Probes | Kubernetes.https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
  3. Python Software Foundation. (Python 3.13.18 documentation). Coroutines and tasks — Python 3.13.18 documentation.https://docs.python.org/3.13/library/asyncio-task.html
Source checks and provenance

Primary publisher pages and relevant passages checked on 13 Nov 2026. Source support is not independent expert fact verification. Main documentation can change; retrieved-body hashes, dates, version labels and saved revision IDs are in the audit download.

[1] Liveness, Readiness, and Startup Probes | Kubernetes

Evidence: Startup probe; Liveness probe; Readiness probe. Role distinctions in current official documentation; concrete probe design is application-specific.

[2] Configure Liveness, Readiness and Startup Probes | Kubernetes

Evidence: Define readiness probes; probe configuration. Failure thresholds and lifecycle behavior; no deployment manifest is applied.

[3] Coroutines and tasks — Python 3.13.18 documentation

Evidence: asyncio.wait_for. Cancellation completion can take longer than the timeout; this is not a hard process kill.

Claim-to-evidence ledger ↓ · Review report ↓