A runbook for Pinnacle API errors.

Four classes of response

A poller only needs four reactions. The table maps each status to one of them; everything after it explains the timing.

Stop errors: 401 and 403

Neither of these gets better with time, so retrying them in a loop only adds noise to the logs and, on a shared account, to the vendor's side. A 401 means the request carried no usable key: the header is missing, the environment variable is empty in this deployment, or the key was rotated. Check that the poller sends the key in the x-portal-apikey header and that the value matches the account panel.

A 403 is subtler: authentication worked, but the plan does not include what was asked for. A valid key can still lack a transport, an add-on or an endpoint, and the fix is on the account, not in the code. The access checklist on pnclODDS separates authentication from entitlement step by step. In both cases log the status and the error code, never the key or the full response body, and page whoever owns the account.

Wait errors: 429

A 429 says the schedule crossed the ceiling: 20 requests a minute or 100 a day on the free key, 10 or 30 a second on the paid REST plans. The Retry-After header is the authority on how long to wait. Honour it, add a little jitter so parallel workers do not resume in lockstep, and share the backoff state between workers, since ten polite processes still add up to ten times the requests. The rate limit page has the backoff loop and the headroom rule that keeps 429s rare.

HTTP/1.1 429 Too Many Requests
Retry-After: 2

A 429 on the daily allowance of the free key is different in kind: no amount of backoff brings the day back. Detect it by the size of Retry-After and stop the poller until the window resets rather than burning the next hour on rejected calls.

Retry errors: 5xx, timeouts and resets

These are transient until proven otherwise. Retry with a delay that starts at half a second, doubles each time and caps at 30 seconds, and give up after three attempts in one poll cycle. Failing the cycle is fine: the next scheduled poll runs anyway, and a snapshot that is ten seconds late beats a poller stuck in a retry loop. Keep serving the last good snapshot to your own consumers, with its age visible, so a dashboard shows stale prices rather than none.

def poll_once():
    delay = 0.5
    for attempt in range(3):
        try:
            res = get(url, timeout=8)
        except (Timeout, ConnectionError):
            sleep(delay + random() * delay / 2)
            delay = min(delay * 2, 30)
            continue
        if res.status in (401, 403):
            raise Stop(res.status)
        if res.status == 429:
            sleep(float(res.headers.get("Retry-After", delay)))
            continue
        if res.status >= 500:
            sleep(delay + random() * delay / 2)
            delay = min(delay * 2, 30)
            continue
        return validate(res.json())
    return None  # this cycle failed; the next one runs

Validate the 200

A successful status is the start of the check, not the end. The markets response is an object with an events array and a last cursor; require both before touching the payload. An empty array during a quiet hour is a valid answer and must not be logged as an error, while a body missing the cursor is malformed and must not replace the snapshot you already hold. Never fill a missing price with zero: a null money line means the market is closed or unavailable, and zero would look like a price to everything downstream.

What should page a human

Four conditions justify waking someone; the rest belong on a dashboard. A 401 or 403 at any time, because the poller cannot recover on its own. More than one 429 in a hundred requests for an hour, because the schedule is too close to the ceiling. A run of 5xx or timeouts longer than five minutes, because a transient failure has stopped being transient. And a snapshot older than three poll intervals on any board during a slate, which is the symptom the other three cause. The monitoring page turns those into metrics with thresholds.

Get API access

Questions from the on-call rota

Status meanings and allowances follow the vendor's published documentation, checked on 26 September 2026. Published 29 September 2026.