clawmates/skills/backend/api-pagination-day-1.md

name, description, when_to_use, tags

name	description	when_to_use	tags
api-pagination-day-1	Every list endpoint paginates from the first version. Cursor-based over offset. Bounded max_limit. Includes total when cheap.	You're the api_designer role, or a coder adding any `GET /...` endpoint that returns a collection.	backend api

List endpoints paginate from day 1

Unpaginated lists are a production incident waiting to happen. The one time the endpoint is called with a million-row table, the response body fills the load balancer's buffer, memory spikes, and everything downstream 503s.

The pattern

#[derive(Deserialize)]
struct ListQuery {
    #[serde(default = "default_limit")]
    limit: u32,
    /// Cursor from a prior page's `next_cursor`. Opaque to the client.
    cursor: Option<String>,
}
fn default_limit() -> u32 { 50 }
const MAX_LIMIT: u32 = 500;

#[derive(Serialize)]
struct Page<T> {
    items: Vec<T>,
    next_cursor: Option<String>,
    /// Present only when it's cheap to compute (COUNT(*) over a small
    /// filtered set). Never for a full-table count.
    #[serde(skip_serializing_if = "Option::is_none")]
    total: Option<u64>,
}

Cursor over offset

• Offset (LIMIT 50 OFFSET 500) — Postgres scans and discards the first 500 rows on every request. Cost grows with page number.
• Cursor (WHERE id > $cursor ORDER BY id LIMIT 50) — O(log n) always. The cursor is the last row's sort key, base64'd (or JWT-signed if you want tamper-detection).

For clawmates, the standard cursor shape is base64(json({"id": "...", "ts": "..."})) — includes id + the ORDER BY tiebreaker so pages are deterministic even under concurrent inserts.

Bounds

• limit.clamp(1, MAX_LIMIT) — server-side clamp, not just server-side validation. Never trust the client to respect max_limit.
• Return 413 Payload Too Large when a caller sends limit=100000 if you'd rather they notice than have their traffic silently clamped.

Sorting

• Every paginated endpoint has ONE canonical sort order.
• Cursor must include the sort key + a tie-breaker (usually id). Otherwise two rows with the same sort key can be missed or duplicated across pages.

Total counts

• Cheap: SELECT count(*) FROM t WHERE small_indexed_filter. Include it.
• Expensive: unfiltered COUNT on a large table. Don't run it. Return total: null and let the UI show "50 of many".

OpenAPI

The response schema declares items, next_cursor, and total?. Never return a bare array — you can't add a cursor later without a breaking change.

Anti-patterns

• GET /foo returning [...] with no pagination shape. Every list route pays for future-proofing on day 1.
• page=N&per_page=M — offset-based. Ships until traffic bites you.
• Cursor that includes an unsigned index into a mutable list — reorders + inserts corrupt it. Cursor must be a stable identifier.