Usage metering: from Flink to durable workers in Go

Context

In a usage-based API, metering is the product’s revenue. Every request must become exactly one usage record and one credit debit — priced by what actually ran, and never twice.

The first version followed what the team already ran in production: a Apache Flink job that consumed request logs, built hourly usage aggregates, and pushed them into an internal ingestion service.

Why it moved

Hourly aggregates were fine for a usage chart. They were the wrong shape for credits:

• Credits are per request. A customer’s balance has to move when the request finishes, not when the hour closes.
• Pricing needs per-event logic. One request’s price depends on its proxy tier, whether a browser rendered it, and how many LLM tokens a JSON extraction used.
• Failures need a home. A malformed event must not stall the pipeline, and a retry must not charge twice.

So within the first week I replaced the Flink job with a queue and a Temporal worker: first in Python to prove the flow, then rewritten in Go.

Design

Off the response path. The engine publishes a metadata-only event through a bounded publisher and returns. Full request and response payloads stay in the logging pipeline; the queue carries only what billing needs.

Long-running workflow, bounded history. One workflow runs several polling activities in parallel. Each activity long-polls the queue, prices each event and records it, then returns after a fixed window. The workflow periodically continues-as-new so its history never grows without limit.

Pricing in one place. The worker turns what ran into credits: proxy tier, browser rendering, and token-metered JSON output. It returns an itemised receipt with the charge, so a customer — or support — can see why a request cost what it did.

Credit cycles as a workflow. Monthly resets run in their own workflow: one heartbeating activity scans for due cycles every five minutes, instead of a cron job that nobody watches.

Failure handling

Most of the work was deciding what a bad event should do:

• A failed request costs nothing. Failures and client disconnects are recorded with zero credits, so usage history stays complete without charging for work that did not deliver.
• Poison messages are acknowledged, not retried forever. An event whose identity conflicts with the record is answered with a named error code and removed, instead of blocking the queue.
• Late is not wrong. Job lifecycle events can arrive out of order. They are accepted with a warning rather than rejected — rejecting them stalled the queue.
• Bound every field. An error code is truncated before it is stored, so a long browser trace cannot stall the pipeline.
• Replays are scoped. A reservation replay applies only to its own caller, so a retry can never settle someone else’s credits.

Result

The pipeline meters ~200k requests a day with per-request pricing and itemised receipts. Adding a pricing dimension — browser rendering, LLM tokens — is a change to one worker, and every failure mode has a defined outcome instead of a stuck queue.