Work
Platform architecture, data pipelines, and LLM tooling — each with the problem, the design, and what it changed.
The platform layer of a web data API — gateway, auth, metering, billing and durable workers — so every request is authorised, priced and charged once.
One public edge for a private engine — keys, plans, credit checks and clear refusals at API Gateway, so only paid-for work reaches the engine.
How every pline.ai request becomes a priced usage record and a credit debit — and why the pipeline moved from a Flink job to long-running Temporal workflows.
How pline.ai returns any page as JSON in the caller's schema — building on an LLM transformation engine for customer data at Grepsr.
Measuring the proxy bandwidth every crawl run uses — from a Node proxy, through a Flink job, to a priced cost per run in the dataset history.
Taking a Temporal pilot to every scheduled crawl, then owning the scheduler for three years — rollout, pod lifecycle, timezones and async.
Turning a plain-English instruction into a validated, sandboxed Python transform — and an LLM path for the rest, with a cost ceiling and real spend.
A central authorization service on Cerbos — one gRPC call that resolves who is asking and what they touch, then lets versioned policies decide.
A Temporal workflow that removes a user safely and in line with GDPR erasure — checks first, hands over what they own, then anonymises and deletes their data.
A service that profiles each crawl run's dataset in its own Kubernetes job, orchestrated by Temporal — with a size gate, reconciliation and per-page reports.
Encrypting stored delivery credentials, delivering into customer S3 by assumed role, modern SSH keys, and adding destinations without touching existing ones.
Moving finished runs out of MongoDB on a schedule set by how the run ended, and restoring them from Avro on S3 in tuned batches when someone needs them.
Routing crawls, task queues and data to a second AWS region one report at a time — and replacing static AWS keys with role-based credentials along the way.
A per-run dedup cache that crawlers call for every record, and a serverless router that turns platform events into Temporal workflows exactly once.