Jeffrey Lowe

Sony doesn't expose a public OAuth flow for third parties. PlatPursuit authenticates with PSN using NPSSO cookies, which mint short-lived access tokens whose state needs to flow cleanly through every checkpoint of a sync pipeline: queue dispatcher, per-token worker, rate limiter, response parser, DB writer. Celery is closed off in that respect; every task hand-off goes through pickle byte-stream serialization, which forces lossy conversion and implicit assumptions at each step. TokenKeeper keeps that state in-process and JSON-shaped, so each piece of the pipeline reads exactly what the previous one wrote, no decoding ambiguity.

So I built it: one long-lived Python process, in-memory token pool, no broker.

TokenKeeper runs three worker threads per token group, pulling jobs from a five-tier Redis priority queue (orchestrator, high, medium, low, bulk). The bulk tier exists specifically so whale accounts with 5,000+ games can't starve normal users. Each token is wrapped in a TokenInstance dataclass tracking busy state (atomic Redis-based locking), expiry, an LRU cache of user lookups, and outbound IP for proxy routing. A health loop runs every 60 seconds: it refreshes any token within 5 minutes of expiring, publishes stats over Redis pub/sub for the dashboard, and runs a circuit breaker that trips on five PSN 5xx errors in a 60-second window. When PSN goes down, sync timestamps get backdated and writes pause cleanly until it recovers.

The hardest bug in this system wasn't conceptual; it was psnawp's default rate limiter, which uses a SQLite-backed bucket and produced "database is locked" errors under concurrent load. The fix was a small custom PSN client subclass that swaps in an in-memory bucket. A PSNManager facade hides all of this from the rest of the app: views, cron handlers, and API endpoints call assign_job() and never see the worker. One Python process owns the full PSN surface: auth, rate limiting, retries, priority queueing, and observability. No broker, no result backend, no separate worker pool to scale. The takeaway: when the obvious choice imposes a heavier operational burden than the workload justifies, building the right thing custom is often cheaper to operate and simpler to reason about. Especially when stateful auth sits at the core of the workload.