pyparsing’s backtracking was the latency ceiling (~3.2k parses/sec). How we identified it, built a hand-written recursive-descent parser with pyparsing as a live parity check, and caught 22 edge cases along the way.
Swapping cachetools for cachebox (a Rust-backed Python cache) required only a thin key-hashing compatibility wrapper. TTLCache operations went from 537 ns/op to 90 ns/op — a 6× gain for a one-line dependency change.
Card search moved from PostgreSQL into a Rust extension via PyO3. The 76× speedup came not from clever SQL tuning but from eliminating the database round-trip entirely — and from hitting a ceiling that Python itself could not clear.
Ten worker processes share a port. A write that clears the cache on one worker leaves the other nine serving stale results. Fixed with a shared multiprocessing.Value generation counter checked on every request.
The /random_search endpoint ran two full-table queries on every request. A 10-minute TTL cache materialized the 30k preferred-printing list once; individual requests then did an in-memory sample. Zero DB round-trips on the hot path.
Switching from row-by-row inserts to PostgreSQL’s COPY protocol meaningfully cut import time. Why COPY is fast, how to stream data into it from Python, and two approaches to expanding a JSON blob into a typed row.
Each AST node emits a SQL fragment and bound parameters. How the node hierarchy works, how different field types generate different SQL, and why user input never touches the query string. Covers text/regex/arithmetic; JSONB operators and the count+results CTE are in separate posts.
The Scryfall query language implicitly ANDs adjacent terms — pyparsing does not. This post covers the preprocessing layer built on top: implicit AND injection, query balancing in two languages, and the edge cases that required fixes.
How I evaluated parsing approaches for Scryfall’s query language: why regex and ANTLR were ruled out, how Lark compares to pyparsing, and why pyparsing won.
Why Sylvan Librarian uses Falcon and Bjoern instead of the FastAPI + uvicorn default: a preference for explicit, close-to-vanilla Python over framework magic.