The site is simple, but search always has fun tricks worth thinking about. These are notes on the nuances like where the data lives, what a query actually costs whether that be $ or in latency.
The next time you go on a website or app, it's a good exercise to think deeper about the "slight of hand" that delivers your experience.
It was designed to feel that way. Empathize with the designer. What were they thinking?
The database that supports this site is a turbopuffer namespace in the gcp-us-central1 region inside a curious state called Iowa.
A turbopuffer namespace is really just a prefix in object storage. Think of vendors like GCP and AWS S3.
I had never used render.com before and this was a good side project to click around. It didn't let me set the region (or I was too tired to see it) and the site got provisioned in Oregon.
This creates a subtle issue!
Picture someone in Florida: their request hops to the site in Oregon, which queries the namespace in Iowa, which is a GET request from object storage. Three geographies for one clue lookup.
To make it even more real. Florida to Oregon ≈ 60ms round trip (coast to coast). Oregon to Iowa ≈ 40ms round trip. In Iowa alone there is a 5-18ms metadata check for strong consistency queries. On a cache miss in memory/disk, it goes to object storage and issues a GET request which is ~100ms. (round-trip figures from napkin-math.)
60ms + 40ms + 10ms + 100ms ≈ 210ms
Not an accurate measure, but the right ballpark. All it took was a few braincells to add some numbers together.
As you expose yourself to more systems, you exercise an important muscle, which is to actually smell it.
There are plenty of ways to work with this latency:
A decent move to hack more speed is to put the site where the database is. Placing in the same region will minimize the latency cost.
If it were truly simple, I could just store the data on the site itself. But this is a dumb site and since I'm already a turbopuffer customer, I figured it was worth entertaining since I'm already paying for it anyways.
The "store it on the site" idea done properly is interesting: keep the cold corpus in turbopuffer, and let the site's own cache absorb the hot queries. The data is tiny. It's a handful of columns x N rows, one row per clue. When I fetch the data, it can sit in the memory of the browser cache to avoid paying network trips.
Or maybe you are super hardcore and you provision the site in the literal server room as the database.
A lot of interesting topologies to consider in the context of geography.
At the database layer there are some other tools to leverage as well. For example: the database doesn't mutate at all (almost no writes). On turbopuffer, strongly consistent queries have a floor latency of around 5-18ms because each request performs a metadata layer check on object storage. But if we turn off strong consistency for eventual consistency, we remove this floor.
Also, we're not always paying that ~100ms on object storage. In turbopuffer, that request to object storage is for a cold query. The data is then placed in memory/disk, which makes subsequent queries must faster; putting them in the world of tens of milliseconds and not hundreds of milliseconds.
It's important to consider this because you want to traverse through the compute-storage contiuum elegantly. Skating through ns-us-ms in computer land is like going to Mars to lace your shoes.
Have a muscle for search latency. In database land, their way of life is keep writes dumb and reads smart; just like in real life, the ceiling for smart always gets higher.
There was a nerd who crushed the query plan to fetch your profile, your reccomendation feed which might include videos, and your messages, which all cleared in ~1 second.
It cost $ to run a site. There is a literal machine out there provisioned to participate in TCP for you. The same goes for the database. Somewhere in the great beyond of Iowa is a machine with a sticker on it that says "Google Cloud Provider" and it's storing these crosswords.
This can make free plans tricky. Any product that lowers in $ is going to disrupt the distributor's revenue graph.
Some would call this unfair to the paying customers because those free plans now compete for the same pool of machines of regular plans.
These decisions aren't always easy. The downstream effect is that it can affect employees, investors, or even brand. When you see starter plans, consider empathizing with the decision maker behind the scenes.
Some other money stuff can get interesting at the database layer. The reason why turbopuffer is an interesting choice is because it's object storage native. AWS S3 for example cost about $.02/GB/Mo. This is an attractive strategy for searchforx.com becaus for the most part, the crossword clue-answer data sits untouched. Who cares about a clue from 2012?
But once a query on that sort of data gets a hit, it becomes a more active member in memory or disk, which is marginally faster.
My site also runs a job to append the most recent puzzles. On a daily basis, I now must write to a namespace in Iowa.
On S3, the standard rate is about 0.005 per 1000 PUTs (or $0.000005 per PUT). Cheap for me but consider an app like instagram with an order of magnitude of a billion requests per second. $0.000005 times 1e9 equals $5000 (every second!).
A common strategy is to drive as much data into a batched PUT request. A simple maneuver, but it means a lot at scale. Especially since it's free to have an instagram account! Every discount they can make on infrastructure in turn increases the available pool in profit.
It's good to consider the money stuff when you go on a site. You don't have to make it too deep; simply add or multiply some numbers. The smartest people you know are too stupid to make things complicated.
Say you search "OREO"
at query time, it's packaged into a lookup for "OREO" and "OREOS" which are both the singular and plural together.
The catch:
Done irresponsibly, the expansion can turn a fragment into a real word. Type "BONU" and the pluralizer helpfully makes it "BONUS" which is a real answer, so you get back every BONUS clue even though you never typed a whole word. Convenient sometimes, surprising other times, but never dangerous for our workload: each variant is still an exact match against real answers, so the worst case is extra rows.
See how a simple feature can add complexity to your query?
The generic lesson to learn about reads & queries is that you can always make reads smarter, but there's a tradeoff.
All three search modes run the same turbopuffer query and only swap the filter, but each leans on a different kind of index:
In), plus the plural
trick above.Feel free to find a way to reach out if you are interested in crosswords or turbopuffer internals. Would love to chat.