I've wanted my assistant to understand geography for a while. Not geography in the capitals-and-rivers sense, but the geography of actual questions: Where are the tennis courts near me? What's the largest park in town? Can you show me every county in a state on one map?
The data to answer all of these questions exists in OpenStreetMap, and it's open. The problem is that the raw APIs around that data are a poor fit for an LLM. They use a specialized vocabulary, return large and sometimes messy geometries, and can quickly bloat the model's context window by passing thousands of coordinates back and forth. This post is a little longer than usual because Geo turned out to be a particularly good example of the difference between wrapping an API and building a toolkit.
OpenStreetMap doesn't call a tennis court a tennis court
OpenStreetMap is an open tagging system, with a perfectly logical
vocabulary that almost nobody actually speaks. In OpenStreetMap, a
tennis court is leisure=pitch plus
sport=tennis. A gas station is amenity=fuel.
A drugstore is amenity=pharmacy. A model that queries
for leisure=tennis_court has produced perfectly
plausible English, but also an invalid query that returns nothing.
So the toolkit translates the request before it queries OpenStreetMap. It includes close to 4,500 natural-language aliases generated from OpenStreetMap's own tagging schema, validates values for the most commonly used tag keys, and expands interchangeable terms such as field, court, pitch, and diamond when they refer to the same kind of sports facility.
If you enter sport=tenis, it suggests tennis,
table_tennis, and paddle_tennis rather than
sending a typo off into the network and waiting politely for nothing
to come back. It also handles multi-use facilities, so a court tagged
sport=tennis;basketball appears when you ask for either
sport.
When the toolkit rewrites a query, it tells you exactly what changed. The correction is visible, not silent.
Shapes you can trust
The second problem is what to do with the geometry once you have it. A lake or city boundary can contain thousands of coordinate pairs. You don't want to hand all of those coordinates to an LLM, so the toolkit simplifies shapes before returning them.
But simplification is also where the map can begin lying to you. Early on, I forced the Lake Washington shoreline down to 80 points and, in the process, apparently created 42.6 percent more lake.
The toolkit now projects the geometry locally and applies topology-preserving Douglas–Peucker simplification in metres. The algorithm removes points while retaining the overall outline, rather than forcing every boundary down to an arbitrary vertex count. It does not guarantee that area will remain unchanged, so area and other measurements still come from the original geometry. The toolkit also compares the simplified area with the original and warns you if it changes by more than 5 percent.
A simplified outline is fine for looking at. Just don't ask the sketch to do the measuring.
The toolkit also handles the geometry work LLMs are not particularly good at: merging all the parks in a city, buffering a boundary by 100 metres, intersecting two areas, measuring perimeter, or testing whether one shape contains another. The operation happens on the server, so the model gets the result rather than a pile of coordinates.
Keeping 21,000 coordinates out of the context window
The context-window problem became obvious in practice. I was passing the Shasta Lake shoreline between tools, and even after simplifying it to 586 vertices, the geometry was copied into the model's context three times, at several thousand tokens each time.
The full boundary has 21,512 vertices and would not fit at all. Simplifying it until it did fit solved the context problem by creating a geography problem: the lake's area shifted by 7.9 percent.
The fix was simple in principle: stop passing the shape around. When
a tool produces a complex boundary, it stores the full geometry
privately and returns a short reference such as aoi_7b8....
Every other Geo tool accepts that reference in place of the geometry itself. Derived operations return new references, so merge, buffer, and intersect can proceed without the shoreline ever taking a tour through the context window.
The model still gets the metadata it needs, including bounds, area, and vertex count, but never sees the coordinates themselves. References expire after about 48 hours. Identical geometry produces the same reference, so retries do not create duplicate lakes.
Rendering geography
Sometimes, after all that geometry, what you need is simply to see
it. The new render_outline tool accepts boundaries by
name, OpenStreetMap ID, or Geo reference and returns a hosted image.
It handles the mechanical parts of turning geography into an image: choosing a projection, fitting the canvas, assigning a colorblind-safe palette, and adding attribution. It also refuses to render an administrative boundary if it could assemble only part of it. A map can be wrong very convincingly.
Labels are harder. A cartographer can move them off-centre, add
leader lines, abbreviate names, or split them across two lines.
render_outline places one label per shape at a
representative point. When two labels would collide, it drops one
rather than overlapping them, and tells you which one it dropped.
That makes the tool better suited to outlines, highlighted regions, and other visual assets than to densely labelled reference maps. The resulting images can be used directly in a presentation or flashcard, or passed into another toolkit.
Here are Tokyo's 23 special wards, with Setagaya-ku highlighted. It took a single request, and none of the underlying geometry entered the model's context window.
Flashcards, because why not
Composability is where toolkits start to get more interesting. Since adding the Anki toolkit last month, I've been thinking of ways to connect it to something else, and geography supplied one almost immediately.
Learning where things are is a classic flashcard subject, but most existing geography decks are built from static image packs assembled by hand.
With Geo and Anki connected, my assistant generated a South Dakota county deck in one conversation. For each card, Geo renders the whole state with one county highlighted and the other 65 dimmed. The Anki toolkit then pulls the image into the deck by URL and creates the card. The front asks which county is highlighted; the back gives its name and county seat.
No image bytes enter the model's context window, and there is no static image pack to prepare in advance. The same process works for states, countries, city districts, or neighborhoods.
What's coming next week
Geo is useful on its own, but the reason I built it now is that I need it for the Earth observation toolkit I'm releasing next week. It will provide satellite imagery and vegetation and water statistics for any area you can describe in words, for individual dates or across a date range.
The two toolkits are designed for composability. A Geo reference for a merged set of parks, a buffered lakeshore, or a town boundary can pass directly into the Earth observation tools.
You'll be able to say, "Merge every park in my city and tell me how green they stayed between June and August," and the geometry will pass from one toolkit to the other without ever entering the model's context window.
The Geo toolkit is useful on its own. It gets a lot more useful next week.
As always, ideas and suggestions are welcome at gerrit@toolforest.io.