Linguabase vs. LLMs

Three problems LLMs can’t solve for word games.

LLMs are architected to give a good answer to a question—not to provide comprehensive data. Word games need three things LLMs can’t deliver: complete inventories (all valid words, not just some), balanced associations (all meanings, not just dominant ones), and reliable output (verifiable, consistent). These aren’t prompting problems—they’re architectural limits. (Example: ask an LLM to list all two-word phrases that name things. It can’t—we solved this.)

1. The Enumeration Problem

Each answer from an LLM is just a sliver of its training. You can’t extract comprehensive inventories without exhaustive queries—and even then, you can’t know what you missed:

• Word lists—An LLM can’t tell you all 400K valid English words. It can check if “syzygy” is a word, but it can’t enumerate every word you might need.
• Obscenity lists—An LLM can’t comprehensively list all obscenities. You’d have to prompt for every candidate and hope the model doesn’t refuse or miss edge cases.
• Soft blocklists—Legitimate words you might exclude from automated puzzle design: “abort,” “stroke,” “balls,” “moist.” Acceptable in a dictionary, awkward in a word game. An LLM can’t give you that list.
• Difficulty rankings—An LLM knows “proboscidean” is harder than “elephant.” But all its judgments are relative to a single prompt—not calibrated against an absolute scale of all your words.
• Path enumeration—Ask for a path from “sugar” to “peace” and an LLM finds one. But word games need all 50,000+ paths—for hint selection, difficulty tuning, alternate routes.

2. The Bias Problem

LLMs sample from training distributions where some meanings appear far more often than others. Ask for associations and you get what’s statistically dominant—not balanced coverage across all senses.

You can ask for “diverse” or “unusual” associations—the LLM will try—but it’s still sampling from the same skewed distribution. The minority senses aren’t in the high-probability zone.

Evidence: LLM vs. Linguabase

We asked Claude for word associations and compared the results. The pattern is consistent: LLMs give you the statistically dominant meanings and miss the rest.

Comparison: Claude vs. Linguabase, January 2025.

Which do you prefer? The leftmost column (unique to Linguabase), or the rightmost (created by an LLM)?

Over time, the sameness compounds—level 49 starts to feel eerily like levels 36 and 85.

What’s Missing from the LLM Responses?

• key—the island meaning (cay, atoll, reef) is absent
• bridge—the card game (bidding, trump, slam) and guitar parts (luthier, nose) are absent
• window—computing (browser, tab, popup) and “window of opportunity” are absent
• nephew—the etymology (nepotism derives from nephew) is absent
• penguin—cultural references (Linux, Happy Feet, Morgan Freeman) are absent
• giraffe—the LLM says “horns”; the correct term is “ossicones”

Good word games retain players through that kind of richness: non-obvious facets, technical depth, cultural touchstones, etymological connections, sensory and experiential associations. Native English speakers notice when puzzle vocabulary is shallow—when every level draws from the same pool of animals, colors, and food. Vocabulary that rewards what players already know is what keeps them coming back.

The Superconnector Problem

Some words connect to everything. “Heritage” appears 110,000+ times across our sources. “Unique” appears 168,000 times. If we didn’t penalize these superconnectors, they’d crowd out more interesting associations. Test your intuition:

Our production pipeline applies inverse-frequency weighting to demote these superconnectors:

Demotions ensure superconnectors don’t dominate, making room for distinctive connections.

3. The Reliability Problem

Even when LLM output is useful, it’s not production-ready. Three issues make it unsuitable for shipping directly:

You can’t ship probabilistic output that might be different tomorrow, or quotes that might not exist.

Validators, Not Generators

You can use an LLM to check whether “key → reef” is a valid association. It’ll say yes. But it won’t generate that association reliably, because reef isn’t in the high-probability zone when the prompt is “what’s related to key?”

That’s the core insight: LLMs are good validators, bad generators—at scale, for lexical data.

We use LLMs extensively for verification and filtering:

• Checking relationships—“Is key → reef valid?” Yes.
• Filtering content—“Is this quote appropriate for a family game?”
• Ranking candidates—Given a list, LLMs help order by relevance.

How We Built It

We spent a decade building the data the hard way:

• 70+ reference sources—Professional thesauri (NASA Thesaurus), public domain lexicons (WordNet, Roget’s), Library of Congress subject headings, specialized vocabularies across scientific, governmental, artistic, and medical domains
• Professional lexicography—Orin Hargraves (contributor to major dictionaries) wrote 2,000+ custom definitions and 4,400+ sense-grouped associations
• 5,000+ thematic word lists—Hand-curated by linguistics grad students and post-docs over several years
• 2.3M supercomputer hours—Pre-LLM computational linguistics (LDA topic modeling, Word2Vec embeddings, Wikipedia extraction) via NSF XSEDE grant
• 648,000 Library of Congress classifications—Semantic clusters discovered by analyzing how human librarians categorized millions of books
• Production-tested—Powering “In Other Words” with continuous refinement from real gameplay
• LLM validation—For ranking and filtering, not generation. LLMs confirm relationships; they don’t reliably discover the non-obvious ones.

Read the full methodology →