# A child can count better than Frontier AI Models

If Your “Frontier” Model Can’t Count, You Don’t Have Reasoning. You Have Vibes.

Here’s a truth that makes people mad:

A 128K context window doesn’t mean your model can **count**.

It means it can **read more** while still doing approximate attention math.

So when you ask an LLM to aggregate a long document—count categories, sum totals, tally events— you’re not testing “reasoning.”

You’re testing whether it can do **exact arithmetic using a tool that isn’t exact**.

That’s why OOLONG hurts.

Here’s the cheat code: **don’t let the LLM count.**

---

Stop Asking LLMs to Aggregate. Make the CPU Count.

## 🐉 The Counting Dragon

This dragon shows up everywhere:

- “How many customers mentioned churn risk?”
- “How many times does the term X appear?”
- “What’s the total revenue by segment?”
- “Count the number of records that match criteria Y”

A pure LLM answer often feels right… until you verify it.

And in production, “close enough” is wrong.

---

## The principle (one line)

### **Use the LLM for classification. Use deterministic code for aggregation.**

That’s it.

That’s the whole trick.

---

## The Counter-Bypass Protocol (Stillwater style)

This is the protocol I use to beat long-context aggregation traps:

### 1) Parse → structured rows (deterministic)

Turn the big blob into rows/fields.

### 2) (Optional) LLM labels each row

Only if the classification requires fuzzy semantics.

### 3) CPU aggregates (required)

Use `Counter()`, `sum()`, exact integers/fractions.

### 4) Verify with executable markers

Emit counts + a reproducible audit trail:

- sample rows that match each bucket
- hashes of inputs/outputs
- “if you rerun, you get the same result”

**LLM is the interpreter. CPU is the accountant.**  
Never swap those.

---

## A simple example (that scales)

Let’s say you have 5,000 support tickets and you ask:

> “How many are billing vs bugs vs feature requests?”

**Bad approach:**

- paste all tickets into the model
- ask for totals
- trust the numbers

**Disciplined approach:**

- store tickets as rows
- ask LLM to label each ticket (billing/bug/feature) with a confidence tag
- use CPU to count labels exactly
- output: totals + 20 example tickets per category + rerun command

Now your result isn’t “a vibe.”

It’s a **report you can audit**.

---

## Why attention fails at counting (plain-English)

LLMs don’t “loop” over tokens the way your code loops over rows.

They compress patterns, and compression is approximate.

Counting is not pattern-completion.

Counting is **enumeration**.

So the fix is not “think harder.”

The fix is: **use the right tool.**

---

## The MrBeast-style challenge (viral loop)

Let’s do a public sparring test.

**Challenge:** paste a long chunk of text (or a dataset summary) and ask for an exact count or aggregation.

I’ll reply with:

- the minimal parse schema
- whether you need LLM classification (or not)
- the exact `Counter()` aggregation strategy
- what receipts you should log so a skeptic can rerun it

Comment: **COUNT** and I’ll drop the “Counter-Bypass Card” you can paste into your agent prompt.

---

## Bonus: the Tower framing (makes it memorable)

This is **Floor 2: Foundation** in the Stillwater Tower.

If your agent can’t beat the Counting Dragon, it shouldn’t be touching:

- analytics
- finance
- ops dashboards
- compliance summaries
- incident postmortems

Because those domains punish “almost.”

---

## The point (one line)

**Long context isn’t memory. It’s a bigger whiteboard.**  
If you want correctness: **determinism + receipts.**

---

**Endure. Excel. Evolve.**  
— Phuc Vinh Truong