Strong types

Early in the ChatGPT days, I was working in a Ruby codebase at Shopify. We had TypeScript definitions describing the shape of the data. The Ruby side had nothing equivalent. I fed the TypeScript definitions to ChatGPT and had it convert them to Sorbet, Ruby's type checker. Then we ran the checker. It caught bugs we would have shipped.

The easy lesson there is that types are good. I don't think I need to argue that anymore. Types fix most of what's wrong with JavaScript, they catch the bug in the file you didn't open, and anyone who has done a big refactor with and without them already knows which one they'd pick. That argument is over.

Looking back, the Shopify story was an early version of a different argument. An LLM was doing the work, and the types were the thing that told us whether the work was right.

That's the argument now. LLMs write a lot of code. Mine do. When I read what an LLM wrote about its own work, I'm reading prose, and prose can be confidently wrong in ways that are hard to see. A type is different. It documents what something is, it defines what it should be, and the checker enforces it without caring who wrote the code. It's a contract both sides can read and neither side can quietly break. The model reads it before writing. The checker reads it after. I read it in the middle, and it's the one thing in the whole loop I don't have to take on faith.

Types are the strong contract that lets me be sure that what I'm doing, and what my LLMs are doing, is safe. Without types, the world scares me nowadays, to be honest.

The contract has to be true, though. At Alloy, the integrations codebase was legacy JavaScript with .d.ts declaration files. They looked like types, so people trusted them, and they were incorrect and incomplete. I wrote a script to convert them into real TypeScript types and we found bugs everywhere, mostly in the types rather than the code. The documentation of the system was lying about the system. Hand that to an LLM and it would build on the lie with full confidence. False types are worse than none, and that gets more true the less of the code a human writes.

I run my own version of this now. When I split a plan into pieces and hand them to cheaper models, the first thing that happens to each diff they send back is a build and a type check. Then a scan for the ways a model cheats: touched test files, skipped tests, stubs, functions nothing calls. The model's own report that it finished isn't something I trust. The checker is.

So if you're letting LLMs write code in your codebase, here's what I'd do.

Make the type check a gate, not a suggestion. Nothing a model produces gets looked at by a human until it builds and type-checks clean. It's cheap, it's fast, and it's the one review step that can't be talked into agreeing with you.

Write the types before you ask for the implementation. The function signature, the shape of the data in and out, the error cases. That's the spec. An LLM working against a real type has far fewer ways to wander than one working against a paragraph of English.

And if you already have type definitions somewhere (declaration files, a schema, an old doc), don't trust them. Generate real types from them and see what breaks. What breaks is what you've been shipping, and what every model you've prompted has been building on.