AGENTS.md Files Are Not Misleading Your AI Agent

There's a hot discussion on this topic on the internet right now. People are resharing the headlines, but I think few actually take the time to read the actual study that sparked the conversation. So we're going to break this down, but let's first talk about what these files actually are.

What are AGENTS.md and CLAUDE.md files

Some people explain them as sort of README files for your AI agent. They give a general overview of the project that you are working with. I would say it's a good enough explanation, but there's a little bit more to it.

These files are sent along with your prompt when you work with Cursor, Claude Code, Codex, or other AI coding agents. They get injected into the context window before your agent starts working on a task. Over the last couple of months, they have become sort of a standard and many people chose to add them to their projects.

However, it matters a lot how you choose to create these files.

What the study actually says

So let's go back to the study. The study shows a 3% performance decrease in tasks done by AI agents on repositories that contain these files over those that don't. This is what's making the headlines today. And it is actually a big deal.

But there's an important caveat.

On average, a 4% performance improvement was measured in those projects where AGENTS.md files were actually created by humans. The difference between the two numbers comes down to one thing - who wrote the file.

It matters a lot whether you create your AGENTS.md file using a /init command, essentially relying on your AI agent to scan your repo, or whether you create it on your own and update it as you go.

Why auto-generated files can hurt

There is a great video by Matt Pocock that goes into explanation on why using the /init command is not a very good idea.

Your AGENTS.md file should not contain things like npm run dev or other scripts that an average AI agent can already read from your package.json. That kind of information just adds noise. And a noisy context window means worse output.

Instead, it should be a place for things like:

• Custom project conventions - naming patterns, file organization rules, or coding standards your team follows
• Non-obvious architectural decisions - why you chose a specific approach over a more common one
• Business logic quirks - edge cases, special rules, or domain-specific knowledge that isn't obvious from the code

Think about all of the things that confuse your new colleagues. Things that you only learn after months of working on a project. That's what belongs into your AGENTS.md file.

Here's an example of what a useful entry might look like:

This is the kind of context that helps an AI agent make better decisions. It's information that can't be easily inferred from scanning the codebase.

The concept is not perfect

I feel it is important to say that the AGENTS.md concept is not perfect. It needs constant updates because it rots like documentation. If you reference a file and that file moves, or you reference a folder and you actually move that folder in your project, then having that stale reference in your AGENTS.md file is going to confuse the hell out of your agent.

There's also the fact that some tools make their own choices on whether to include these files. They can potentially introduce noise to the context window, which is probably the main reason for that 3% performance decrease in the study.

What I take from this

Overall, I think the study opened up a good discussion about AI workflows, context windows, and what actually matters when you interact with an AI agent. The key takeaways for me are:

1. Don't blindly auto-generate your context files - a /init command will fill your file with information your agent already has access to
2. Write them by hand - human-written files showed a 4% improvement
3. Focus on non-obvious knowledge - document what's hard to discover, not what's easy to find
4. Maintain your files - treat them like documentation that needs regular updates

But let me know what you think in the comments. As always, happy coding!