The fix was pretty simple. It’s what you would do with a friend who’s a patient expert. I took a screenshot and sent it over, and the agent saw the menu as it actually was that day and told me it had moved to a different submenu.

The agent’s map is out of date

An agent’s knowledge of a dashboard comes from whatever existed when it was trained. Cloudflare, GitLab, cloud consoles, really anything that ships UI changes constantly, will have moved on since then. So the agent gives you directions from an old map, and it has no way of knowing the map is old. A screenshot is today’s version of the page, with the actual labels and the actual layout, so it doesn’t have to guess.

Describing what you see is lossy

The same thing happens with your own work. When something looks off in a UI I’m building, my first instinct used to be describing it. “The table lines are basically invisible.” “The wrong toggle is highlighted.” “The diagram is broken somewhere in the middle.”

Each of those takes effort to put into words, and then the agent has to guess back from my words to the pixels. Half the time it fixed something next to the actual problem. A screenshot skips both steps. The agent sees the same thing you do, and you don’t have to be good at describing UI bugs.

That’s pretty much the whole point. Agents do better work with more information, and a screenshot is about the cheapest information you can give them.

Tell it where to look

Dragging images into the chat works, but it gets old fast. What made it stick for me was adding one line to my global CLAUDE.md:

1

If I mention screenshots, they can be found in "~/Pictures/Screenshots".

Now I take the screenshot like I normally would and type “see my latest screenshot.” The agent goes and finds it. Put the equivalent in your AGENTS.md or whatever your agent reads, pointing at wherever your screenshots actually land.

One caution though. Dashboards like Cloudflare’s love to show API tokens and account IDs, so crop those out before you share.

Agents can take their own screenshots too, with tools like Playwright, but that’s another post. Next time you’re lost in a menu or squinting at a UI bug, skip the description and send a screenshot. Try it out yourself!

Related posts

• Stop installing things your agent doesn’t need
• Writing an AGENTS.md for a national cluster

Models have taste

Different models are tuned to write code in different ways. Give two of them (of relatively the same capability, of course) the same task and you’ll get two different shapes back. One adds structure where the other would cut stuff. One treats a pattern as normal and the other flags it. Neither is always right. They just have different taste, the same way two senior devs on a team do. That’s why a second opinion is worth having. A different model is more likely to question the stuff the first one took for granted.

How I run it

I personally keep the reviews independent. Each model reviews the same project in its own worktree, so neither one sees the other’s take before writing its own. Then I have one agent merge and deduplicate the findings into a single write-up, or a parent issue with sub-issues if there’s real work to do. What’s great about this is that you don’t necessarily need any sort of fancy agent setup. Just two different terminal windows to get you started.

This does cost tokens. It’s two full reviews instead of one. Personally, I think it’s just good hygiene if you have the tokens to burn. If you finish most weeks with headroom on your weekly cap, this is a really good way to spend it. If you’re paying API rates for every token, it’s a harder sell. But then again, you’re probably in enterprise with a code review product.

Change the reviewer, not just the model

You don’t always need a second model to get a second perspective. You can tell an agent to review the code as someone specific.

Ponytail is a good example. It makes your agent think like the lazy senior dev who, in its README’s words, “has been at the company longer than the version control” and replaces your fifty lines with one. Its /ponytail-review and /ponytail-audit commands review a change or a whole repo from that point of view.

The idea is really easy to extend. Have a security person read the code looking for what an attacker would try. Have a junior who just joined the team read it looking for what they can’t follow. It’s been very useful for me to get feedback on sections of code from perspectives I don’t have by default. I don’t naturally read my own code like an attacker, or like someone seeing it for the first time.

Next time you finish a change, hand it to a different model, or ask your agent to review it as someone who isn’t you. Try it out yourself!

Related posts

• When to reach for Opus
• Why I still use Sol on low
• Start your five-hour window early

It arrived in the middle of a messy time for software licensing. The chardet library was rewritten with Claude Code and relicensed from LGPL to MIT, and its original author objected. PhotoCraft is building a “clean-room” Photoshop in Rust. Someone even launched Malus, a satire site selling “Clean Room as a Service”.

However, I think most of that unease is about commercial clones and relicensing other people’s work, and security concerns. There’s a much more boring use case that I think is really valuable though, getting your own research out of a tool you’re stuck with.

What REA actually does

REA doesn’t decompile anything itself. Your agent calls it over MCP, REA drives whichever analysis tool fits the target, and the findings come back with the evidence behind each one. The agent can then use those findings to explain the behaviour, or to write and test a new implementation.

The docs even describe how to keep it clean-room. The agent writing the new code runs somewhere that can’t read REA or its analysis output, and only gets a spec. That’s the classic two-team clean-room setup, just with LLM agents instead of people.

The paradigm you get stuck with

Lots of labs have an experiment or piece of code like this. The paradigm was built years ago in E-Prime, Presentation, or a MATLAB toolbox that only ships compiled code or something too difficult to modify. It works, but the licence is expensive, it only runs on one old Windows machine in the corner, and nobody who built it is still around. You’d like it in PsychoPy, but the details that matter are hidden. How the trials are randomised, which trigger codes go out when, and what goes in the output file.

That’s pretty much exactly what REA is for. Point the agent at the old paradigm, let it recover those details, and have it write them down as a spec. A second agent builds the PsychoPy version from that spec alone. That used to be a big chunk of somebody’s entire year as a Masters student when they didn’t even have a Computer Science degree. Now it’s a lot more doable.

Stay in your lane

Two things keep this on the right side of the grey area.

Ask first. Email the vendor or the original author before you take anything apart. They might just send you the spec, or the source, and then you don’t need REA at all. Or better yet, you and your lab are the owners anyway!

Validate against the original. Run the old and new versions side by side and compare the trial order, the trigger timing, and the output files. The rewrite is only useful in a paper if you can show it behaves the same, so honestly that comparison matters more than the code.

Cloning a commercial product to sell, or rewriting someone’s library to drop their licence, is a different thing. That’s where the chardet fight came from, and personally I’d stay far, far away from this.

Try it

If a closed tool is the only thing between you and a reproducible study, this is now a realistic project. Pick the paradigm that’s been stuck on that old lab machine, and see how far an agent and REA get you.

I’d been using a small CLAUDE.md for myself on Nibi for a while, and I was asked to turn it into something for everyone. The Alliance clusters get their environment from ComputeCanada/software-stack-config, so I opened PR #111 to add an agents/AGENTS.md there on request. Claude Code reads /etc/claude-code/CLAUDE.md system wide, and most other harnesses look for an AGENTS.md, so one file and a symlink covers a lot of ground.

Only say what the model can’t know

My opinion after going through this is that an AGENTS.md should only hold what the model can’t already know. Site policy, the stuff that’s different about these clusters, and when to stop and ask the human. That’s it.

Everything in the file is in the context of every session, so every line should carry a lot of value. Things like the module commands and Slurm commands are already known by the frontier LLMs. There’s no need to give specific guidance on them. The lower tier LLM services won’t even read these docs anyway.

Review is important

The next morning after sharing there were about 20 inline comments, and nearly all were things no model could know from training. Not every compute node has internet, so tarballs and wheels need to be staged from the login node. Bundle anything that runs under 15 minutes and checkpoint anything over 12 hours. Avoid uv. If a wheel or module is missing, tell the user to contact support instead of getting creative.

It also caught my own mistake. My Python section said to load the module, make a virtual environment, and install from the wheelhouse. It never activated the environment. A classic.

1
2
3
4

module load python/<version>
virtualenv --no-download <venv>
source <venv>/bin/activate
pip install --no-index <package>

So the people who run the stack should absolutely be writing these with you. They know the local gotchas way better than any model does.

Where I pushed back

The thing that kept coming up in review was adding specific wiki links to each section, plus fuller examples of commands like module spider. My problem with this is it’s just bloat. I think we can just tell the agents to check the documentation, or perhaps just say the page name.

What came after

A few more rules went in after that, similar to ones other sites have landed on, like CHTC at UW–Madison. The file now says human users remain responsible for the actions of their agents. It also says to point out stale VS Code processes and offer to kill them, to wait at least 60 seconds between Slurm queries, and to never insert sleep commands into jobs. Those are the same kind of thing, site behaviour a model wouldn’t guess.

The latest change points agents at a read only markdown copy of the Alliance wiki at /cvmfs/soft.computecanada.ca/custom/docs/, in English and French. Now the agent can go read the actual page when it needs the details, and none of it has to live in the file.

Since then, it has been deployed to the Nibi cluster. To see for yourself without SSHing to Nibi, visit the PR.

Related posts

• Stop installing things your agent doesn’t need
• Send your agent a screenshot

To be clear, they didn’t do anything wrong. The service is set up to let them do it. They had a pile of small work and a working endpoint, so they wrote some sort of loop. I probably would have done the same. But a loop that sends hundreds of thousands of tiny prompts is a batch job, and this was a really good reminder of why batch clusters are still super important.

What that load does to a shared service

A shared LLM endpoint is set up for lots of people asking questions here and there, interactive use. One person sending 800,000 of them back to back is a pretty different workload. Spread over a day, that’s roughly nine requests per second, all day and all night.

Every one of those requests costs something before the model even sees it. Each is a network round trip with its own connection overhead and its own spot in the queue. It also messes with caching. Inference servers lean on caches to keep things quick for the people already using them, and a steady flood of unrelated prompts pushes that useful state out. Time-to-first-response goes up for everyone, including the person asking a single question in a chat window who did nothing different.

Why a batch job is faster

On a GPU node in a batch job, you get the whole device to yourself. Nobody else’s traffic is in the queue, and there’s no network between your data and the model.

The bigger win is how the work gets handed over. Offline inference with a tool like vLLM takes the entire list of prompts at once and packs them onto the GPU as tightly as it can. Small prompts are where that pays off most, because one small prompt on its own leaves most of the GPU idle. Hand it 800,000 together and it keeps the device busy the whole time. You load the model once, point it at your dataset, and collect the results when the job ends.

If this is you

If your workflow loops over a dataset and calls an LLM once per row, please don’t forget about classic HPC. Write it as a batch job, run it on a GPU node, and keep the shared service for the interactive stuff it’s good at.

You’ll get more done quicker, and everyone else gets their response times back!

Related posts

• Serial Farms: Package Options and When to Switch to Farming

To be clear, there are actually two limits here, and they do pretty different jobs. The five-hour window controls when you can use your plan. The weekly cap controls how much you get in total. Most people worry about the five-hour one because it’s the wall they keep hitting, but in my opinion the weekly cap is the real budget.

How the windows work

On Claude Pro and Max, the five-hour session starts with your first message. It’s a rolling window, not a fixed clock time, and it resets five hours after that first message. The weekly limit is separate. It resets at a fixed time for your account, and waiting out a session won’t clear it. Chat, the desktop and mobile apps, and Claude Code all draw from the same pool.

Codex is split by plan. Plus and Standard Business have a five-hour window. Pro currently has none. Weekly limits “may also apply,” and OpenAI’s docs send you to your usage dashboard for the actual reset times rather than spelling them out.

Neither provider really counts messages. Model choice, context length, reasoning effort, and tool use all change how fast you burn through a window. A long agent session in a big repo with a top model eats through it way faster than a handful of chat questions for each of them though.

These rules change a lot, so check your own numbers instead of trusting a blog post, this one included. In Claude Code, /usage shows your session and weekly bars. In Codex, /status shows both percentages and when they reset.

Start the clock before you sit down

The five-hour clock starts on your first message, so you get to pick when it starts. If something sends a message at 7am, your first window closes at noon and you get a fresh one right after lunch. Now the morning is one window and the afternoon is another, instead of one window running right through the middle of your day.

The simplest version is pretty much just a ping. Most agent tools have a headless mode, so a cron job can send a throwaway prompt to a cheap model:

1

0 7 * * 1-5 claude -p "Reply with OK" --model haiku

codex exec "Reply with OK" does the same for Codex. Cron runs with a bare environment, so you may need the full path to the binary. A local cron job also only fires if your machine is on at 7am.

The nicer version is to have that early job do something useful. Most of the modern tools can run scheduled tasks, so set up a custom news brief. Have it cover what changed in the repos you care about, release notes for your tools, or whatever you’d otherwise skim over coffee. It starts the clock as a side effect, and you have something to read when you sit down.

The catch

This doesn’t ever raise your weekly cap. It gives you a second full window during your working hours, so you can burn through that cap faster. If you were already finishing most weeks with headroom, that’s a good trade. If you regularly hit the weekly wall on Thursday, you’ll hit it on Wednesday instead. Be careful.

What about peak hours?

For a while earlier this year, the time of day you used Claude mattered too. Weekday mornings from 5 to 11am Pacific drained five-hour sessions faster than the rest of the day. Anthropic removed that reduction for Claude Code on Pro and Max in May, and doubled Claude Code’s five-hour limits at the same time. Weekly limits didn’t double.

So off-peak usage isn’t worth more on Claude Code right now. Providers have done it before, though, and could easily do it again, so check whether it applies to your plan before you plan your day around it.

What I’d do

If the afternoon wall is what keeps getting you, try the early ping or a morning brief. Just keep an eye on that weekly bar!

Related posts

• Why I still use Sol on low
• Get a second opinion from another model

In my opinion, Tectonic and a coding agent are now a pretty practical alternative to Overleaf if you’re working on a document by yourself. The agent can work directly on local files and run the compiler. This way you don’t need it clicking around a browser through computer-use tools trying to diagnose how your tables are formatted.

A simpler local setup

If you haven’t used LaTeX, it’s a way to write documents as source files and compile them into typeset output, usually a PDF. It’s useful for résumés and papers, especially when you want precise formatting or have lots of equations. Your textbooks use it, for example.

Getting a local LaTeX setup working can be more effort than you wanted to spend on editing a document. Some of this technology is unfortunately pretty ancient. Tectonic makes that part simpler. It’s a TeX/LaTeX engine distributed as a single executable, and it downloads the support files it needs instead of requiring you to install a full LaTeX distribution up front.

There are installation options for different systems. Once it’s installed, compiling a document can be as simple as:

1

tectonic resume.tex

That produces resume.pdf. Tectonic also handles the repeated compilation passes that TeX and BibTeX can need. You’ll need network access to download support files that aren’t already cached, but you don’t have to manage those downloads yourself.

Let the agent keep trying

The useful part for me was making the changes interactively. LaTeX has plenty of classic formatting problems. Something doesn’t fit, spacing looks wrong, or a change in one place moves something somewhere else. An agent doesn’t really run out of things to try, which is pretty useful when you’re working through that kind of problem.

Ask the agent to edit the source, compile it, and then render the PDF pages to images for visual inspection if it has the tools to do that. If the change didn’t work, it can keep trying. Tectonic makes it easy for the agent to build/compile the PDF, and then other standard agent tooling can view the PDF for issues.

That last step is the big win. A successful compile tells you that you produced a PDF, not that the layout looks right. Ask the agent to review the rendered pages rather than stop as soon as the command succeeds. You should still look at the final document yourself every time of course.

This doesn’t replace Overleaf’s shared editing and collaboration features. But for a document I’m working on myself, letting an agent edit and compile the files directly is a much simpler workflow than having it operate a web editor.

If you’ve got an existing LaTeX document, try giving your agent the source and asking it to make a change with Tectonic. It worked pretty well for my résumé!

Say I have a ticket to implement an entire feature branch. The agent has to read the existing code, decide what needs changing, deal with whatever it finds along the way, and test the result. With Sonnet, I’d expect to need to help break that into smaller jobs. I’d essentially be pair programming with Sonnet, with it controlling the keyboard. This is totally fine, but missing a lot of the strengths of the frontier models.

On the other hand, Opus is the one I’d try giving the whole ticket to. No single step has to be especially hard. The agent is just doing more on its own without steering. Maybe it finds that the existing tests don’t cover the change, or that the first implementation needs work once it runs them. With Sonnet I’d expect to stop, look at that, and decide what to ask for next. With Opus I’d give it room to investigate and keep going. That’s the difference I care about when I say “wide,” not whether Opus writes a better line of code.

This isn’t a claim that Sonnet can’t implement the feature. If I’m there to break down the work and steer it, I’d expect it to do fine in the average case. Models like Sonnet and Luna have their place, but I wouldn’t choose them for a job where I want to leave the agent alone to work through all those branches.

Of course, I still read the changes, run the tests, and check the result against the ticket. If the ticket involves decisions I need to own, I don’t hand those over either. I just don’t want to have to tell the agent what to do next every five minutes.

If you’ve only been giving agents small jobs, try giving Opus a whole ticket and see how far it gets. Then check its work.

Related posts

• Why I still use Sol on low
• Get a second opinion from another model

Consider an ERP study and two cleaning policies. One rejects nearly every blink, along with a lot of trials. The other leaves a little ocular activity but keeps more trials. If I only count blinks, the first one wins. If I only count retained trials, the second one wins. Neither one is really better than the other.

I’d be wary of picking a method because it has a higher signal-to-noise ratio or fewer outliers despite how many publications in the field report this way. Those numbers depend on how you define signal and noise. My old metric counted the difference between raw and cleaned data as noise. By that definition, removing more data looked like progress. A flat recording could also score well on a variability metric and be useless for the study.

That doesn’t make artifact measures pointless. I just want to know whether a blink correction reduced blink contamination. I also want to know what it cost.

Compare what you get and what you lose

For an actual study, I’d put a few plausible preprocessing choices next to each other and ask:

• How much targeted artifact remains? Which type of artifact hurts us the most?
• How many channels, time periods, and trials did each choice remove or reconstruct? Did that differ between conditions or participants?
• What happened to the effect estimate and its uncertainty, such as its confidence interval?

A narrow confidence interval isn’t automatically good. If a pipeline systematically distorts the effect, it can give a precise answer to the wrong question. A smaller p-value isn’t always the goal either. If two pipelines disagree, I’d look at what each one removed before picking the result I like better.

Real recordings usually don’t tell us the true effect, so comparing their results can’t establish which pipeline is correct. If I can simulate data or inject a known effect into realistic recordings, I can ask a harder question: does the method recover the effect without systematic bias? Across repeated simulated datasets, do nominal 95% confidence intervals contain the known effect about 95% of the time? How wide are those intervals, and how much data did the method discard to get them? This is the sort of thing I want to see in pipeline style publications.

The answer might depend on the study. A method that throws away a lot of time could be acceptable for one analysis and disastrous for another. An aggressive trial-rejection rule might remove artifacts while leaving too few trials for a useful estimate, or reject one condition more often than the other. That’s information a single cleanliness score won’t give me.

I don’t expect every EEG study to run a giant pipeline benchmark or accidentally turn into a methods paper. A useful first pass is to run a couple of defensible cleaning choices on the same data and put the artifact measurements, data-loss counts, and effect estimates side by side. If we can test against a known effect, we should check whether the method recovers it. That’s more work than picking the cleanest-looking trace, but at least it tells us what we paid for the cleaning.

Related posts

• How Do You Test Whether EEG Got Cleaner?

It also overengineered some open-ended problems I gave it. And I can’t use it every day without upgrading my plan. That matters more to my recommendation than how impressive it was.

What I actually use

GPT 6 Sol on low is my default in Pi. So far it feels like a flat upgrade over 5.6 Sol for my work. The change I notice most is the prose it writes without any writing skills. Low is enough for most tasks. I only turn up the reasoning for really hard ones.

Fable is amazing for code I can trust enough to review and merge. I can give it a wildly broad task, and if it gets stuck, it comes back to me instead of burning tokens forever. I don’t have it on my current plan, though. Sol on low does enough that I rarely need to reach for the big guns.

Opus 5.5 is what I’ve started using for my longest-running tasks with lots of context. It feels fast, and the prose is way better. The “Claude speak” that bothered me before seems to be gone. Opus 5 was a flat no for me. 4.8 was alright, but Sol 5.6 beat it easily. I didn’t expect to come back to Opus, but here I am.

The plan changes the answer

My recommendations assume subsidized subscription plans, not pay-as-you-go API pricing. Running agents at API rates every day would be too expensive for me. Check what’s included in your plan before copying anyone’s model ranking. The expensive option needs to do something your cheaper default can’t.

I’ve updated my Recommended AI Tools page with what I use now. If you’re choosing a setup, start with a model you can afford to run often, then work out which tasks actually need something bigger.

Related posts

• When to reach for Opus
• Check what your AI plan does with your data
• Stop installing things your agent doesn’t need
• Start your five-hour window early
• Get a second opinion from another model