Skip to main contentSkip to navigation
All posts

Notes on working with AI

Give your first agent read access and nothing else

An agent is data access, written rules and a scheduler. The design choice that matters most is keeping read permissions apart from action permissions, then proving the rules in a dry run.

4 min read
  • ai-agents
  • agent-architecture
  • permissions
  • automation
  • human-in-the-loop

An agent has three parts, and only one is hard

A useful agent is made of three components. Data access, which means connectors or tools that let it read from a source. A set of rules, written out explicitly. And a scheduler that runs the check without anyone asking.

The first and third are plumbing. The second is where agents succeed or fail, because a model can only apply a condition you managed to state. "Tell me if anything interesting happens" is not a condition. "Alert me when a supplier's price on any item moves more than 5% against last week" is. If you cannot write the trigger down, the agent has nothing to check.

Reading and acting are different permissions

The most consequential design decision is to treat read permissions and action permissions as separate grants. An agent that can read market data, a price list or an inbox should not hold the right to place an order, send a message or change a record unless you gave it that on purpose.

The reasoning is the cost of a mistake. A wrong read produces a bad alert, and you lose a minute. A wrong action produces an order, an email a customer already received, a payment that left the account. The first is a nuisance. The second is an incident, and some incidents cannot be undone.

This is why many first agents are built backwards. They are given a broad connection because it is easier to set up, and the limits are meant to come later from the prompt. A prompt is an instruction, not an access control. If the permission exists, it can be used, so the narrowing has to happen where the access is granted.

Irreversible actions need a person

When an agent does need to act, split the actions by reversibility. Drafting a reply, adding a row to a sheet or tagging an email can be undone, and can reasonably run on their own. Sending, paying, ordering and deleting cannot. For those, the agent prepares the action and a person approves it.

This keeps the division of labor honest. The agent does what people do badly, which is checking the same source every day without losing attention. The person does what the agent cannot, which is judging whether a given change matters to the business.

A dry run is an evaluation of your rules

Before widening any permission, run the agent in dry-run mode. It reads, applies the rules and reports what it would have flagged, but touches nothing. Many fields have a native version of this. In trading it is called paper trading, where a strategy runs against real conditions with no real money.

Treat the dry run as a test of the rules, not of the model. The metric is alert precision: of the alerts it sent, how many were worth your attention, and how many were noise. Also note what it missed that you caught yourself. Two weeks of alerts is usually enough to see a pattern.

Then tighten the rules and run again. Low precision almost always means a vague rule, a threshold set too close to normal variation, or a source that changes layout often. Only when precision has held for a stretch do you extend permissions, one step at a time.

One more rule belongs in the instructions: if nothing matched, say nothing. An agent that reports "no change" every morning teaches you to ignore it within a week. Silence only carries information if the agent speaks when it should.

An example, kept narrow

Take a procurement team that needs to know when a supplier changes a price list or a public tender appears in its field. The agent reads the supplier's page and the tender listings daily, compares them with yesterday, and messages the owner only when a rule fires. It has read access and nothing else. Placing an order stays with a person.

The same structure applies to any monitored source, including market data. Nothing here is investment advice. This describes how to build the agent, not what it should watch or what to do with its alerts.

What to do this week

Pick one source you check by hand. Write the one sentence that describes the change that would make you act. Give an agent read access to that source and a schedule, run it in dry-run mode for two weeks, and count the precision. If you cannot write the sentence, the rules are the work, and the agent was never the hard part.

Claude training for teams

Want your team working this way?

This is the material we teach in workshops, at your office or online. Tell us what you're trying to solve, and we'll come back with a plan.