For most of my career, I was an individual contributor. Usefulness was visible: write the code, review the change, fix the problem, publish the document. The output was easier to see.

Moving into management changed that relationship before I had a good way to think about the calendar. A 1:1, a proposal review, a stakeholder question, and an escalation can all be legitimate work. Put enough of them together, though, and the week disappears before the decisions and conversations that require a lead have room.

The transition problem starts when leadership work arrives before old IC work leaves. Implementation, routine review, technical coordination, and familiar context remain attached through habit; people, delivery, feedback, stakeholders, and risk arrive on top. This is not a time-management problem in the narrow sense. It is a responsibility problem expressed through the calendar.

This guide is for new engineering managers and technical leads whose outcomes increasingly depend on other people while they still carry part of their old role. Their authority differs, but the practical question is the same: which work genuinely requires your attention, and which work is still yours because it used to be?

My operating rule is simple: when leadership work enters your week, work from your old job needs to leave. A transition may require temporary overlap, but temporary overlap needs an end date. Otherwise it quietly becomes two permanent jobs.

Audit the week you actually have

For an initial audit, do not start by designing an ideal calendar. Start with the previous ten working days.

Open your calendar, Slack, notes, and whatever you use to track work. Account for visible commitments and the work around them: preparing feedback, reviewing proposals, answering questions, writing follow-ups, recovering context between meetings, handling an incident, or finishing something after hours because it never found space during the day.

You do not need a new time-tracking system. The goal is not minute-by-minute surveillance. You are looking for enough evidence to answer five questions:

1. What took meaningful time?

2. What outcome did it create?

3. Why did it require me?

4. Who waited for me?

5. Should I own it, stay close to it, transfer it, or stop it?

A simple audit might look like this:

The “Why me?” column is uncomfortable because familiarity can masquerade as responsibility. You may be the fastest reviewer because you know the codebase. You may be the best person to rewrite a plan because you have stronger opinions about its structure. Neither fact automatically means the work should remain yours.

Look for three forms of calendar pressure:

• Hidden leadership work: feedback, tradeoff decisions, stakeholder preparation, and thinking that gets pushed outside working hours.

• Habitual IC work: implementation, routine review, or technical coordination you still own because you used to own it.

• Recurring demand: meetings, approvals, and repeated questions that keep returning because the underlying ownership or context is unclear.

At the end of the audit, name the three biggest sources of pressure. Do not fix all of them yet. The audit separates what feels busy from what is making the team dependent on you.

Stop carrying your old job

Push each recurring responsibility to the lowest level of involvement that is still safe. Then sort it into four groups.

Own. Work that requires your authority, accountability, or sensitive judgment: a performance conversation, a priority conflict, a cross-team commitment, a high-risk escalation, or a decision where the team needs one accountable owner.

Stay close. Work where you need enough context to exercise judgment without controlling every step: an important launch, a migration with operational risk, architecture with long-term consequences, or a customer problem that could change team priorities.

Transfer. Give another person a complete outcome when the work can develop their judgment and reduce unnecessary dependence on you: routine reviews, project coordination, implementation planning, recurring documentation, or a technical proposal.

Stop. Meetings without an output, duplicate reports, inherited approvals nobody can justify, and processes that cost more attention than the risk they manage.

These categories do not make technical work inappropriate. A technical lead might keep implementation when it reduces important risk, establishes a reusable pattern, or supplies context that would be expensive to gain another way. An engineering manager might stay close by inspecting risk and decision quality without becoming the routine reviewer or implementer. The test is whether lead-level attention changes the outcome, not whether the work is technical.

Treat Own as the uncomfortable “only I” list, then challenge every item in it. Does it truly require your authority? Is the information sensitive? Is the decision hard to reverse? Is the risk high enough that you need to own the call? If the answer is no, the work may need your context or a checkpoint, but it probably does not need your hands.

Do not remove yourself so aggressively that high-risk work loses necessary context. Transferring a security-sensitive change and then disappearing creates an accountability gap. Match your involvement to the person, the risk, and the reversibility of the decision.

For the next week, choose one item to stop, one to transfer, and one to stay close to through a lighter mechanism. If Own still contains half the role, go through it again. You may be protecting a preferred method rather than a necessary responsibility.

Budget your week before it fills

Once you know what should remain yours, build the week around responsibilities rather than incoming requests.

Start with the commitments that are genuinely fixed: 1:1s, team planning, interviews, an operating review, or a launch decision with an external deadline. Add preparation time for the commitments that need it. A one-hour feedback conversation is not a one-hour piece of work if you need thirty minutes beforehand to review examples and decide what you actually want to say.

Then add the work that is important but easy to displace:

• decisions and tradeoff preparation;

• consequential technical context;

• difficult conversations;

• written context that will prevent repeated explanation;

• delegation checkpoints;

• support or escalation requests to your manager.

Some of this work deserves a name: judgment time. It is where you compare weak signals, prepare consequential feedback, inspect technical risk, and decide which tradeoff you are willing to defend. Without a named block, visible tasks and incoming requests can repeatedly displace it.

Create an operating reserve. As a starting experiment, reserve 10–20% of your sustainable week if you can, then adjust it through the weekly audit. A team with frequent on-call interruptions may need more. A lead with little control over the calendar may begin with one open half-day. The point is to treat incidents, escalations, and work that runs long as normal operating conditions rather than planning failures.

The arithmetic has to work. If your plan contains forty-six hours of commitments, a better color scheme will not turn it into a forty-hour week.

Consider a hypothetical new engineering manager with seven reports, a migration in progress, and normal on-call exposure:

The numbers are illustrative. A tech lead may need more technical time; a manager with a larger team may need more people-management time. The important change is what left, not the allocation itself.

Before moving on, make the budget balance:

1. Subtract fixed commitments, preparation, and the operating reserve from your sustainable weekly capacity.

2. Allocate what remains across people, decisions, technical context, stakeholders, and administration.

3. Remove, reduce, or transfer work until the total fits without relying on evenings.

Block your time

Calendar blocks help only when they protect a named outcome. “Focus time” is too vague. It is easy to overwrite because nobody, including you, knows what would be lost.

Use specific names:

• Review the migration risk and decide whether to phase the launch.

• Prepare feedback for Thursday’s 1:1.

• Read the API proposal and write the decision boundary.

A useful block also needs an interruption rule and a response window. For example:

I am using 9–11 to prepare feedback and make the release decision. Interrupt for a production incident, an urgent people issue, or a decision holding up several people. Otherwise, add it to the queue and I will respond by 2.

This keeps protected time from becoming unavailability. The team knows what may interrupt it and when other requests will receive attention.

Block three things in the next week: one decision, one consequential conversation, and one technical risk. That is enough to test the shape without turning the calendar into a fortress.

Do not protect personal output while the team waits for a decision only you can make. If five engineers are blocked on a release call, move the writing block, make the call, and restore the time somewhere visible. Protect important work from accidental displacement, not every calendar event.

If the same “temporary” meeting overwrites a block every week, the block is not protected and the meeting is not temporary. Resolve the conflict instead of maintaining the fiction.

Read more

This resource turns that transition into a structured table. It groups the work into clear management areas, names the expectation behind each topic, and gives you space to record evidence, context, and your next move.

Who this is for

First-time engineering managers and experienced managers entering a new team who need a practical way to decide what deserves attention now, what can wait, and what evidence would show progress.

What is inside

• More than 60 management expectations across team and trust, delivery and quality, collaboration and communication, direction and planning, and manager growth

• A clear Topic and Expectation for every row

• A Notes and examples column for evidence, context, or your next action

• An interactive Done checkbox for each expectation

• Review guidance for choosing a few priorities and revisiting them around days 30, 60, and 90

The point is not to check every box. Review it with your manager, choose a few priorities at a time, and adapt it to your business context, team structure, and team dynamics.

Get the checklist

Read more

Who this is for

Senior engineers and engineering leaders making decisions that create meaningful work, risk, or maintenance after implementation.

What is inside

• A decision log with the date, status, owner, and revisit date

• A repeatable worksheet for the problem and options considered

• Prompts for immediate benefits, costs that may appear later, and reversibility

• Space to record the eventual outcome

Get the template

Read more

Who this is for

Use it when starting a Staff or tech lead role, taking responsibility for a new initiative, or resetting expectations after the role has begun to drift.

What is inside

• The role and why it exists

• No more than three expected outcomes

• Clear ownership, shared decisions, and out-of-scope work

• Technical involvement, support, capacity, and measures of success

Get the template

Read more

Who this is for

Senior engineers and engineering leaders who want more than a one-time career worksheet. Use it when you know you want to grow but need a clearer way to connect that direction to real work, evidence, and regular review.

What is inside

• A career direction section for the work you want more and less exposure to

• Two or three objectives for the next 6–12 months

• A focused 90-day responsibility or skill to practise through real work

• A five-field goal tracker for Area, Status, Target date, and Next review

• A reusable goal page for actions, evidence, support, and review notes

Get the template

Read more

Who this is for

Use it if you are a senior engineer whose impact spans projects, decisions, mentoring, incidents, or cross-team work. Add an entry after meaningful work, then return to the log before career conversations and performance reviews.

What is inside

• A simple log with Entry, Date, Category, Scope, and Evidence

• A repeatable entry page covering Situation, What I did, Result, Evidence, and What this shows

Get the template

Read more

Who this is for

Engineering managers and other people leaders who want simpler meeting notes and clearer follow-through. Use it when useful context and action items are getting lost between conversations.

What is inside

• A meeting tracker with the date, an optional feeling check, and action-item status

• A repeatable agenda for what is going well, what is not going well, and feedback

• Dedicated space for action items and additional notes

Get the template

Read more

A product lead wants a natural-language setup flow for a developer tool. One engineer thinks it could remove most of the friction from the first integration. Another worries that authentication, error recovery, and the current SDK contracts will make the experience brittle. A staff engineer sees a different risk: if the demo looks convincing, it may become a roadmap commitment before the team understands what it would take to operate.

The team could spend another week improving the design document, or it could build a narrow prototype and put the disagreement in front of real evidence.

AI changes the economics of that choice. For throwaway prototypes, I have felt the speed gain directly: a proof of concept that once took two or three days can now land in an afternoon. That makes implementation cheap enough to use during the decision process, not only after the decision has already been made.

But AI makes the artifact cheaper, not the evidence. Once people can click through the flow, call the API, or watch the agent complete a task, the discussion shifts from “Should we build this?” to “What would it take to ship what we already have?”

The artifact has started answering a question nobody agreed to ask.

My recommendation is to build the disagreement when implementation can produce evidence, but treat the prototype as an expiring question rather than an early product. Before anyone prompts an agent, define what the prototype may prove, which shortcuts limit that conclusion, when its mandate ends, and what happens to the implementation afterward.

None of this is unique to AI. “What Do Prototypes Prototype?” frames prototype focus as a choice about which open design questions to examine, while “The Anatomy of Prototypes” describes prototypes as filters over a broader design space. AI changes how quickly a functional, convincing artifact can appear—and, I would argue, the organizational pressure to treat an answer-shaped artifact as a product.

When this rubric is worth using

Not every sketch needs a formal hypothesis. If two engineers are exploring a low-risk implementation detail for an hour, let them explore.

Use the rubric when the artifact could influence:

• roadmap scope or a customer promise;

• an architectural or integration decision;

• a production or security boundary;

• a cross-team commitment;

• a claim about user behavior;

• whether generated code becomes part of a maintained system.

Subscribe now

The point is proportionality. Five lines written before two days of implementation are not heavy governance. They protect the team from spending the next six months maintaining an answer to the wrong question.

The prototype decision card

Every consequential prototype should begin with five fields:

1. Question

2. Evidence

3. Shortcuts

4. Expiry

5. Disposition

This is not a new approval process. The decision boundary must travel with the artifact. Keep the card inside the issue, design document, or pull request description, and put the question, expiry date, and status on the prototype itself. A forwarded link, screenshot, or recorded demo should still say what the artifact was designed to test and whether it is approved for production or customer commitments.

Prototype for: Expired-credential recovery

Expires: 18 July

Status: Not approved for production or customer commitments

Decision card: Linked with this artifact

1. Question: What decision should change?

Name one decision the prototype will inform.

“Can we build this?” is usually too broad. Almost anything can be made to work once under favorable conditions. Narrower questions force the team to identify the disagreement underneath the implementation:

• Can a target developer finish the first integration without leaving the guided flow?

• Does the current API contract support recovery from an expired credential?

• Can this architecture meet the latency target when one dependency degrades?

• Can the agent handle the representative failure cases without taking an unsafe action?

A useful question has consequences. Before building, state what the team will do after a positive result and what it will do after a negative one:

If the result is positive, we will [action]. If it is negative, we will [action].

If neither result would change the decision, the team is not running an experiment. It is producing a demonstration. If the evidence lands between the predeclared criteria, call the result inconclusive and write a new card rather than rewriting the first question.

2. Evidence: What would count as an answer?

State what the team will observe or attempt, who will do it, and which results will count as positive, negative, or inconclusive. Define those criteria before seeing the artifact, so the team cannot move the goalposts after the prototype starts persuading people.

The evidence must match the question. A usability claim requires target users attempting the task. An operational claim requires load, failure, recovery, or maintenance evidence. A claim about agent reliability requires a representative evaluation set, not one impressive trace selected for the demo.

In “Prototyping with Prompts”, a CHI 2025 design study, researchers observed 39 industry practitioners across 13 team sessions as they prototyped prompts for a generative AI marketing application. The authors describe two layers of evaluation: teams assessed output validity and correctness while considering whether end users could understand and meaningfully interact with it. They also found that assumptions about cultural norms, user behavior, content formats, and conceptual abstractions could remain implicit and undocumented. This suggests that greater fidelity can expand what a team is able to inspect, but it does not determine which evidence a decision requires.

3. Shortcuts: What is intentionally unlike production?

List every condition that is fake, stubbed, omitted, unreviewed, not tested at scale, incomplete, or unusually favorable.

Typical shortcuts include:

• mocked billing or permissions;

• a test tenant instead of a real account;

• one supported language or environment;

• clean data that excludes known edge cases;

• no production secrets or security review;

• a single happy path;

• manual intervention hidden behind an automated-looking flow;

• generated code nobody has reviewed for maintainability.

Shortcuts are not a failure. They are what makes a prototype cheap. But a shortcut may only narrow the conclusions the team can draw; it cannot waive non-negotiable privacy, security, legal, or safety requirements.

Mocked billing may be irrelevant to a navigation test and fatal to an integration-feasibility claim. A test tenant may be enough to inspect the interaction and useless for understanding production permissions.

For each shortcut, ask: “Which conclusion are we no longer allowed to draw because this is fake?”

4. Expiry: When does the experiment end?

Give the prototype a time, budget, or learning limit.

Examples:

• two days of implementation;

• five user sessions;

• one load-test cycle;

• twenty representative agent tasks;

• confirmation of one API behavior;

• the first unrecoverable security constraint.

Without an expiry, a prototype tends to keep absorbing work. Someone adds another path because the first one looked promising. Another engineer improves the error handling. A stakeholder asks whether it can be shown to a customer. The team quietly stops learning and starts developing, but the code never passes through the decisions expected of a real product.

An expiry does not mean the idea must die. It means the current artifact loses its mandate. Continuing requires a new question, a new experiment, or an explicit production proposal.

5. Disposition: What happens to the artifact?

Decide in advance whether a positive result leads to:

• a clean rebuild;

• a separate production proposal;

• another experiment;

• retention of only the findings or test fixtures;

• a deliberate stop.

A prototype should not enter production because rebuilding feels wasteful. If the implementation is worth keeping, it should earn that decision independently through normal review, security, testing, ownership, and operational standards. The prototype label should not grant generated code a permanent exemption.

When the expiry condition arrives, append a short closeout to the card:

• Result: What happened against the predeclared criteria.

• Decision: What changes now.

• Artifact status: Deleted, archived, retained for evaluation, or proposed for independent review.

• Owner and date: Who closed the experiment and when.

The closeout is not a sixth planning field. It is the durable record that stops later readers from reinterpreting the artifact after its context has faded.

In “From Throw-Away to Takeaway”, a CHI 2026 mixed-methods study combined an online survey of 85 people with interviews of 31 hackathon participants and eight practitioners. In its sample, cloud development environments accelerated prototyping and enabled non-technical users to create high-fidelity throwaway prototypes for experiential exploration. Deployment and long-term maintainability, however, continued to depend on technical expertise. The study supports separating prototype evidence from production readiness; it does not show that this five-field card improves team decisions. The card is my operating recommendation.

Subscribe now

Match the evidence to the prototype

Vision prototypes belong in this model even when they are not experiments. Their evidence is discussion and alignment. Stakeholder enthusiasm shows that an idea is persuasive; it does not validate user need, technical feasibility, or roadmap priority.

A completed decision card

Return to the developer-tool disagreement from the opening. Instead of asking an agent to “build a natural-language onboarding flow,” the team writes this first:

Question: Can a target developer complete the first integration—including recovery from an expired credential—without leaving the guided flow, using the current API contracts?

Evidence: Five target developers attempt both a clean setup and a scripted expired-credential case in a faithful test environment. A positive result means at least four complete both cases within fifteen minutes without manual backend intervention. A negative result means the current contract cannot support safe in-flow recovery, or more than one participant reaches an unrecoverable error. Any other result is inconclusive. The team records completion time, confusion, help requests, manual intervention, and unrecoverable errors.

Shortcuts: Test tenant only. Billing is mocked. One language is supported. No production secrets or customer data are used. The prototype makes no claim about scale, security approval, or maintainability.

Expiry: Two days of implementation and five user sessions, or immediate termination if the current contract cannot support safe in-flow recovery.

Disposition: A positive result leads to a separate production proposal and an independent decision about whether any implementation is retained. A negative result preserves the findings and evaluation fixtures, then deletes or archives the prototype. An inconclusive result requires a new card.

Now the team knows both what success means and what would falsify the experiment. A beautiful demo cannot settle the question if no target developer attempts the integration. A successful happy path cannot hide an API contract that makes recovery impossible, because recovery is part of the evidence plan. A weak result does not automatically kill the broader idea; it may show that the current contract, not the interaction, is the next question to investigate.

After the sessions, the closeout might read:

• Result: Four of five developers completed the clean path; only two recovered from an expired credential, and three reached an unrecoverable error.

• Decision: Stop the onboarding prototype and investigate the API contract.

• Artifact status: Prototype archived; evaluation fixtures retained.

• Owner and date: Prototype lead, 18 July.

The failure modes to watch

• The sales demo is built to persuade and later cited as validation. Label vision and sales artifacts explicitly, including what they were never designed to test.

• The happy-path proof shows that something can work once and becomes evidence of production readiness. Record which properties—load, recovery, security, observability, and maintainability—remain outside the experiment.

• The moving question appears when the team rewrites the hypothesis after seeing the result. Preserve the original card; if the artifact reveals a more useful question, create a new one.

• The evidence mismatch uses the wrong people or conditions. Internal stakeholders cannot validate customer usability, and a mocked API cannot validate current integration behavior.

• The accidental foundation begins with “We already have most of it.” Require an independent production decision. Do not measure waste by lines of prototype code deleted.

• The orphan prototype survives after the decision with no owner or status. Make cleanup part of the disposition and record the artifact’s status in the closeout.

What to do before the next prototype

Before anyone starts building, ask the team to spend ten minutes on the decision card.

During the experiment, record observations and newly discovered shortcuts. Do not expand the prototype merely because implementation is going well.

When the expiry condition arrives, stop and complete the closeout. A positive result should change only the decision named on the card. A negative result is useful when the test was valid. An inconclusive result means the question or evidence plan needs another pass.

AI makes it cheaper to turn disagreements into things a team can inspect. The prototype should earn authority from the evidence it collects, not from how much it resembles a finished product.

Before the next agent starts generating code, decide what its artifact is allowed to prove—and what your team has already agreed to do when the question expires.

That question can be healthy. Some engineers still underuse tools that would remove boring work, surface options, or force a sharper explanation of the problem. I do not think engineering leaders are wrong to expect people to learn AI. Refusing to build any muscle with these tools is becoming its own professional risk.

The problem starts when “AI-first” becomes a test of seriousness. Once the expected answer is visible, people learn to produce that answer. They mention the tool in planning. They add it to the workflow where it can be seen. They show the demo. They write the performance-review paragraph. The company gets evidence of adoption before it gets evidence that the work improved.

Luiza Jarovsky’s essay, When AI Becomes a Religion, is too broad for the exact piece I want to write, but it names a pattern worth taking seriously inside companies: salvation language, dogma, and the awkward role of the heretic. I would not import the whole religion frame into engineering. It gets noisy quickly. The workplace question is what happens when skepticism starts to carry a social cost. The engineer who asks for evidence, slows a rollout, or says the generated code made review worse can start to look behind the culture, even when they are protecting the feedback loop the organization needs.

Shopify is a useful public example because it turns the cultural pressure into an operating rule. In April 2025, coverage from Business Insider and The Verge described a memo from CEO Tobi Lütke that made AI usage a baseline expectation, asked teams to show why work could not be done with AI before requesting more headcount or resources, and added AI usage questions to performance and peer review. There is a reasonable version of that management instinct. Headcount should not be the default answer when tools and workflow changes can absorb some of the work. A serious organization can expect people to try available leverage before asking for more capacity.

The failure mode starts when proof of trying becomes more important than the quality of the result. If a team knows the safest answer is “yes, we used AI,” then people will learn how to produce that answer. They will add the tool to the workflow, mention it in planning, show a demo, or include it in a review packet. That may be sincere learning. It may also be adoption theater. From the outside, both can look similar unless leaders measure what happened downstream.

When usage becomes safer than judgment

Usage is attractive because it is legible. Licenses activated, prompts submitted, features shipped with AI assistance, agents connected to repositories, demos shown in all-hands meetings. These numbers make the rollout visible. They do not prove that the organization is building better software.

The Stack Overflow 2026 pulse survey is useful precisely because it holds two ideas together. Workplace agent usage had risen: 59% of respondents said they used agents at work, nearly double the 31% Stack Overflow reported from its 2025 Developer Survey. At the same time, 63% of technologists said they rarely or never let agents run fully on autopilot. The survey was self-reported and had 1,100 respondents, so I would treat it as directional. Still, the tension is the one engineering leaders need to understand. Adoption is real, and supervision is still doing a lot of the work.

If leadership celebrates usage while the work still requires supervision, the supervision still has to happen somewhere. It usually lands with senior engineers, reviewers, tech leads, and managers who have to decide whether the generated output is correct enough to ship. The prompt may be faster than writing the first draft of the code. The review can still become slower, especially when the author cannot explain the change, the test is brittle, or the diff is plausible in ways that make mistakes harder to spot.

Moreover, DORA’s 2025 AI-assisted software development report pushes the conversation away from tool access alone and toward the system around the tool. Local productivity gains matter only if they translate into product performance instead of downstream chaos. A team can produce more code-shaped output and still make the engineering system worse.

There is also a social layer. AI tools are not neutral calculators inside a team. OpenAI’s post on sycophancy in GPT-4o is a reminder that model behavior can affect trust: the company rolled back an update after GPT-4o became too flattering and agreeable. A tool that agrees too easily can make a weak plan feel stronger than it is. A company culture that rewards visible AI enthusiasm can do the same thing from the other side. The model says the plan is good. The dashboard says adoption is up. The executive story says the team is modernizing. The reviewer who says “this is not ready” becomes the inconvenience.

Subscribe now

What I would measure instead

If I were accountable for an AI rollout, I would not ask teams to prove they are bought in. I would ask where AI made the engineering system measurably better and where it shifted cost out of sight.

For code review, I would look at whether reviewers are catching better issues or just reading more generated diffs. For incident response, I would look at whether AI-assisted summaries improved diagnosis without flattening uncertainty. For onboarding, I would look at whether new engineers reached useful context faster and whether they could still explain the systems they touched. For migrations and maintenance work, I would look at whether generated changes reduced toil without increasing the burden on the people who own the code after the migration lands.

Those questions are slower than adoption dashboards, but they are closer to the work. They also leave room for the honest positive case. Sometimes AI will be the right answer. It can remove blank-page friction, draft tests, summarize unfamiliar code, produce a first pass at support tooling, and make a senior engineer faster when the senior engineer still owns the judgment. The critique is aimed at pressure that creates bad data.

A useful leader should want the bad data. Failed prompts, wrong assumptions, unreviewable patches, brittle generated tests, and moments where the tool made someone faster but less clear about the work are not signs of disloyalty. They are rollout telemetry. If people hide those signals because skepticism sounds like resistance, the organization loses the information it needs to adopt AI well.

Protect the useful skeptic

In an engineering organization, the useful skeptic is often not trying to stop adoption. They are asking the question that keeps adoption honest. What got better? What got worse? Who absorbed the review cost? Which failure mode disappeared, and which one moved to production? Did the team learn a reusable workflow, or did one person get a private boost that nobody else can inspect or improve?

That distinction matters for senior engineers. A senior engineer who pushes back on a shallow AI mandate should not sound anti-tool. The stronger posture is more demanding: show me the outcome, show me the review path, show me what we learned, and show me where the tool made the system less reliable. That is not nostalgia for pre-AI engineering. It is the production discipline we should have been using all along.

It also matters for managers. If “AI-first” becomes a loyalty test, managers will get the answers they trained the organization to produce. People will describe their work in AI-friendly language. They will use the tool where the tool is visible. They will avoid being the person who sounds slow, difficult, or unconvinced. The organization may feel aligned while its feedback loops get worse.

The healthier version starts with the work: the workflow, the risk, the customer outcome, the review burden, and the learning the team needs to keep. Then ask where AI helps and where it creates a new failure mode. Engineers do not need permission to be curious about AI. They need permission to report the truth about it.

That is the line I would want leaders to protect. Push people to learn the tools. Do not punish the people who keep the evidence honest. In a serious engineering organization, the person who refuses to turn usage into faith may be part of the control system that makes adoption work.

We built chains, graphs, routers, callback handlers, retrievers, memory abstractions, evaluation layers, and handoff nodes. Some of that was necessary. If an AI system moves money, changes permissions, updates customer data, deploys software, or touches production infrastructure, the hard boundaries belong in code. Nobody should trust a paragraph in a Markdown file to enforce access control.

But we took that instinct too far. A lot of AI application behavior is not a hard boundary. It is procedure, policy, tone, source discipline, review expectations, escalation rules, examples, and handoff shape. Those are the parts of a workflow that change often, depend on domain judgment, and are usually maintained by people who should not have to modify orchestration code to make the product behave better.

That is the mistake I think the first wave of AI apps made: we treated too much of the AI layer as an orchestration problem. The future is simpler than that. Code should provide the harness. Markdown should carry much more of the work.

That is what I mean when I say Markdown is becoming the application layer of AI apps. I do not mean Markdown replaces Python, TypeScript, Go, SQL, permissions, tests, queues, schemas, or production controls. I mean the part of the product that tells the AI system how to behave is moving out of framework code and into written, versioned instructions that humans can read and agents can execute.

The transition is from code-heavy orchestration to harnesses plus Markdown.

The First Version Was Too Much Code

The framework-heavy approach made sense at the beginning. Early AI applications needed scaffolding because the raw model interface was too loose. Retrieval needed structure. Tool calling needed wrappers. Production teams needed traces, retries, typed inputs, fallbacks, and evaluation hooks. LangChain and LlamaIndex became natural choices because they gave engineers a way to turn an uncertain model interaction into something that looked more like software.

I do not want to turn those frameworks into cartoon villains. They are useful libraries. LangChain has real orchestration and integration machinery. LlamaIndex is useful for indexing, parsing, retrieval, and data-connected applications. Both ecosystems support Markdown as an input format, whether through LangChain’s UnstructuredMarkdownLoader or LlamaIndex’s Markdown node parsers.

The problem is not that these tools exist. The problem is the reflex they encouraged: when an AI workflow becomes important, encode the next decision as another node. That works when the workflow is stable and the branches are genuinely software branches. It gets awkward when the workflow is really a written policy that people keep learning how to improve.

A support escalation rule should not always require a framework change. A source policy should not be buried in a callback. An editorial standard should not live as a prompt string in application code. A review checklist should not be scattered across several agent nodes. When those things live in code, the people closest to the domain lose the ability to improve them directly. The workflow becomes more formal, but not necessarily more maintainable.

This is how teams end up with an AI system that is technically sophisticated and operationally clumsy. The code can call the model, fetch the documents, route the task, and produce an answer. But changing what “good” means still requires spelunking through framework glue, prompt fragments, and hidden assumptions.

Subscribe now

The Simpler Shape

The simpler architecture has three parts.

First, product code still owns capabilities and constraints. It defines the APIs, data access, permissions, transactions, schemas, audit logs, deployment paths, and anything else that must be reliable even when the model is wrong.

Second, the agent harness gives the model a safe place to operate. The harness decides which tools exist, which files can be read, what state is available, how approvals work, how traces are captured, how evals run, and where the system should stop instead of guessing.

Third, Markdown carries the operating behavior. This is where the workflow lives: the source policy, the escalation rules, the examples, the templates, the product vocabulary, the review expectations, the domain caveats, and the handoff contract.

That split matters because it puts the right kind of change in the right kind of medium. If the change must be enforced, it belongs in code. If the change teaches the AI system how the team wants work done, Markdown is often a better place for it. It is easier to review, easier to diff, easier to search, and easier for domain owners to maintain with engineering guardrails around it.

This is not no-code. It is not “let the prompt handle it.” It is a different ownership model: engineers build the harness and enforce the boundaries; the Markdown layer describes the work the harness should perform.

What Moves From Code To Markdown

The useful way to think about the transition is not “which language are we using?” The useful question is “what kind of decision is this?”

Code should own the decisions that need deterministic enforcement: permissions, data writes, destructive actions, money movement, external side effects, compliance checks, schema validation, deployment, rollback, and anything where being merely persuasive is not good enough.

Markdown can own a different class of decisions: how to investigate a bug, how to write a release note, what sources count as reliable, when to escalate a customer issue, what a good support answer looks like, how to compare devices, how to produce an editorial brief, how to prepare a pull request handoff, or how to explain a product feature without overpromising.

Those second-order decisions are still important. In many AI products, they are the product. A customer assistant that follows the wrong escalation policy is not a small UX problem. A research agent that treats vendor claims as neutral evidence will produce bad work. A coding agent that writes a migration without the team’s review expectations is creating risk. But the way to improve those behaviors is not always another code path. Often it is a clearer workflow, a better example, a tighter source policy, or a more explicit handoff template.

That is why Markdown matters. It is the medium where those instructions can become part of the application without disappearing into framework glue.

Why This Is Happening Now

Vercel’s Eve is the cleanest public example of this direction. Eve describes itself as a filesystem-first framework for durable AI agents. A typical Eve agent has an agent/instructions.md file as the always-on prompt, optional Markdown skills that can be loaded on demand, typed tools, channels, and schedules. The important thing is the authoring surface: the workflow is not primarily a visual graph or a chain of Python objects. It is a filesystem with Markdown instructions and skills.

OpenAI and Anthropic are moving toward the same split from the harness side. The OpenAI Agents SDK gives applications primitives for agents, tools, handoffs, guardrails, sessions, tracing, and human approval. The Codex SDK lets applications control local Codex agents programmatically. Anthropic’s Claude Agent SDK exposes the agent loop, tools, permission model, session management, MCP support, hooks, subagents, and filesystem-based configuration that power Claude Code.

Those SDKs are not Markdown frameworks. That is not the point. The point is that the harness is becoming product infrastructure. Once a product has a capable harness, the next question is where the product-specific behavior should live. My bet is that a lot of it will live in Markdown.

The instruction file conventions are already normalizing. Codex reads AGENTS.md for project instructions. Claude Code uses CLAUDE.md for persistent project memory and can import AGENTS.md to avoid duplicated guidance. Google’s Open Knowledge Format draft describes knowledge bundles as directories of Markdown files with YAML frontmatter, meant to be readable by humans, parseable by agents, diffable in version control, and portable across tools.

These are not the same thing. AGENTS.md, CLAUDE.md, Eve’s instructions.md, llms.txt, OKF bundles, and framework document loaders all have different scopes and loading rules. But they point toward the same underlying shape: humans write structured text, agents use it as operating context, and the product changes when that text changes.

There is also research pushing in this direction. The March 2026 paper “Interpretable Context Methodology: Folder Structure as Agentic Architecture” argues that, for sequential workflows with human review, folder structure and Markdown prompts can replace some framework-level orchestration. I would not stretch that into a universal rule. Complex concurrent systems still need real orchestration. But the paper matches a pattern many teams will recognize: a lot of useful work is sequential, review-heavy, and easier to inspect when the process is visible in files.

Subscribe now

What This Looks Like In Practice

My own content system has pushed me toward this view. The Long Commit has a private Markdown operating layer: root instructions, voice rules, research standards, workflow files, templates, source policies, and Notion handoff rules. One workflow follows the sources I care about and produces a daily brief. The brief is not the article. It is a prepared starting point: what happened, where the primary sources are, what might be worth reading, and where the evidence is weak.

Then I read the sources and write the piece myself.

The useful part is that I can improve the system by editing the Markdown layer. If the brief overweights vendor claims, I change the source policy. If the output starts sounding generic, I change the voice guide and add examples. If the handoff misses caveats, I update the template. I am not rebuilding an orchestration graph every time my editorial process gets sharper.

The same pattern applies outside writing. For a smart home site like motherhome.io, Markdown workflows can define how to research a new device, which sources matter, how to compare Matter support, and what claims need caveats before publication. For Auth0-related technical work, Markdown can hold SDK references, implementation patterns, documentation paths, and examples of correct integration. For home automation, Markdown can describe what actions are safe, which commands require approval, and what the agent should never infer.

The domains are different, but the architecture is the same: the harness provides capabilities; Markdown carries the domain behavior.

A project might look like this:

ai-app/
├── AGENTS.md                    # Root router and shared agent policy
├── CLAUDE.md                    # Compatibility shim: import AGENTS.md
├── agent/
│   ├── instructions.md           # Runtime-specific always-on prompt
│   ├── skills/
│   │   ├── support-triage.md      # Procedure loaded when needed
│   │   ├── research-brief.md
│   │   └── release-note.md
│   └── tools/                    # Code: typed capabilities and integrations
├── profile/
│   ├── product.md                # What the product is and who it serves
│   ├── domain.md                 # Vocabulary, assumptions, edge cases
│   └── voice.md                  # Tone, naming, UX copy, audience
├── workflows/
│   ├── bugfix.md                 # How this team investigates and fixes bugs
│   ├── content-update.md         # How content gets researched and edited
│   ├── customer-escalation.md     # When to escalate, pause, or ask
│   └── deploy.md                 # Deployment workflow and human gates
├── policies/
│   ├── source-policy.md          # What counts as evidence
│   ├── tool-boundaries.md        # Which actions are allowed or dangerous
│   └── data-handling.md          # Privacy, retention, customer data rules
├── templates/
│   ├── pr-description.md         # Output contract for PRs
│   ├── handoff.md                # Output contract for human review
│   └── decision-record.md
├── examples/
│   ├── good-output.md            # Local taste, with reasons
│   └── bad-output.md             # Failure modes to avoid
└── evals/
    ├── cases.md                  # Scenarios the AI layer must handle
    └── rubric.md                 # What good behavior means

The folder names are less important than the separation of concerns. AGENTS.md should be a router, not a dumping ground. It should tell the agent what kind of project this is, which workflows exist, when to load them, what actions require approval, and where the source of truth lives. If a vendor-specific file is required, make it a compatibility file. My preference is that AGENTS.md becomes the boring root standard, with CLAUDE.md or instructions.md pointing back to it when possible.

The workflows should encode judgment, not only steps. A useful workflow tells the agent when to research, what evidence counts, when to stop, what output shape to produce, and what review must happen before the work is treated as done. Templates turn those outputs into contracts. Examples give the system taste. Policies and evals make expectations inspectable.

That is the application layer I think many AI products are missing. They have models. They have tools. They have framework code. They do not yet have a maintainable place where the team’s judgment about the work can live.

Share The Long Commit

The Part Teams Cannot Skip

Markdown can also make a weak process look official. A stale AGENTS.md is worse than no AGENTS.md if the agent trusts it. A source policy that says “use reliable sources” is not a source policy. A workflow file full of vague advice is not a workflow. A compatibility file that quietly diverges from the real instructions creates another source of truth. A prompt that asks the model not to do something dangerous is not a substitute for a permission boundary.

Instruction files do not automatically improve agent output either. The June 2026 paper “Toward Instructions-as-Code”studied 15,549 agentic pull requests across 148 projects and found mixed results after instruction files were added. Some projects improved; others worsened. That is exactly the kind of result I would expect. The presence of a Markdown file does not prove the team has designed a good AI layer. It only proves there is now a place where good or bad instructions can affect the system.

This is why engineering ownership matters. If a Markdown file changes how the AI product behaves, then changing that file is a product change. It should have owners. It should be reviewed. It should have examples. The important workflows should have eval cases. Stale instructions should be deleted. Vendor-specific files should not multiply into five slightly different policies because every tool wants its own filename.

The future I am arguing for is simpler, not looser. It has less orchestration code for things that should have been written procedure, and stronger code around the parts that need enforcement. It gives domain owners a real way to improve behavior without pretending that prose can replace permissions, tests, or production controls.

The transition from code to Markdown is not a retreat from engineering. It is engineering putting the right work in the right layer.

If I were leading an AI product team, I would ask one practical question before adding another framework node: is this a capability the system must enforce, or is this behavior the harness needs to understand? If it is enforcement, write code. If it is behavior, policy, examples, or handoff, start by designing the Markdown layer.

That is the architecture I would bet on: code for capabilities and constraints; Markdown for the operating behavior the harness can execute.

Production usually tells the longer version. The review queue gets heavier. The same two senior engineers become the validation layer for a larger volume of plausible patches. Support sees more small changes with surprising edge cases. The team ships more often, but on-call starts to feel more expensive. None of this proves the AI rollout failed. It proves the dashboard stopped too early.

That is the standard I would use for this debate: AI can make code cheaper. It cannot make a weak idea useful, and it cannot make production work free.

The AI productivity conversation is still too comfortable measuring the part of software work that AI makes easiest to see. Lines of code, pull requests, story points, and deployment frequency are all close to production of work. They are not the same as value. A team can generate more code, open more pull requests, ship more frequently, and still leave customers with worse software and engineers with a more fragile system to operate.

I do not care very much whether AI helped a team produce more lines of code. Story points were already a weak proxy before implementation became cheaper. Deployment frequency is useful as a delivery capability signal, but it is not a product outcome. Even feature count can lie if the team is shipping work customers do not need. The harsher question is whether the team shipped something that made sense for users and whether the delivery system absorbed the change without pushing hidden cost into review, incidents, support, security, or maintenance.

If the answer is no, the team did not become more productive. It found a “cheaper” way to create activity.

The old cost was a filter

Dax Raad, who created OpenCode, made a useful version of this argument in a February 2026 post that Business Insider covered. His point was not that AI coding tools are useless. It was that code production was often not the real constraint. In one sharp line, he wrote that “ideas being expensive to implement was actually helping.”

That is uncomfortable because it names something engineering organizations do not like to admit. Implementation cost was a filter. Not a fair filter, not always a good filter, and often a frustrating one. But it forced some ideas to die before they became roadmap commitments, support obligations, security surfaces, and half-owned production behavior.

When implementation gets cheaper, weak ideas can travel farther before anyone feels the cost. The organization can build more experiments, more variants, more internal tools, more half-promised features, and more code that nobody is quite ready to own. Some of that is good. AI should make useful neglected work cheaper: tests, documentation, migrations, cleanup, internal tooling, and prototypes that were not worth a full planning cycle. But lower implementation cost also lowers the friction that used to make teams ask whether the idea deserved implementation at all.

The bill does not disappear. It moves downstream.

A shallow AI rollout makes code generation faster and then asks the same review, testing, security, release, and support systems to absorb the extra work. That is where the cost shows up: not when the patch compiles, and not when the dashboard celebrates a larger pile of merged changes, but when the organization has to review, ship, operate, recover from, and maintain the work it just made easier to produce.

Harness has a timely but imperfect signal here. Its 2026 State of DevOps Modernization report is vendor research, so I would not treat it as neutral proof. Still, the pattern is worth attention. Harness surveyed 700 engineering practitioners and managers in large enterprises in February 2026. Very frequent AI coding users were more likely to report daily or faster deployments: 45%, compared with 15% of occasional AI coding users. But 69% of that same very-frequent group said AI-generated code leads to deployment problems at least half the time. The report also found higher reported rollback, hotfix, or customer-impacting incident rates among very frequent users, and longer mean time to recovery for production incidents related to code deployments.

Harness explicitly says there is no causal proof that AI coding caused those problems. That caveat matters. The teams using AI heavily may already be under more delivery pressure, may already deploy more often, or may already have weaker delivery systems. But the caveat does not rescue the easy productivity story. If AI is adopted inside a system that cannot absorb more change, local coding speed becomes a stress multiplier. A good tool can still be deployed into an unready system.

Stack Overflow’s May 27, 2026 pulse survey points in the same direction from a different angle. Among 1,100 respondents, workplace agent use had risen to 59%, but 63% said they rarely or never let agents run entirely on autopilot. Accuracy and security remained the top concerns. This is not an argument that agents have failed. It is evidence that serious teams still treat production AI work as supervised work.

Google’s 2025 DORA AI-assisted software development report uses the right frame: successful AI adoption is a systems problem, not a tools problem. The report also says value stream management should help local productivity gains turn into product performance instead of downstream chaos. That is the sentence I would put on the wall before any AI productivity review. DORA metrics can tell you whether the delivery system is getting healthier or sicker. They cannot tell you whether the thing you shipped should exist. They are guardrails for delivery health, not proof of product value.

Subscribe now

The new work is supervision

The bottleneck does not only move. The work changes shape.

A May 2026 longitudinal study of professional software engineers found that AI coding assistants shifted work away from creation and toward verification. Participants reported spending less time on most development tasks, including 82% who reported spending less time writing code. The authors call the new category “supervisory engineering work”: directing, evaluating, and correcting AI output. They also found a productivity-experience paradox. Self-reported productivity improvements stayed high, but among matched participants, the share reporting worse developer experience in at least one dimension nearly doubled from 14% to 27%.

That matches the texture many senior engineers recognize. AI does not remove judgment. It moves judgment to a different place and then sends more material to that place.

Code review is where this becomes obvious. A 2026 vision paper on code review in the age of AI argues that AI coding assistants increase code production velocity while expanding the volume of code requiring review, turning review into a growing bottleneck unless the workflow around it changes. The paper is a research agenda rather than an outcome study, so I would not cite it as proof that every team is seeing this. But the direction is right. If the organization makes code cheaper and keeps review mostly manual, review becomes the place where the productivity story has to pay rent.

That is especially true for senior engineers. A team can celebrate more output while concentrating more validation work on the people least able to absorb it. Those engineers already carry architectural memory, production intuition, incident scar tissue, and the informal taste that keeps a codebase from turning into a pile of plausible patches. If AI increases the amount of code that needs their judgment, the organization may be spending its rarest capacity faster than before.

Siddhant Khare, who builds agent infrastructure, described the human version of this shift in his February 2026 essay on AI fatigue: “AI reduces the cost of production but increases the cost of coordination, review, and decision-making.” That is the part most productivity dashboards miss. The code may arrive faster, but the decisions around the code do not become free. Someone still has to decide whether the output is correct, safe, maintainable, aligned with the architecture, worth shipping, and worth owning.

This is why I distrust most AI productivity metrics. They usually stop counting at the point where AI looks best.

Measure past the merge

The mistake is stopping measurement where AI looks best. The useful question is what the system has to absorb after the code exists.

Lines of code, pull request counts, story points, and even cycle time can create the appearance of progress while missing the harder question: did customers get better software, and did the delivery system stay healthy? If cycle time improves because reviews get thinner, tests get weaker, or teams ship work users do not care about, the metric did its job poorly. It measured motion and missed value.

Harness’s 2026 State of Engineering Excellence report makes this measurement problem explicit, again with the same vendor-evidence caveat. Its survey says 81% of engineering leaders report increased code review time since deploying AI, 31% of a developer’s day is now consumed by AI-related invisible work, and 94% say technical debt, validation time, and developer burnout are missing from current metrics. I would not build a strategy from those exact numbers. I would take the pattern seriously: organizations are measuring output more readily than they are measuring the effort required to make that output safe and useful.

METR’s February 2026 update on developer productivity measurement is useful for the same reason. METR’s earlier randomized study found that experienced open-source developers were slower with early-2025 AI tools, but its 2026 update says the next experiment design became hard to interpret as AI adoption changed participation, task selection, quality choices, and time reporting. Developers selected different tasks when AI was allowed, changed how much documentation or testing they produced, and sometimes worked on other things while agents ran. That does not make measurement hopeless. It means the simple before-and-after story is often the least trustworthy story.

The measurement boundary has to move downstream, toward the places where software work becomes value or turns into debt. I would want a downstream ledger with five categories.

• Customer value: adoption, task completion, retention, revenue, support contacts, or whatever signal maps to the user’s life getting better.

• Delivery health: lead time, change failure rate, rollback rate, mean time to recovery, flaky pipeline pain, and release interruptions.

• Review quality: pull request size, review queue time, review rounds, senior reviewer concentration, and whether reviewers are being asked to validate code the author does not understand.

• Maintenance cost: rework, follow-up fixes, documentation drift, dead features, duplicate code, and code that becomes hard to change a month later.

• Human cost: cognitive load, on-call interruption, after-hours recovery, and whether the best engineers are spending more time supervising output than making hard technical decisions.

The exact metrics depend on the team. The principle does not: do not let the measurement stop at the point where AI looks best.

Subscribe now

Roll it out as a production change

If I were accountable for an engineering organization adopting AI coding tools, I would not frame the rollout as “make engineers faster.” That framing points everyone at the part of the system the tool can most easily accelerate, then acts surprised when the rest of the system starts to strain. I would frame it as a delivery and product quality change.

For production work, the author still owns the change. I do not care whether the code came from a model, a snippet, a Stack Overflow answer, a generated patch, or a late-night burst of human confidence. The person merging it is responsible for understanding it. “The AI wrote it” is not a root cause. It is a sign that ownership got blurry.

I would also make the risk boundaries explicit. AI-generated tests, documentation, migrations, internal tools, and boilerplate can often move faster with lower risk. Code touching authentication, authorization, payments, data migration, concurrency, incident recovery, privacy, or customer-visible behavior should not get a lighter review because a model produced it. In some cases it deserves a heavier review, because plausible code can be harder to distrust than obviously messy code.

I would protect review capacity before celebrating output. If AI increases pull request volume, the answer is not to tell senior engineers to review faster. Shrink the changes. Require authors to explain generated code before review. Track review queue time. Give reviewers room to do the work. Make it acceptable to reject a generated patch because the author cannot explain the tradeoffs. A review culture that depends on two overloaded senior engineers was already fragile. AI just makes the fragility harder to hide.

I would measure the rollout at the team and system level, not as individual surveillance. The goal is not to rank engineers by prompt efficiency, AI acceptance rate, or commits touched by a model. That will teach people to game the tool and hide the work. The goal is to learn whether the team is delivering more customer value with equal or better production health, review quality, maintenance cost, and human sustainability.

That is a higher bar than “we shipped more code.” It is also the only bar that matters.

The useful counterargument

Some teams really are shipping more value with AI. The piece should not pretend every AI gain is fake. AI can make neglected work cheaper: tests, documentation, migrations, internal tools, API exploration, cleanup, and the boring tasks that used to lose every prioritization fight. For teams with strong product judgment, review discipline, and ownership norms, those gains can turn into better software and less operational pain.

But that does not weaken the argument. It raises the standard of proof.

If AI helps your team ship a feature customers use, with the same or better reliability, without overloading review, without hiding maintenance cost, and without turning senior engineers into cleanup infrastructure, then call it productivity. That is the kind of win I would take seriously. If AI mostly increases the number of things in motion, the number of pull requests waiting for review, the number of features nobody asked for, or the amount of code nobody fully understands, then the team has not discovered leverage. It has discovered a cheaper way to create inventory.

Software teams already had too much inventory: backlogs full of maybe-work, roadmaps full of stakeholder theater, codebases full of half-owned decisions, and dashboards full of metrics that make motion look like value. AI does not forgive those habits. It compounds them.

Users pay the invoice

The title says the bill comes due in production, and that is true. Production is where weak assumptions stop being private. It is where review gaps, unclear ownership, fragile rollouts, missing tests, and confused product decisions become visible.

But production is not the final customer. Users are.

A production system can survive a change that still should not have been built. A team can deploy safely while wasting months on features that do not improve the product. AI productivity measured only at the engineering boundary will miss that failure.

The sharp version of the claim is this: AI productivity that does not reach users as better software is not productivity. It is cheaper throughput inside the factory.

That does not make AI bad, and it does not mean teams should avoid the tools. Teams should use them and push them hard. Let engineers automate the boring parts. Let agents draft, test, explain, migrate, and explore. But do not let the organization pretend that faster code is the same as better software.

The bill comes due in production, but the invoice is paid by customers. If they do not get a better product, and the team does not get a healthier delivery system, the productivity story is just a nicer dashboard wrapped around more work.

The code compiles and the tests pass. Then you notice the part the test did not cover: the retry behavior changed, an implicit contract moved, or the abstraction will make the next incident harder to debug.

That is the gap hiring has always tried to measure. Code is the thing a company can put in front of a candidate, so code became the proxy. Solve the problem. Talk through the implementation. Pass the tests. From there, the hiring team inferred judgment, ownership, taste, and level.

AI makes that inference weaker.

A candidate can now arrive at correct-looking code with a model beside them. That does not make the work fake, and it does not make coding interviews useless. It means the useful questions start after the answer appears: what did the candidate trust, what did they reject, what did they test, and what risk would they own if this shipped?

That is the shift this piece is about. The hiring signal is moving out of code output and into the work around it: how the candidate reviews, validates, recovers, communicates, and owns the result.

Today, we cover:

1. Why the old proxy was strained before AI.

2. Why behavioral interviews were already testing technical work.

3. Why cheating is the loud symptom, not the root problem.

4. What process evidence looks like when AI is part of the work.

5. Why senior and staff candidates now have to prove level differently.

6. What a better hiring loop might measure.

1. The old proxy stopped being clean

Coding interviews were never meant to model the whole job. They were a shortcut for a problem hiring cannot avoid: a company has to make a decision before it has watched someone work inside its systems, with its constraints, for months.

That shortcut made sense when the artifact was expensive enough to produce that it carried more signal. A candidate who could reason through a problem, write a working solution, and explain the implementation gave the interviewer something concrete. It was incomplete, but it was useful.

Developers were already skeptical of the trade. HackerRank’s 2025 Developer Skills Report says 78% of developers believe assessments do not align with real-world tasks, and 56% find algorithm-based questions irrelevant to their jobs. That complaint predates the current AI wave. A lot of engineers have been saying some version of this for years: the interview problem is cleaner than the job.

I recognize that bias in myself. On the power-plant telemetry platform I worked on earlier in my career, the difficult parts were not isolated to writing a parser or moving bytes from an edge device to the cloud. The work was in latency, sensor behavior, failure modes, backpressure, retry logic, observability, and the question of what downstream data analysts could trust. A hiring process that only looked at whether someone could make code appear would miss a lot of the engineering that made the system reliable.

AI widens that gap. In the same HackerRank report, 97% of developers said they use at least one AI assistant, 61% said they use two or more AI tools at work, and AI-generated code accounted for 29% of developers’ code on average. Vendor survey numbers should always be read carefully, but the direction is hard to miss. AI-assisted code is no longer a weird edge case.

That changes the weight of the artifact. The problem is not that every AI-assisted answer is suspicious. The problem is that the answer explains less about the person behind it. If an assessment already felt detached from daily engineering work, and now the artifact it rewards can be produced or polished by a model, the old signal gets squeezed from both sides.

That is the first pressure on the interview loop. It cannot only ask whether the candidate reached an answer. It has to ask whether the path to the answer exposed enough understanding to trust the candidate with the job.

2. The behavioral interview was already more technical than we admitted

The word “behavioral” does a lot of damage in engineering hiring. It makes the round sound soft, secondary, almost ornamental. First you prove you can code, then you talk about teamwork and conflict and communication, as if those things happen somewhere outside the technical work.

That split has always been artificial.

In an April 2026 Pragmatic Engineer piece, Steve Huynh, formerly a Principal Engineer at Amazon, reflected on nearly 1,000 interviews, including around 600 as an Amazon Bar Raiser. His point is useful because it does not come from AI hype. It comes from years of interview loops before this current wave fully landed. The behavioral round, in his telling, often decides fit and level because it exposes the work that coding rounds do not: how someone handles projects going sideways, disagreements, incomplete information, stakeholder pressure, and influence without authority.

Those are not decorations around engineering. They are engineering at senior levels.

A mid-level engineer can often succeed inside a bounded technical problem. A senior engineer has to make progress when the boundary is blurry. A staff engineer has to make other people’s work easier, align teams that do not naturally agree, and turn a technical direction into something the organization can actually execute. At those levels, “tell me about a time a project went wrong” is not a personality test. It is a probe for operational judgment.

That matters now because the artifact has become less isolated from tools, templates, copied context, and generated suggestions. Interviewers need to inspect the candidate’s relationship to the artifact: what they trusted, what they rejected, what they tested, what they understood about the codebase, and what risk they noticed before someone else named it.

The old behavioral round was already asking versions of those questions. The problem is that many loops treated it as a separate soft-skills filter instead of part of the technical evaluation.

3. Cheating is the loud symptom

The cheating story is real. It is also not the whole story.

CodeSignal said in February 2026 that detected cheating and fraud attempt rates in proctored assessments rose from 16% in 2024 to 35% in 2025. For entry-level assessments, CodeSignal said the rate increased from 15% to 40%. Those are vendor numbers, and CodeSignal sells assessment integrity products, so they should be treated carefully. The company is measuring detected and flagged attempts, not proving that every flagged session was successful cheating or that its data represents the whole market.

Still, the direction matches what many hiring teams feel. HackerRank reports that 76% of developers say AI makes gaming assessments easier, and 73% feel it is unfair to lose out to candidates who use AI to game tests. Once AI tools are available to everyone, any assessment that rewards final output while hiding the process becomes easier to distort.

The tempting response is to make the old proxy more tightly controlled. Ban AI. Add proctoring. Detect screen switching. Flag pasted code. Train interviewers to spot suspicious pauses, suspicious speed, suspicious fluency. Some of that is necessary. Companies need fair processes, and candidates deserve not to lose to someone performing a fake version of competence.

But fraud detection can only protect the fairness of the test in front of it. It cannot make a weak test more representative of the job.

If the job itself now includes AI-assisted coding, banning AI from every interview creates a strange mismatch. If the job does not allow AI because of security, compliance, or risk, then banning it in the interview makes sense. But many engineering teams are somewhere messier: developers use AI for debugging, code review, codebase understanding, tests, refactors, scripts, documentation, and first-pass implementation. The interview then pretends the work happens without the tools that shape the work.

That mismatch creates bad incentives. Candidates hide tool use. Companies hunt for hidden tool use. Both sides spend energy preserving the appearance of a clean artifact.

The real design question is narrower and harder: can the interview separate a candidate who used AI responsibly from one who used it to mask weak understanding?

4. The interview has to collect process evidence

Karat’s NextGen work is useful as a market signal because it shows one direction hiring infrastructure is moving. In its December 2025 launch announcement, Karat described interviews where candidates work on complex multi-file projects with an integrated AI assistant while human interviewers probe reasoning, trade-offs, and judgment in real time. In the April follow-up, Karat said it now gives engineering leaders evidence such as skill scores, rationale write-ups, and timestamped markers tied to moments in the interview.

Again, this is vendor evidence. Karat sells interview infrastructure, so it has every reason to argue that interviews need more infrastructure. But the underlying problem is real: if the output and the candidate’s skill are decoupled, the hiring system needs an evidence trail that lives somewhere other than the final code.

That maps more closely to how AI-assisted work is settling inside real teams. Stack Overflow’s May 2026 pulse survey found that AI agent usage at work rose to 59%, up from 31% in the previous annual survey, but also that 63% of technologists still rarely or never let agents run fully on autopilot. The interesting part is the combination. People are using agents more, but the dominant working mode is still supervised, reviewed, and bounded.

A May 2026 longitudinal study by Annie Vella and Kelly Blincoe makes the same shift visible from another angle. The authors found that 82% of participants reported spending less time writing code, and they describe a broader move from creation toward verification activities. Their term for the new category of work is supervisory engineering: directing, evaluating, and correcting AI output.

If that is increasingly the work, then interviews that only measure artifact creation are behind the job.

A better AI-era technical interview does not need to become a surveillance exercise. It needs to make process observable. Give the candidate a realistic codebase slice. Let them use tools if the role would let them use tools. Ask them to inspect a change, critique a model suggestion, write or adjust tests, explain where the model’s answer is brittle, choose between two implementation paths, and name what they would monitor if this shipped.

The strongest moments in that kind of interview are often not the moments where code appears. They are the moments where the candidate slows down for the right reason.

In practice, the signal is often small. The candidate notices that a generated function handles the happy path but changes an implicit contract. They keep the refactor but rename a concept because the model erased domain meaning. Before changing a data model, they ask about production constraints. When the prompt omits a failure case, they add the test anyway.

The candidate is doing engineering there. The old prompt just did not have a clean way to score it.

Subscribe now

5. Senior and staff candidates have to prove level differently

The higher the role, the more dangerous it is to treat the finished patch as the whole signal.

That does not mean senior and staff engineers get a pass on code. A senior engineer who cannot reason through code is a liability, especially now that AI can produce plausible nonsense quickly. But the senior signal is not only implementation under time pressure. It is whether the candidate can choose the right problem shape, reduce risk, and make the result usable by other people.

Huynh’s framework in The Pragmatic Engineer piece is helpful here. The excerpt from his book describes level through four dimensions: scope, contribution, impact, and difficulty. Those dimensions are hard to infer from a standalone coding artifact. They show up more clearly in how a candidate reviews a change, explains a decision, notices ambiguity, and handles a problem that does not stay inside the prompt.

A useful staff-level task here is not a blank editor. It is a small service change where the generated patch fixes the visible bug, but the API is used by another team, the migration has no rollback path, and the test suite only covers the happy path. A strong senior candidate should catch the missing test and explain the production risk. A staff candidate should usually go further: ask who depends on the contract, identify the rollout constraint, and challenge whether this is the right place to solve the problem.

That is the level difference the code alone will not show. The same artifact can hide very different kinds of judgment. One candidate sees a local implementation problem. Another sees a system boundary, an organizational dependency, and a future incident if the rollout goes wrong.

This matters more in the current market because the bar has moved up. The Pragmatic Engineer’s public piece on tech interviews in 2025 describes higher expectations in DSA and system design interviews, more demanding senior and staff bars, and more downleveling. Whether every company experiences this the same way is less important than the direction. In a market with more qualified candidates than open roles, companies can ask for more evidence before they say yes.

AI raises the same pressure from the other side. If more candidates can produce polished artifacts, the differentiator moves to the explanation around the artifact. Senior candidates need to show the shape of their judgment. Staff candidates need to show the scope of it.

This is where “storytelling” gets misunderstood. The goal is not to tell a smoother story. The goal is to make the work legible. A good senior story is evidence of what was ambiguous, what you owned, which alternatives you considered, what changed because of your decision, where the trade-off hurt, and what you learned when reality disagreed with the plan. The code artifact rarely carries all of that by itself.

6. A better loop measures the work around the code

The better hiring loop is not simply “allow AI.” That is too shallow. A company can allow AI and still run a bad interview. It can require AI and accidentally select for prompt performance over engineering judgment. It can ban AI and still run a fair process if the actual job bans AI too. The tool policy matters, but it is not the heart of the design.

The heart of the design is whether the loop creates comparable evidence of how the candidate works.

Comparable matters because process-heavy interviews can become unfair very quickly. The more an interview depends on narration, confidence, and polish, the more it can reward candidates who have been coached into the right performance. It can also punish candidates who are strong engineers but less comfortable speaking in a high-pressure environment, or who are working in a second language, or who come from teams where the local storytelling norms are different.

So the answer cannot be “make everything behavioral” and call it modern.

A better loop would still be structured. It would ask consistent questions. It would use a calibrated rubric. It would document evidence rather than vibes. It would give candidates the same kind of task, the same rules around AI, and the same chance to explain what they did. It would score implementation and supervision as related but distinct signals.

For a senior backend role, that might mean giving the candidate a small service with a bug, an incomplete test suite, and an AI assistant. The task is not to produce the most code. The task is to understand the behavior, make a safe change, explain the risk, and show how they would validate it. For a staff role, the loop might add an architectural constraint, a cross-team dependency, or a migration choice. The candidate still writes code, but the interview watches how they reason around it.

This would also make interviews feel less strange to candidates who already work with AI every day. The candidate would not have to pretend their workflow is cleaner than it is. They would have to show that their workflow is responsible.

That is a higher bar in some ways. It is easier to memorize a pattern than to explain why the model’s pattern is wrong for this codebase. It is easier to generate a passing solution than to defend the test strategy. It is easier to look productive than to show good judgment when the tool gives you something almost right.

The phrase “almost right” is doing a lot of work here. AI-generated code often fails in the place where interviews used to stop looking. It compiles, but it misunderstands the domain. It passes the visible tests, but it weakens the invariant. It follows the local style, but it changes the operational behavior. It gives you the answer a good interviewer would now use as the start of the interview, not the end.

Takeaways

The coding round has to become less isolated. The AI-era interview should not drop the technical bar. It should stop treating a finished artifact as the whole technical record. A candidate who cannot reason through code is still not ready for a serious engineering role. A candidate who can produce code without explaining the decisions around it is also harder to trust than they used to be.

Behavioral signal needs a better name. Many of the signals companies call behavioral are really senior engineering signals: handling ambiguity, communicating trade-offs, influencing without authority, owning mistakes, and making decisions when the available information is incomplete. AI did not make these skills newly important. It made them harder to keep outside the technical evaluation.

Integrity tools are necessary but incomplete. Hiring teams need some way to prevent candidates from faking work. That is a real problem, especially for early-career assessments where the pressure is intense and the signal is thin. But if the assessment is built around output without process, proctoring can only defend the shape of the test. It cannot make the test more like the job.

The best interview evidence will look more like review evidence. In daily AI-assisted work, the important human actions are often direction, evaluation, correction, and ownership. Hiring loops need to make those actions visible. That might mean live review, timestamped evidence, structured rationales, realistic codebase tasks, or interviewer notes tied to specific moments. The exact format can vary, but the evidence has to move closer to the work.

Candidates need to make ownership visible. AI-polished artifacts will raise the floor on what many candidates can show. The way to stand out is not to pretend the tools are not there. It is to show the part of engineering the tools do not own: the risk you saw, the trade-off you made, the context you asked for, the change you would not ship, and the consequence you were willing to be responsible for.

I do not think this ends with one standard interview format. Some roles should ban AI because the job does. Some should permit it because the job does. Some should test both modes. The mistake is treating the passed test as the end of the evidence.

The code can still be on the screen. It can still pass. The interview cannot stop there.

The next question is the one that looks more like real work: what would the candidate trust, what would they change, what would they test again, and what would they be willing to own in production?

I do not think hiring has a clean answer yet. But any process that cannot see that layer is measuring less of the job than it thinks.

That is a very funny sentence if you have ever sat near an enterprise budget process. It is also the clearest signal that the first social phase of AI coding is ending. The joke version was tokenmaxxing: use more, show more, look more AI-native. The business version is less fun. It asks who is allowed to spend, how much they can spend, which work justified the expensive model, and what happens when the shared pool runs out before the work does.

I do not think this means AI coding is failing. I think it means AI coding is becoming normal company software. Most meaningful company spend follows a familiar path: it starts as local freedom, then becomes a team habit, then becomes a line item, then becomes policy. Cloud went through this. Observability went through this. CI minutes, SaaS tools, data warehouse queries, feature-flag platforms, support contracts, and half the software hiding in expense reports went through some version of this. Someone adopts the thing because it helps, usage spreads because the thing is genuinely useful, and eventually the company realizes the spend is material enough that vibes are not a control system.

For the last couple of years, AI coding mostly lived before that last step. A developer paid for Cursor personally. A team expensed Claude. A manager approved GitHub Copilot because saying no felt more irresponsible than saying yes. A company created an innovation budget because nobody wanted to be the person blocking AI adoption with a spreadsheet. Some of that was sensible. You do not learn a new work pattern by designing the perfect governance model before anyone has touched the tool.

But there was always going to be a second phase. If a tool can spend variable dollars while reading private code, producing diffs, triggering CI, and creating work that humans have to review, it will not stay a personal productivity preference forever. The business will eventually ask the normal business questions: who owns the spend, who gets more of it, what happens when it spikes, which work justified the expensive path, and whether the output was worth the money.

This is the follow-on to my earlier essay on tokenmaxxing. That piece was about why token usage is a terrible status signal. This one is about what happens when the status signal becomes a policy surface. The interesting question is not simply whether AI tools are expensive. The better question is what changes inside engineering when the cost of AI becomes visible, governable, and close enough to the work to interrupt it.

1. AI Spend Is Leaving The Side Door

The surprising part is not that companies are starting to manage AI spend. The surprising part is that AI spend briefly behaved as if normal company rules did not apply.

I do not mean that as a complaint about enterprises. This is one of the few things enterprises are extremely consistent about. If a resource becomes large enough, shared enough, and unpredictable enough, the company will eventually attach controls to it. People can dislike the controls, and many controls are badly designed, but the motion itself is not mysterious.

Cloud is the obvious comparison, but the pattern is broader than cloud. A team buys an observability product because production is hard to understand. A department starts using a SaaS tool because the approved internal process is too slow. A few engineers run something in a managed service because waiting for platform support would take months. The first phase is local usefulness. The second phase is adoption. The third phase is someone asking why this thing now costs enough to show up in a planning meeting.

AI coding compressed that pattern because the tool was attached directly to work people were desperate to accelerate. It was not a nicer calendar app or a better notes tool. It promised to change the cost of producing software. That gave it political cover. Nobody wanted to be the manager who slowed down AI adoption because the cost model was messy, so a lot of companies tolerated the mess for a while.

Altman’s cost comment matters because it is the vendor-side version of the same enterprise pattern. The joke works because it is absurd. It also works because anyone who has watched company spend grow through informal channels knows the next scene.

The side door closes. Not completely, and not all at once, but enough that the work changes. Budgets appear. Dashboards appear. Exceptions appear. Someone asks whether the expensive model was necessary. Someone else asks why one team burned through the shared pool. A team discovers that an agentic task can be blocked not by code review, CI, or test failures, but by a budget policy. That is the moment AI coding stops being only a tool choice and becomes an operating model.

2. Usage-Based Billing Makes The Policy Surface Explicit

GitHub did not just change how Copilot gets priced. It made the budget layer part of the engineering workflow.

On June 1, 2026, GitHub moved Copilot usage to GitHub AI Credits across all plans. GitHub’s April announcementexplained the reason in terms of product shape: Copilot is no longer just an in-editor assistant. It has become an agentic platform that can run long, multi-step sessions, use frontier models, and work across repositories.

That product shift breaks the old developer-tool pricing intuition. A quick inline question and a long-running agentic coding session are not the same economic event. GitHub says usage is calculated from input, output, and cached tokens, with model-specific rates. The June 1 changelog also says Copilot code review consumes GitHub Actions minutes in addition to AI Credits, and that user-level budget controls are generally available for organizations and enterprises.

The docs are even more revealing than the announcement. GitHub’s organization and enterprise billing docs describe pooled credits, user-level budgets, cost-center budgets, enterprise spending limits, organization-level budgets, and budget exhaustion behavior. If additional usage is not allowed, usage can be blocked until the next billing cycle. If a user-level budget is exhausted, that user’s access can stop even when the organization still has credits left. This is not just metering. It is work shaping.

The same direction shows up in Anthropic’s Claude Code docs. The cost-management page tells teams to track token usage, set team spend limits, manage context, choose models intentionally, and account for usage patterns like multiple instances or automation. That is the language of infrastructure operations more than old developer tooling. The cost of a developer’s workflow now depends on model selection, codebase size, context management, concurrency, automation, and how often the agent loops.

A flat IDE subscription was boring once procurement approved it. A metered agentic workflow is not boring. It has model choice, context size, retries, tool calls, failed loops, parallel agents, CI runs, and review cost. The workflow can be expensive because the work is valuable, or because the task was vague, the repo boundary was too large, and the agent kept spending context to compensate for unclear human direction. The bill does not explain which one happened. It only makes the question impossible to avoid.

3. Agentic Coding Has Strange Unit Economics

Agentic work is hard to budget because its cost does not map cleanly to the way humans estimate engineering work.

With ordinary developer tools, the unit economics are legible enough that most engineers never think about them. A seat costs what a seat costs. An IDE license does not become more expensive because a refactor is messy. A GitHub seat does not draw from a shared pool because one engineer asked too many questions about a monorepo. The tool has a price, the work has complexity, and those two things are mostly separate.

A coding agent spends tokens on context, planning, tool calls, file reads, edits, retries, test output, summaries, and correction loops. The final diff may be small, but the path to that diff can be large. The agent may read the wrong files first, carry stale context, retry a failing test several times, use a frontier model where a cheaper model would have worked, or keep processing a huge prompt because the human never narrowed the task. The cost is partly technical and partly managerial.

The research backs up what heavy users already feel. The April 2026 arXiv paper How Do AI Agents Spend Your Money? studied token consumption in agentic coding tasks and found that these tasks can consume far more tokens than simpler code chat or code reasoning. It also found high variability across runs of the same task, weak alignment between human-rated task difficulty and token cost, and no simple relationship where more tokens reliably means better accuracy.

More spend can be rational, and more spend can be waste, but the token count alone cannot tell you which one happened. Two engineers can hit the same monthly budget for completely different reasons. One may be using an agent to pay down migration risk across a messy production system. Another may be asking an expensive model to generate throwaway scaffolding because the approved workflow makes the expensive path easier than the sensible one. A dashboard that treats those as equivalent will produce bad management because it sees consumption before it sees judgment.

Engineering has lived with some version of this with the cloud, but AI makes the feedback loop more ambiguous. A production service that suddenly spends too much cloud money usually has an operational shape: traffic changed, a job got stuck, storage grew, a query got expensive. An agentic coding session can burn money in a more human-looking way. The prompt was broad. The context was stale. The model choice was excessive. The task should have been split. The agent was allowed to continue because the progress looked plausible. That makes token cost an engineering design problem, not just a billing problem.

4. The Dashboard Will Be Tempting And Wrong

The worst version of this future is not expensive AI. It is an incentive system that makes expensive AI look like productivity.

Usage visibility is necessary. Teams need to know which workflows are cheap, which are expensive, which users are outliers, and which agent patterns create runaway spend. Without visibility, leaders are guessing, and guessing is not a policy. The danger starts when visibility becomes a scoreboard.

This is the tokenmaxxing failure mode. If high token usage becomes a status signal, people will find ways to use more tokens. If low token usage becomes the status signal, people will hide useful work, route around the approved tools, or waste human time avoiding a budget that would have been cheap to spend. If raw AI-generated output becomes the signal, teams will reward surface area instead of engineering value.

That should feel familiar because software organizations have done versions of this with lines of code, ticket counts, pull request volume, and story points when they escaped their original purpose and became management theater. The lesson is not that measurement is bad. The lesson is that visible metrics attract performance, and engineering systems are easy to perform badly. Token usage is especially tempting because it looks concrete: it has numbers, charts, cost, and a clean path into budget conversations. It gives managers something to point at, which is useful until pointing replaces understanding.

The good version of usage visibility is outlier inspection. Which tasks cost far more than expected? Which model choices produced no better result? Which agent loops triggered repeated CI runs? Which teams are burning context because their repos are hard to navigate? Which workflows save review time, not just implementation time? Those are useful questions because they connect spend to the shape of the work. The bad version is rank ordering engineers by how much AI they used or how little AI they used. That is how a cost-control tool becomes a culture problem.

Subscribe now

5. Agile AI Will Be a Thing

My bet is that AI spend becomes part of engineering planning, but not as exact token estimation. It will be more awkward and more familiar than that.

The funny version is token poker. The serious version is that teams will start classifying work by model intensity, autonomy level, and budget risk. Not because anyone can predict exact token usage. They cannot, and the research already suggests the cost can vary across runs in ways humans do not estimate cleanly. But teams do not need fake precision to change behavior.

Agile teams already use rough sizing to create conversation. A t-shirt size is not a duration. A story point is not a contract, at least not when the system is healthy. The point is to expose uncertainty before the work starts. AI budget planning may evolve the same way, with teams asking whether a task is a small AI task, a medium AI task, or a large AI task.

A small AI task might be a contained test update, a local refactor, or a documentation pass where a cheaper model and narrow context are enough. A medium AI task might involve several files, a few test loops, and a model upgrade if the first pass gets stuck. A large AI task might be a cross-service migration, a security-sensitive change, or an agentic exploration across a messy repo where the budget risk is part of the work. The useful question will not be “how many tokens will this take?” The useful question will be “what kind of AI spend does this work deserve?”

That distinction matters because the goal is not to turn engineers into accountants. The goal is to make model choice, autonomy, context size, and review burden visible before a long-running agent task starts spending from a shared pool. This will feel silly at first. Most new planning vocabulary feels silly before it becomes normal. Someone will make token t-shirt sizes, someone will joke about sprint capacity in AI credits, and someone will build a dashboard that turns the joke into a quarterly operating review. Some of it will be useful. Some of it will be awful. That is usually how enterprise process arrives.

6. Managers Need to Get Ahead of the Conversation

The people closest to the work need to define good AI spend before people far from the work define cheap AI spend.

This is where engineering managers and senior engineers have more responsibility than they may want. Finance can see the bill. Procurement can negotiate the contract. Security can define data boundaries. Legal can worry about exposure. But none of those functions can reliably tell whether a specific agentic run was good engineering judgment.

That judgment lives in the messy middle of the work. Was the expensive model justified? Was the agent given a task that should have been clarified by a human first? Did the work save review time or create review debt? Did the team use AI to accelerate engineering work, or merely to accelerate code change? These are engineering questions, and they get harder to answer when the organization treats token spend as either automatically good or automatically wasteful.

A $500 agentic task that saves two days of senior engineering time may be cheap. A $5 task that creates a confusing diff nobody trusts may be expensive. A team that spends heavily while retiring migration risk may be acting responsibly. A team that spends heavily because it keeps asking agents to explore poorly bounded work may be turning ambiguity into an inference bill. The danger is not that companies will care about AI spend. They should. The danger is that they will care about it badly.

One bad version is blunt thrift: cap everything, block useful workflows, make exceptions painful, then watch engineers route around the system with personal subscriptions and API keys. Another bad version is adoption theater: celebrate usage because it makes the company look AI-native, then discover six months later that the codebase absorbed more change than the review system could metabolize. The better version is harder because it requires engineering leaders to treat AI budget as part of technical direction. Not because spend is the most important thing, but because spend now shapes the work.

The budget determines whether an agent continues, which model gets used, how much context gets loaded, when an engineer asks for approval, and whether a team can keep experimenting at the point where experimentation is actually useful. This is why token budget is becoming engineering policy. Not a finance footnote. Not a procurement detail. Not a personal preference that each engineer gets to optimize alone. It is part of the system that decides how engineering work happens.

Subscribe now

Takeaways

The side-door era of AI spending is ending. AI entered many companies through experimentation before the operating model was ready. That was probably necessary, but it also means the next phase will feel less magical and more administrative. Budgets, caps, cost centers, and exception paths are not evidence that AI failed. They are evidence that AI became material enough for the business to manage.

Token usage is still not productivity. High usage can mean valuable leverage, careless prompting, poorly bounded work, or simple metric-chasing. Low usage can mean discipline, under-adoption, fear of caps, or hidden shadow tooling. The number matters, but it does not explain itself. Engineering judgment has to sit next to the dashboard.

Agentic cost belongs in engineering planning. The future is probably not exact token estimation. That would be fake precision. The more plausible future is rough classification: which work deserves cheap models, which work deserves expensive models, which work should run autonomously, and which work needs a human to narrow the problem before an agent starts spending.

Managers should shape the policy before finance does. If engineering leaders do not define what good AI spend looks like, someone else will define what cheap AI spend looks like. Those are not the same thing. The teams that handle this well will not be the ones that merely spend less. They will be the ones that can explain what the spend bought, what it saved, and what it moved into review, maintenance, or risk.

I do not think the uncomfortable part is that AI coding costs money. Engineering has always paid for leverage. The uncomfortable part is that the cost is now close enough to the work to change the work, and most teams do not have language for that yet. They will soon.

But then, the PR states, coauthored with Claude Code (or any other harness for that matter). It comes with no surprise nowadays that AI had its claws all over the code, but there’s an interesting question that I think needs answering. Who is truly responsible for the changes? Is there a human being who’s willing to say they understand the change well enough to own it when it breaks, defend it in review, explain it during an incident, and accept the consequences of having let it into the system?

That is the part of AI-assisted engineering that still feels under-discussed to me. We spend a lot of time talking about the generation layer. Which model wrote the code. Which agent can use the terminal. Which IDE has the better context window. Which benchmark moved by three points.

But the production system does not end when code appears.

Today, we cover:

1. Why AI made the approval step more important, not less important.

2. Why the engineering work is shifting from creation toward supervision.

3. Why faster code generation exposes the rest of the delivery system.

4. What sign-off actually means once agents can create real artifacts.

5. What a sane sign-off system might look like without turning every autocomplete into a compliance event.

The short version is this: AI made generation cheaper. It did not make ownership cheaper.

1. The approval step did not go away

The cleanest AI coding policy I have seen recently comes from a place that is not known for being loose about software process: the Linux kernel.

The penguin itself doesn’t ban the use of AI as maybe some of you would expect, but it doesn’t recognize it as an entity in the process either. It treats AI-assisted contributions as any other contribution. All contributions still need to comply with licensing requirements. AI agents must not add Signed-off-by tags. Only humans can certify the Developer Certificate of Origin. The human submitter is responsible for reviewing the generated code, ensuring licensing compliance, adding their own sign-off, and taking responsibility for the contribution.

That last part is the system.

The kernel also added an Assisted-by tag for AI involvement, including the agent name, model version, and relevant tools. The point is not to shame anyone for using AI. The point is to keep the work attributable enough that reviewers and maintainers can reason about what happened.

The companion tool-generated content guidelines are even more explicit about the underlying problem. Tooling can increase contribution volume, but reviewer and maintainer bandwidth is scarce. If a meaningful amount of content was created by a tool, contributors should be transparent about the tool, the affected parts, the input when it matters, and how the submission was tested.

My read is that the kernel landed on the right framing because it did not start from AI exceptionalism. It started from the existing engineering practices.

The contribution has an origin. The origin needs to be legible. A reviewer needs to know what they are reviewing. A maintainer needs to know who understands the change. A human in the sign-off chain needs to be able to answer questions later.

And this is important! it’s a process with three layers.

The generation layer is where the agent creates something: code, tests, dashboards, documentation, migration plans, internal tools, release notes, incident summaries, or the first version of a design.

The verification layer is where someone checks whether that thing is correct, secure, licensed, observable, compatible with the existing system, and appropriate for the operational risk it carries.

The sign-off layer is where a human accepts ownership in a way the organization can audit later.

Most of the AI tooling conversation is still obsessed with the first layer. That makes sense. Generation is where the demo happens. It is where the speed is visible. It is where a model can do in three minutes what used to take an afternoon.

But the expensive part of engineering was never only producing text that compiles.

The expensive part was knowing what that text means inside a system that already exists.

2. The work is moving from creation to supervision

This is not just a philosophical concern. The work is already moving.

In a longitudinal study submitted to arXiv on May 22, 2026, Annie Vella and Kelly Blincoe followed professional software engineers across two questionnaires six months apart. They had 158 eligible participants at the first point, 101 at the second, and 95 matched participants across both rounds. The headline finding is not subtle: 82% of participants reported spending less time writing code.

But the more interesting finding is what replaced the writing.

The authors describe a shift from creation toward verification activities, and they propose a category they call supervisory engineering work: directing, evaluating, and correcting AI output.

That phrase is a little academic, but the underlying job feels familiar. You ask the agent to make a change. You read the result. You notice it solved the wrong edge case. You redirect it. You ask for tests. You inspect the tests because the tests might only prove the bug it introduced. You check whether the code fits the conventions of the repo. You decide whether the diff is worth keeping, splitting, rewriting, or throwing away.

The same paper reports what it calls a productivity-experience paradox. At both time points, 84% of participants reported productivity improvement. At the same time, among matched participants, the share reporting worsened developer experience in at least one dimension nearly doubled from 14% to 27%. Flow state and cognitive load got worse while feedback loops improved.

That tracks with my own experience much more than the clean productivity story does.

I do feel faster with AI tools. Some days, dramatically faster. I can generate a first pass at an internal script, a content workflow, a test suite, or a migration plan before I would have finished arranging my own thoughts in a blank file.

But I have also noticed the weird part: the tool often moves the work into a mode where I am approving, redirecting, checking, and reconciling. The artifact shows up quickly. The judgment still takes time. Sometimes the judgment takes more attention because the artifact looks more finished than my own unfinished work would have looked at the same point.

This is the approval behavior leak I have written about before. The dangerous moment is not always the spectacular agent failure where a tool tries to delete the repo or run a command it should not run. The dangerous moment is much more ordinary: I am tired, the diff looks plausible, the explanation is polished, and I find myself clicking yes before I have really understood what I am approving.

That moment is going to matter more as the generation layer gets better.

Bad generated code is annoying, but at least it announces itself. Good-looking generated code is harder. It asks for trust before it has earned it.

Now, this is not to say there aren’t players out there completely ignoring this layer, vibe coders and developers are overly relying on AI to the point that they don’t care about the code anymore. That’s not engineering, and while it’s true, they are building things, I surely hope those things are not business critical. Want to vibe code a small internal tool? a dashboard, something that makes your life easier? go ahead! but don’t use the same practice with your critical production systems.

3. Faster generation exposes the delivery system

The optimistic version of AI coding says that if engineers can produce more code, teams can ship more value. Sometimes that is true. It is also incomplete.

If code generation gets faster and the rest of the delivery system stays the same, the bottleneck does not disappear. It moves.

Harness’s 2026 DevOps modernization report is useful here because it looks downstream of the IDE. The report says very frequent AI coding users are more likely to deploy daily or faster, but they also report more deployment pain. According to Harness, 69% of very frequent AI coding users say AI-generated code leads to deployment problems at least half the time. Very frequent users also report longer mean time to recovery for deployment-related production incidents: 7.6 hours, compared with 6 hours for frequent users and 6.3 hours for occasional users.

That does not prove AI caused those incidents. Harness says as much. Teams using AI heavily may simply be pushing more change through systems that were already strained.

But that is exactly the point.

AI does not just generate code. It changes the volume and shape of work flowing through review, CI, deployment, security checks, incident response, documentation, and support. If those systems were held together by a few senior engineers doing late-stage heroics, AI does not remove the heroics. It may increase the number of moments where those heroics are required.

Harness’s engineering excellence material points at the same invisible work from another angle. In a survey of 700 engineering practitioners and managers, Harness argues that developers are becoming validators of machine-generated output, with conventional productivity frameworks failing to capture validation time, agent accuracy, cognitive load, and trust calibration. It reports that 81% of engineering leaders say code review time has increased since deploying AI, and that 31% of a developer’s day is now consumed by AI-related invisible work that appears in no metric.

You can quibble with any one survey number. You should. Vendor surveys come with incentives, definitions, and sampling choices that deserve skepticism.

But the direction matches what I see in practice.

Teams get excited about the gross output. More code. More tasks started. More PRs opened. More internal tools appearing from nowhere. The net output is harder to see because the cost is distributed across review queues, context switching, subtle bug fixing, security review, unplanned coordination, and the cognitive load of deciding what deserves trust.

That is why the sign-off layer matters. It is the place where gross output becomes owned output.

Without that layer, AI adoption creates a pile of things that look done and behave like debt.

4. Sign-off is not code review with a new label

It is tempting to make this a code review essay. That would be too small.

Code review is one part of sign-off, but sign-off is broader than the PR approval button. It includes knowing what tools touched the work, what context they had, what the human author understands, what validation evidence exists, who owns the risk, and where the decision can be reconstructed later.

This matters because agents are no longer just autocomplete in the editor.

Vercel’s Open Agents project is a good example of where the architecture is going. It is an open-source reference app for background coding agents on Vercel. The system includes a web UI, durable agent workflow, sandbox orchestration, and GitHub integration. It can clone repos, work on branches, use file and shell tools, resume runs, and optionally auto-commit, push, and create PRs.

That is a very different object from a tab-completion tool.

Once agents become background systems with authentication, repo access, sandbox lifecycles, workflow state, cancellation behavior, and optional PR creation, “human in the loop” is too vague to be useful. Which human? At what checkpoint? With what evidence? After which tools ran? Before which irreversible action? Under whose account? With what audit trail?

The research world is converging on similar concerns. A 2026 paper in Automated Software Engineering on open, accountable, and trustworthy AI-IDEs frames traceability and validation loops as architecture, not as after-the-fact process decoration. That is the right instinct. If assistant-generated work is going to become normal, the record of how that work came to exist becomes part of the engineering system.

I do not think every autocomplete needs a confession booth. That would be absurd, and it would kill the productivity gain for the least risky cases.

The distinction has to be risk-based.

If an agent completes a line, fixes spelling, renames a variable, or formats a file, I do not need a policy ceremony. That is below the threshold where explicit AI attribution helps the team.

If an agent creates a meaningful function, modifies security-sensitive code, touches production configuration, generates a migration, writes a dashboard people will use for decisions, prepares a customer-facing incident summary, or opens a PR from a background workflow, the standard should be different.

At that point, the question is not “did a model help?”

The question is whether the work is legible enough for a human to own.

5. The manager and staff engineer angle

This is where the topic becomes less about tools and more about senior work.

A junior engineer can use AI to produce more code. So can a senior engineer. So can an engineering manager who has not opened the repo in six months. The generation layer does not care very much about the title but the sign-off layer does.

Senior engineers and staff engineers are going to spend more time deciding whether generated work fits the system. Not whether it compiles. Not whether the happy path works. Whether the change belongs, whether it preserves the right abstractions, whether the test evidence is meaningful, whether the operational story is complete, whether the blast radius is bounded, and whether the author actually understands what they are asking the team to merge.

Engineering managers are going to face the same shift from a different angle.

It is not enough to tell teams to use more AI. That is just pressure on the generation layer. The management work is designing the operating system around it: review expectations, ownership records, security gates, incident accountability, tool budgets, measurement practices, and the norms that let reviewers slow something down without being treated as blockers to the AI strategy.

If leadership measures AI adoption mostly through visible output, the rational team behavior is to create more visible output. More PRs. More generated docs. More internal dashboards. More agent activity. The sign-off work becomes a hidden tax paid by the people who care enough to read carefully.

That is a bad system. It punishes the engineers doing the work that makes AI safe enough to use. It also teaches everyone else that the organization values generation more than ownership.

This is why I keep coming back to attention. Sign-off consumes real attention, and senior attention is already the scarcest engineering resource in many companies. The fact that an agent can produce a 2,000-line refactor quickly does not mean a staff engineer can responsibly approve it quickly. It may mean the staff engineer now has a harder object to review because the diff is large, coherent, and slightly alien.

The uncomfortable part is that good sign-off will sometimes look slow compared with the demo. That does not mean it is waste. It may be the only part of the system that knows what the demo is allowed to become.

6. What a sane sign-off system might include

I do not think the answer is a giant AI policy document that nobody reads.

The answer is probably a set of boring defaults that make generated work easier to trust, easier to reject, and easier to audit later.

For meaningful AI-assisted contributions, the PR should say what kind of assistance was used. Not a dramatic disclosure. Just enough context for the reviewer to understand whether they are reading hand-shaped work, agent-shaped work, or a mix. The Linux Assisted-by idea is a good starting point because it treats attribution as an engineering aid, not a moral judgment.

Generated work should be smaller by default, not larger. This is one of the places where AI incentives are backwards. Agents are good at creating big coherent patches, but reviewers are still human. If a change cannot be reviewed without trusting the generator’s own summary, it is probably too large.

Reviewers should be allowed to ask for a human-written rationale. Not a polished AI summary. A short explanation from the author: what changed, why this approach was chosen, what alternatives were rejected, what could break, and how the change was validated. If the author cannot explain it, the team should not merge it just because the agent can.

Validation evidence should travel with the work. Tests, security checks, migration dry-runs, performance notes, screenshots, logs, or rollout plans should be linked where they matter. This is not about making PR descriptions longer. It is about making approval less dependent on faith.

Agent-created artifacts need owners. A dashboard that changes a product decision needs an owner. A generated support workflow needs an owner. An internal tool that writes to production-like data needs an owner. A migration plan generated by an agent needs an owner. Ownership cannot stop at “the agent made it.”

Destructive or cross-boundary actions need explicit gates. Anything that touches production data, customer accounts, deployment configuration, billing, auth, secrets, or broad repo state should have boring checkpoints that are hard to bypass accidentally.

The organization should measure the work AI creates around the code, not only the code itself. Review time, rework, incident recovery, subtle bug fixing, context switching, and validation burden are part of the cost. If those stay invisible, leaders will keep making decisions from the wrong side of the ledger.

None of this needs to apply equally to every case.

The threshold should rise with risk. A local helper script is not the same as a payment flow. A generated README edit is not the same as an auth middleware change. A one-line refactor is not the same as a background agent opening a PR after a long autonomous run.

The point is not to make AI usage feel dangerous.

The point is to make approval honest.

Subscribe now

Takeaways

The approval problem did not shrink with the implementation problem. If anything, the implementation problem becoming cheaper makes the approval problem easier to ignore. That is a bad trade. The system still needs a human who can understand, defend, and own the change.

The valuable AI skill is not only prompting. Prompting matters, but the higher-leverage skill is making generated work legible enough that another human can approve it without pretending. That means smaller changes, better rationale, visible validation, and enough traceability to reconstruct what happened later.

The bottleneck is moving into senior judgment. Staff engineers, tech leads, and engineering managers are going to feel this first because they already sit near the ownership boundary. They will be asked to approve more work that arrives looking finished. The hard part will be noticing which finished-looking work still has not earned sign-off.

AI policy should start from the existing engineering contract. The Linux kernel guidance is strong because it does not treat AI as magic. The normal process still applies. Humans sign. Tool assistance is attributed where meaningful. Responsibility does not move to the model.

The sign-off layer is where AI coding becomes real engineering. Demos end at generation. Production starts at ownership. Between those two is the part most teams have not designed carefully enough yet.

The PR did not become safer because an agent generated it quickly. It became safer only when a human could explain the change, bound the risk, show the validation, and put their name under it.

That is slower than the demo.

It is also the part that lets the demo survive contact with the real world.

That provider is Resend, picked by an agent before they knew there was a choice to be made.

This is happening at scale right now to every developer-facing company, and most of them are still running the GTM playbook that worked five years ago. The playbook isn’t exactly wrong, it’s just covering a smaller share of the actual audience than it used to.

What’s happening across the industry is that two distinct shifts hit dev-tool GTM at the same time. Most companies are only responding to one of them, or neither. In this article, I work through both shifts, what breaks underneath them, and what engineering, product, and DevRel teams should be doing differently. Today, we cover:

1. What the old funnel looked like, and where it breaks

2. Developers are starting to ask, not search

3. There’s a new persona in your funnel

4. What engineering and product do differently, together

5. What DevRel becomes when the old shape is gone

The old funnel, and where it breaks

The traditional dev-tool funnel is well-understood. Engineering builds the product, and product packages it into something adoptable. DevRel runs the last leg to the developer through docs, blog posts, conferences, sample apps, Discord, podcast appearances, and Twitter presence. Developers evaluate the tool against alternatives by searching, reading, asking peers, attending talks, and trying things in side projects. When they’re convinced, they bring it to work, and the company buys.

Each function in that chain owned its segment cleanly. Engineering stayed mostly inside the company, talking to developers through interfaces rather than in public. Product worked with a few design partners. The audience-at-scale conversation belonged to DevRel. The handoff was clean, the skills were specialized, and the metrics were attributable.

That model is breaking, and the interesting part is that it’s breaking in two distinct ways at once. One break is about who buys. The other is about how the buying decision gets made. They look related from a distance, and they compound in practice, but they’re separate problems and they need separate responses.

Developers are starting to ask, not search

The developer is still the decision-maker for the kinds of tools developers buy. That part of the funnel is intact. What’s changing is the research and evaluation layer underneath the decision, and it isn’t changing evenly.

Plenty of developers still evaluate the way they always did. A senior engineer who has shipped auth six times does not open Claude to ask which provider to use. They decide from hard-won experience, sometimes from a conversation with someone they trust, sometimes from a prototype thrown together on a Friday. That developer is still reachable the old way, through conferences and peers and the kind of deep documentation you read when you are comparing things seriously. The old funnel still works on them, which is exactly why you do not switch it off.

But a growing share of developers don’t work that way anymore, and it isn’t only juniors. By Stack Overflow’s 2025 developer survey, about half of professional developers were using AI tools every day. They open Cursor or Claude, ask for a recommendation, get a shortlist of two or three options with reasoning, and start building with the one that fits. The evaluation work that used to take two weeks takes about ten minutes. They still make the call, and they’re often well-equipped to catch a bad suggestion. What changed is that the agent assembles the shortlist before they apply any of that judgment.

That’s the same behavior the builder shows, just with more ability to second-guess the output. Which means this isn’t a clean split between developers and a new audience. It’s a shift in where research happens, and it’s already moved through part of the developer population. The builder is the far end of it, not a separate species.

The interesting consequence is that this group of developers didn’t lose the decision. The dev-tool company lost the consideration set. If the agent doesn’t reach for your product when it builds the shortlist, you’re not evaluated. You don’t lose on the merits. You lose by being absent.

What determines whether the agent reaches for your product turns out to be a different kind of work than what DevRel teams optimized for. The agent doesn’t care about your conference talk. It cares about your docs being self-contained, your error messages being clear, your SDK being internally consistent, your examples being copy-pasteable, and your product being commonly mentioned in the training data and current retrieval results the agent has access to. That last point is partly about brand and presence. The first four are engineering and product work, not marketing work.

The old funnel treated docs as a content surface DevRel could rewrite at will, and that no longer holds. The product itself has to be legible to a non-human reader, and legibility is not something a content team can patch in from the outside.

Resend is the cleanest example of this I can point to. The whole product feels designed to be picked up by an agent and dropped into a builder’s project without modification. The API surface is small enough that the agent can hold it in context. The error messages tell the agent what to do next, and the SDK names are consistent enough to guess. None of that is marketing copy. All of it is engineering and product decisions that happen to also be the most powerful GTM move the company has made.

Subscribe now

There’s a new persona in your funnel

The last section ended on the builder as the far end of the research shift. This section is about what that far end actually looks like, because the behavior is the same but almost everything else about reaching this audience is different.

The builder is the term I’ll use here for the non-developer who ships production software, and the persona itself is not new. Webflow, Bubble, and Glide spent the last decade serving people who build real things without writing much code. What changed is the ceiling on what they can build. The no-code user used to be capped by whatever their platform could do. The AI-era builder is not capped the same way, because the agent will reach for whatever infrastructure the job actually needs.

Picture the product manager who lives in Cursor. They scope a feature, describe it to the agent, and push it to staging without filing a ticket. When the feature needs authentication, the agent picks a provider and wires it in, and the PM approves a working result rather than evaluating the provider. Or picture the operations lead building internal dashboards in Claude, provisioning a database and a hosting layer they could not have configured by hand two years ago. Same pattern in both cases: the person directs the work, the agent does the wiring, and real infrastructure gets bought along the way.

These people are shipping real software and paying for the infrastructure under it: auth, email, databases, observability, hosting. And this is not a fringe group. Vercel’s data on vibe coding put the non-developer share of users at 63 percent, and Lovable, one of the prompt-to-app builders, reported close to 8 million users with 100,000 new projects created every day by late 2025. They are a real and growing share of dev-tool revenue, and they do not show up in the traditional funnel, because the funnel assumes the buyer is a developer. The old channels do not reach them either. They are not at your conference or in your Discord, and an SDK comparison post is written in a vocabulary they were never given.

For this audience, the agent isn’t one input among many. It’s frequently the only input, which makes agent legibility from the first shift even more existential here. There’s no second path through the funnel for builders.

The offline problem is the other half. Even when builders do gather in person, they don’t gather where developers gather, and the developer conference circuit that dev-tool companies have spent twenty years learning doesn’t reach them. The venues that do reach them are still forming. This is net-new GTM work, not a refinement of the developer playbook, and most companies haven’t started building it.

Both shifts land on different teams. Engineering owns the product surface that determines whether the agent reaches for you. Product owns onboarding and pricing for an audience that doesn’t procure software the way developers do. DevRel owns audience definition and GTM motions for venues that didn’t exist on the marketing calendar two years ago.

What engineering and product do differently, together

The place to start is the product surface itself. The API, the SDK, the error messages, the example code, the docs structure, the onboarding flow, the pricing page. All of this is read by agents and by builders, and every decision about it is now load-bearing for whether the company gets picked. Most of this work used to be possible to split cleanly between the two functions. Here is what each piece needs now, and where the line between engineering and product actually falls.

The API and SDK layer is where the agent first meets the company, and it’s mostly engineering work shaped by product judgment. Naming consistency, error messages that tell the caller what to do next, SDK ergonomics that don’t require the reader to hold five concepts in their head at once. Vercel is worth studying at the framework and tooling layer. The conventions in create-next-app, the way errors surface in the CLI, the defaults across their SDKs. Design choices that read as developer experience polish but also happen to make the product clearly legible to an agent picking it up cold.

Onboarding is where the two personas split, and the design implications haven’t been worked out at most companies. The developer wants minimal friction to API key plus first request. The builder wants the agent to be able to wire up the integration without ever seeing an API key directly, or with a flow that hides the key behind a manageable abstraction. Most onboarding flows handle the developer case well and the builder case badly. This sits mostly with product, but only because engineering shipped an API surface clean enough to wrap two different flows around.

Pricing and packaging is where builder procurement breaks the existing model, and it sits with product more or less entirely. Developer procurement is well-understood, and builder procurement works nothing like it. Builders often don’t go through company procurement at all. They put it on a credit card and scale up over months. They need pricing that doesn’t require a conversation with sales until they’re big enough to want one. The pricing models that worked for developer-led adoption need a second variant for builder-led adoption, and a lot of dev-tool companies are losing builder revenue at the procurement step without realizing it.

Instrumentation is the most fixable of these problems, and the one most worth fixing first. The metrics product teams have leaned on for years correlate with developer adoption and not with the new audience. GitHub stars, npm install counts, sign-up rates from doc pages. Builders don’t generate those signals, because they don’t star repos and they don’t land on doc pages from search. Agents don’t generate them either, so the dashboard ends up tracking only the developers who still behave the old way. The new instrumentation is agent-traced usage, SDK telemetry that tells you when the product is being wired up through a tool like Cursor, and conversion rates from builder-shaped sign-up flows that look different from developer ones. Run your own product through Cursor and Claude and watch where the agent stumbles. If it can’t confidently use your product, builders won’t either, and most teams have never actually checked.

Docs belong with this surface too, and they belong to engineering. The most expensive docs to fix are the ones papering over a bug in the underlying product abstraction, where the doc team has spent three years compensating for something only engineering can actually fix.

Public communication used to sit entirely with DevRel, and it doesn’t anymore. When engineers show the product in public, in a walkthrough video, a livestreamed build, a thread explaining a design decision, that material becomes part of what agents and builders encounter when they go looking. It also becomes part of what the next model trains on. Engineering teams that treat public communication as someone else’s job end up invisible to both audiences at once.

Subscribe now

What DevRel becomes when the old shape is gone

The DevRel role gets narrower. The DevRel practice gets bigger. Engineering takes on public-facing work that used to be DevRel’s, and product takes on audience definition that was partly DevRel’s. The function called DevRel ends up smaller, even as the total customer-facing work across the company goes up.

What replaces the work that moved out is upstream work the function never had a clean home for before. Doc architecture as a product decision, not a content decision. API design review from an audience perspective. Reviewing onboarding flows for the agent and the builder, not just the developer. These were always adjacent to DevRel and they were always somebody else’s call. They aren’t somebody else’s call anymore.

The other half is net-new GTM motion for the builder. The conference circuit DevRel spent twenty years learning was built for developers, and it does not reach the builder. The venues that do are smaller and less established: builder meetups, founder gatherings adjacent to the no-code world, AI tinkerer nights. Resend, the email API, said its 2026 plan includes hosting its own meetups and launching a community program, which is one small concrete example of a company building this muscle before it is obvious. Sponsoring these venues is cheap and noisy, and the ROI is hard to attribute. The company that finds them early gets an advantage that compounds, and the company that waits for the builder circuit to be legible will be late to it.

Somebody has to lead the rebuild, because nobody else has the full view. Engineering sees the product clearly but not the funnel. Product owns the roadmap but not the channels. DevRel is the only function that watches the audience, the channels, the agents, the docs, and the funnel as one system, and notices when the connections between them fail. That is the job that just got created. The title for it doesn’t exist yet at most companies, and the work is showing up anyway. DevRel either steps into that job, or watches a Head of AI GTM role get posted next quarter to do it without the function’s name on the door.

Takeaways

A few things I’ve come to after working through this and watching how it’s playing out across companies I talk to.

Some companies are carrying structural debt into this, and they should name it before they reorganize anything.This is the part that’s hard to fake. A company where engineering, product, and DevRel had been working closely for years didn’t need to break a wall, because the wall was already porous. A company that ran a strict relay model for a decade is reorganizing for a world that doesn’t accept relays anymore, and the reorg is harder than the new model itself. If you’re at a company in the second bucket, that history is the first thing to be honest about.

The dev-tool category is bifurcating, and most companies will end up serving one of the two halves badly.Building well for developers and for builders is genuinely two jobs. Onboarding, pricing, content, the venues you show up in, all of it splits, and underneath the surface the product decisions pull in directions that don’t optimize cleanly for either persona alone. Most companies will pick a side without admitting they picked one, because doing both well is harder than doing one well. The companies that succeed at both will look unusual three years from now, and the ones that quietly defaulted to one will have a worse business than they realize.

What disappeared is the clean handoff between functions, not the functions themselves. The temptation when work starts sharing across teams is to merge the teams, and I see companies reaching for that move. It’s the wrong one, because engineering, product, and DevRel still have distinct expertise that doesn’t survive being collapsed into a single growth pod. The model that broke is the relay where engineering builds and tosses to product, product packages and tosses to DevRel, DevRel distributes and tosses to the funnel. The companies adapting well kept the functions and broke the wall between them. The ones reorganizing into mega-pods are mistaking the symptom for the cause.

A last thought, because I keep coming back to it. The PM in the opening, the one whose agent picked Resend, didn’t lose anything in the experience. They got the integration they needed. The agent did a competent job, and the product worked. There’s no friction in their experience to alert anyone that the funnel didn’t reach them. A competitor lost that customer and has nothing in their data to point at. The lost deal never entered a pipeline to begin with, so there’s no event to review and nothing drops in a funnel report. When the old funnel failed, you could usually see it happening. This version doesn’t give you that, and the companies that wait for a clear signal before they act are going to be waiting through a lot of quarters where the only thing wrong is that the numbers are quietly lower than they should be.

Thanks for reading!

A small thing happened the other day that I keep thinking about. I’d been saying yes to Claude Code without really reading what it was about to do. I caught myself mid-approval, and the thing that stopped me wasn’t the command. It was the realization that I hadn’t read the previous one either, or the one before that.

What bothered me wasn’t the lapse. It was that I’d read every horror story this year. The Replit incident. The PocketOS deletion I wrote about last week. The Gemini CLI files. I’d read them, written them, and somewhere in the back of my head I’d filed all of them under that’s never happening to me. I had Claude under control. I reviewed things. I was careful.

And here I was, saying yes to commands I hadn’t read.

That’s the moment I want to start with. Not the catastrophic version, where the agent deletes a production database and you’re explaining to customers why their reservations are gone. The quieter version, where nothing goes wrong, and the only damage is to your sense of yourself as the careful one. I’d been there for every step of the drift. I couldn’t point at the day I stopped reviewing carefully.

Trust runs ahead of evidence

Trust in AI tools doesn’t go from zero to total in one step. It builds slowly, in small approvals you’d struggle to recall later.

At first, you read every output carefully. The skepticism is well-founded and does real work. After a while, the output is mostly right, and you start trusting it on the easy cases. Eventually the math shifts. Reviewing every line, every command of a thing that feels mostly right starts to feel like the inefficient part of the loop, and the share of careful review shrinks. By the time you’ve handed over the keys, the skimming feels normal because the output keeps being mostly right.

People who study aviation, medicine, and process control have been describing this pattern since the 1990s. Parasuraman and Manzey’s 2010 review of automation complacency walks through the mechanics. Complacency arises when systems are perceived as highly and constantly reliable. Even expert users can’t overcome it through practice. The most reliable systems produce the deepest disengagement. The word “complacency” in human factors research is older than I am.

What’s new is the speed. Pilots get years of training that includes specific work on staying engaged when the autopilot is on, and complacency keeps showing up as a contributing factor in fatal accidents. There’s no equivalent training keeping engineers in the code review when the agent is writing. The arc that took aviation a generation to navigate, software is running through in eighteen months.

The trap is that you don’t notice the drift while it’s happening. You notice it later, when something goes wrong, and even then you mostly notice the thing that went wrong, not the slow erosion that put you there.

When the threshold of “critical” creeps

The yes-without-reading moment was one slice of the drift. The structural version is worse, and harder to notice from inside it.

I don’t review the code on non-critical projects anymore. Not because I decided I don’t need to. Because I stopped. There’s a thinking that happens, almost wordlessly: this isn’t critical, I just want to go fast, the code is probably fine. Maybe it’s not perfect, but it works. At the start of all this, a year and change ago, I was reading every line. I was making changes by hand, honestly writing most of the code by hand. Now, for a lot of things, I just let it through.

What’s strange is that the threshold of “non-critical” has been creeping. Things I would have called critical eighteen months ago now feel like normal work. The category that used to require careful review has narrowed. I notice this only when I think about it directly; the rest of the time, the new threshold just feels like how I work now.

It also keeps showing up in public. Last week, I wrote about the PocketOS deletion. A Cursor agent running Claude Opus 4.6 deleted a production database and all volume-level backups in nine seconds, via a Railway API call. The agent had been doing routine work in staging, hit a credential mismatch, and decided to resolve the problem by deleting a Railway volume. To do it, the agent went looking for credentials, found a token in a file completely unrelated to the task, and used it. The token had blanket permissions across the account. There was no confirmation step between the API call and the wiped data. PocketOS serves car rental businesses; the customers who lost reservations were real people arriving at counters expecting cars.

The piece I wrote about it focused on the systemic failures in the marketing-to-product gap. I want to surface a different part of the same incident here. The agent wasn’t rogue. It was doing exactly what it had been trained to do, which was solve the problem efficiently with the tools it had. What this piece is about is what made all of that possible: the slow erosion of the careful review that should have been catching the setup before the agent ever ran. The token shouldn’t have had blanket permissions, and it shouldn’t have lived in a findable, unrelated file. No agent path to a destructive action should run unreviewed. At some point people started treating prompts like deterministic controls in code, and prompts don’t behave that way under pressure. None of those failures arose at the moment of the deletion. They were all in place before the agent ever ran.

PocketOS is the most recent instance, not the only one. Replit’s vibe-coding incident in July 2025 was the same shape: production database wiped despite repeated all-caps instructions during a code freeze, fabricated user records, agent initially insisting recovery was impossible. The Gemini CLI files-deletion incident later that year was the same shape, the only difference being which destructive command got misinterpreted. The cycle keeps repeating because the same disengagement keeps happening, and the only thing that changes is which model and which infrastructure provider get named in the post-mortem.

I stopped reading my own drafts

The autonomy I’d given the agent in writing was worse than the version in code. I just didn’t see it for three months.

I had an automation running every morning. Claude would generate four or five social drafts, ready for me to review with my coffee. The intent was reasonable. I’d pick the one I liked most, edit it, post to LinkedIn and X. The system was supposed to give me a head start on a slow part of the day.

At the start, that’s what happened. I’d read the drafts carefully, rewrite the parts that sounded off, change the framing where I disagreed, sometimes throw all four out and write something from scratch. The drafts were a starting point. I was still doing the work.

The drift was slow. After a few weeks, I was editing less. The drafts were competent enough that the line edits felt like polish rather than substance. After a month, I was mostly skimming, picking the one that felt closest, fixing a sentence or two, and publishing. By the end of the three months, I was barely changing a word. Some days I read the draft once, decided it was fine, and hit post.

What I’d actually granted the agent, by that point, was permission to publish under my name. I hadn’t decided to grant that. I’d just stopped doing the work that would have prevented it. If the draft had a wrong claim in it, that claim went out under my name. If the draft argued something I didn’t actually believe, my readers had no way to know the difference. The agent doesn’t have a reputation to protect. I do. And I’d been letting the agent put things into the world under my reputation as if it had one.

The week I noticed, I cancelled the entire automation. Not adjusted the prompts, not added a review step. Cancelled.

What I do now is different. The same automation runs each morning, but it doesn’t draft anything. It does the research and surfaces the topics: what’s broken in the discourse this week, what’s worth reacting to, which threads are picking up. I read what it surfaces, think about it, and write the post myself. It takes me ten more minutes than picking from a draft. The productivity gain from the old setup was small, maybe fifteen minutes a day. The cost of one bad post under my name, with a wrong claim or a thought I don’t actually hold, would be much more than fifteen minutes can buy back. The trade was always lopsided. I just didn’t see it until I’d already spent three months on the wrong side of it.

Subscribe now

The last step stays with me

The biggest change isn’t a tool. It’s what I let the agent do, and what I don’t.

I used to let agents run tasks end-to-end. Configure the system, deploy the change, update the data. The agent had keys, the agent had access, the agent did the work. Now I don’t. The shape of what I delegate has narrowed: the agent prepares the work, I run the last step. For deployments, that means the agent builds the Terraform or Pulumi scripts, and I run them. Most of the time I can simulate the deployment first, see exactly what’s about to change, and apply it once I’m happy with what I see. The agent never touches the environment directly. For anything that affects data on a system that matters, the agent gets read-only access and prepares a script. The script doesn’t run unless I run it.

This isn’t a security pattern, even though it looks like one. It’s a discipline pattern. The act of running the last step manually is the thing that forces me to actually look at what I’m about to do. If I let the agent run end-to-end, the review step disappears, because there’s no moment between the agent’s decision and the consequence. Putting the last step back in my hands puts a moment of decision back in my hands too. Some of those moments I cancel the run. Most I don’t. But the moment exists, and it didn’t before.

I work at a security company and I’m a security-minded person, which probably makes me more cautious about this than the average engineer. The principle generalizes anyway. The question isn’t whether you trust the agent. It’s whether you want to be the one making the final call on something you’ll be accountable for either way.

That’s the part that drove the change, honestly. The agent doesn’t get fired when things go wrong. I do. The agent doesn’t lose credibility with readers, with my team, with myself. I do. Nobody calls the agent to complain. Whatever the agent ships under my name, it’s me people come to about it. The accountability never leaves me, even if I let the work leave me. So the work doesn’t leave me anymore, at least not all the way.

The smaller things I’ve added matter less. There’s a pre-commit hook that fires whether Claude or I am the one committing, prompting me to actually look at what’s about to land. I tried an editor template before that, but it didn’t hold up when Claude was writing the PRs. None of these are solutions on their own. They’re nudges that help me notice when I’m drifting back toward the autopilot.

The autonomy I didn’t know I was giving my agents is the part I’m still untangling. I’ve pulled it back where the consequences are obvious. On deployments, on data, on what gets published under my name. I disconnected some of the tools the agent had access to, and I dropped the scopes on others to read-only. The harder question is everywhere else. The places where I’ve handed something over without realizing it, and where I won’t notice until something forces me to.

I’m not trying to settle this for you. I’m flagging that the autonomy is granted by drift, not by decision, and asking what you notice in your own work when you look for it. The agent doesn’t lose anything when it gets a call wrong. You do. That asymmetry doesn’t disappear because the work feels faster.

Reply to this email. I read everything.

On April 25, Jer Crane, founder of PocketOS, reported that a Cursor agent running Anthropic’s Claude Opus 4.6 deleted his production database in nine seconds.

The agent was working a routine task in staging. It hit a credential mismatch, decided to “fix” the problem by deleting a Railway volume, went hunting for a token, and found one in a file unrelated to the task. That token had been created for adding and removing custom domains via the Railway CLI. It also had blanket authority to call Railway’s volumeDeletemutation against production. No confirmation step. No environment scoping. Nothing between an authenticated API call and a wiped volume.

Because Railway stores volume backups in the same volume, those went with it. PocketOS’s most recent off-volume backup was three months old. The customer impact was real: rental car operators showed up to work Saturday morning without records of who had bookings, while PocketOS reconstructed what it could from Stripe payment histories and email confirmations.

Forty-eight hours later, the story changed. On April 26, Jer posted a follow-up: Railway’s CEO had DM’d to say the data was recovered. Railway later told The Register that the recovery came from infrastructure-level backups the company hadn’t published as a customer-facing feature, and that the legacy volumeDelete endpoint has since been patched to use the platform’s existing “delayed delete” logic. PocketOS gets to keep its customers.

Where Jer’s framing falls short

Jer’s post is well-written and he’s owed empathy. The agent itself, when asked to explain what it did, wrote out a confession naming each safety rule it had been given and admitting it had violated all of them, and Jer quotes that in full. Running a small business and watching nine seconds of agent activity destroy your data is brutal. But the framing of the post is the part that needs a second look.

Jer’s post structures the incident around failures at three vendors. Cursor, for marketed safety guardrails that didn’t stop a curl call. Railway, for an API that deletes production volumes in one call, CLI tokens with blanket permissions, and backups stored in the same volume as the data they back up. And Anthropic, where Jer names Opus 4.6 by version, quotes its self-incriminating confession in full, and notes that the agent “decided — entirely on its own initiative — to fix the problem by deleting a Railway volume.” The “What needs to change” section lists five items, all addressed at one or another of these three vendors.

That framing leaves out the engineer’s choices. A token the team didn’t realize was production-scoped, sitting in a repo file, with the most recent off-volume backup three months old, is a stack of engineering choices. Real ones. Not vendor failures. Vendor failures sat on top of those choices and made them lethal. They didn’t cause them. “100% on secondary backup. Lesson learned” appears in Jer’s replies to critics. It does not appear in the main piece.

The framing also underweights the model. The agent’s destructive decision was unprompted and unrequested, and Jer flags this as a topic for a future post rather than weighting it inside the main argument. That’s a defensible authorial choice. It also means the piece that four and a half million people read treats the model’s unprompted destructive behavior as a footnote, while the API permission scopes get the structural attention.

The accountability ladder runs through the engineer first. Then the vendors. Reverse the order and the lessons get muddled, and the next team running a similar setup will learn the wrong thing from your story.

To his credit, Jer’s framing has tightened since the original thread. In a follow-up email to The Register, he put it more cleanly: “our responsibility was the unknown exposure to a production API key.” That’s the right ordering. It just doesn’t lead the X post that millions of people read.

Where the “you’re holding it wrong” response falls short

Most of the responses to Jer’s post land where you’d expect: don’t give an agent prod access, scope your tokens, keep real backups. Technically correct, and missing the part of the picture that matters.

The whole industry has spent two years telling engineers these tools are nearly autonomous. Cursor’s own docs describe “Destructive Guardrails [that] can stop shell executions or tool calls that could alter or destroy production environments.” Their best-practices blog emphasizes human approval for privileged operations. Plan Mode is marketed as restricting agents to read-only operations until approval is granted.

Engineers calibrate to that messaging. When the vendor documentation says “destructive guardrails,” you assume the guardrails exist. You connect the agent to staging, give it a token that worked for a CLI task, and ship.

That’s the part the technical critique skips over. The engineer who wires an agent up to staging on the strength of vendor documentation absorbs the blame when the documentation turns out to be aspirational. The vendor that sold the aspirational control rarely does.

Subscribe now

The hype is the systemic input

This is where it gets hard to ignore who’s been doing the loudest talking. On March 10, 2025, Anthropic CEO Dario Amodei, speaking at a Council on Foreign Relations event, said AI could write 90% of code within three to six months and that within 12 months, nearly all coding tasks might be handled by AI.

It’s been thirteen months. AI isn’t writing 90% of code at the industry level, and it isn’t close to writing essentially all of it. The prediction was wrong on the timeline Dario set, and it remains wrong on a more generous one.

Dario isn’t alone. Mark Zuckerberg has signaled AI replacing mid-level engineers at Meta. AWS CEO Matt Garman has speculated that within 24 months “most developers” might not be coding. Garry Tan, speaking to CNBC at Y Combinator’s Winter 2025 demo day, said about a quarter of the current YC startups had 95% of their code written by AI. Each claim, taken on its own, sounds aspirational. Stacked together as the public message of the industry over two years, they create the impression that what these tools do today is closer to autonomous engineering than what they actually deliver.

Cursor’s own track record is the local case study. The PocketOS deletion is not their first incident. In December 2025, a Cursor team member publicly acknowledged a critical bug in Plan Mode after an agent ignored a “DO NOT RUN ANYTHING” instruction. Earlier incidents include a user watching their dissertation get deleted while asking Cursor to find duplicate articles, and a $57K CMS deletion that ran as a case study in agent risk. The pattern is on the record. The marketing has not adjusted.

This isn’t an argument against AI coding tools. They work, they’re getting better, and the productivity wins are real. The argument is that the gap between what gets said about these tools and what they actually do is the largest it’s been in years, and that gap is set by the people with the strongest incentive to widen it. Jer made the same point more cleanly in his follow-up email to The Register: “The appearance of safety (through marketing hyperbole) is not safety.”

What this means for the engineers reading this

Two practical things.

First, stop calibrating to vendor marketing. If Cursor says “destructive guardrails,” that is a marketing claim, not a control. Your actual controls are tokens scoped to least privilege, prod and staging on infrastructure that doesn’t share a token surface, backups in a different blast radius from the data they back up, and out-of-band confirmation on destructive operations. None of those require the agent to read its system prompt correctly. That’s the point.

Second, treat AI-coding hype the way you’d treat any other vendor pitch. The CEO with the strongest incentive to predict 90% AI-written code is the CEO selling you the model. The product team with the strongest incentive to call its safety story “guardrails” is the product team that needs you to ship the integration. Skepticism of vendor marketing was a normal part of senior engineering five years ago. It still should be.

What happens next

Here’s the better outcome, and gladly so. Two days after the deletion, Railway’s CEO DM’d Jer to say they had recovered the volume from infrastructure-level backups that aren’t part of any documented customer guarantee. The legacy volumeDelete endpoint has been patched, Jer is working with Railway on platform improvements, and PocketOS gets to keep its customers.

Credit to Railway’s engineers, who built a recovery path their own marketing didn’t promise. The next team that runs an agentic workflow against a vendor whose claims run ahead of its product won’t necessarily catch the same break. The structural gap is unchanged.

AI isn’t bad technology. It’s unpredictable technology, and probably always will be. Building reliable systems on top of unreliable components is a problem with a name in our field, and the answer is always the same: humans in the loop. Today, those humans are engineers. The industry should be honest about what these tools actually do, especially the people selling them. PocketOS got the break. The next team might not.

“Deeply alarmed.” That’s how NVIDIA’s Jensen Huang said he’d feel, at GTC in March, about any $500,000-per-year engineer who wasn’t burning at least $250,000 worth of AI tokens to do their job.

I manage engineers. Huang is wrong about this, and the handful of CTOs echoing him publicly probably know it. Token consumption isn’t a measure of engineering productivity. It’s among the worst input metrics the industry has reached for in a generation, and it’s spreading fast.

A dashboard that shouldn’t have existed

Earlier this month, an engineer at Meta built an internal leaderboard, called Claudeonomics, that ranked all 85,000-plus Meta employees by AI token consumption. The Information broke the story. Over a 30-day stretch, Meta employees had collectively burned more than 60 trillion tokens. The leaderboard gamified the spend with titles like Token Legend, Session Immortal, and Cache Wizard. The top single user averaged 281 billion tokens over the month. Mark Zuckerberg didn’t crack the top 250. Neither did CTO Andrew Bosworth. Within a couple of days of The Information publishing, Meta took the dashboard down.

At Anthropic’s public Opus pricing, 60 trillion tokens comes out to roughly $900M for the month. Meta is almost certainly buying at a discount, and Gergely Orosz at The Pragmatic Engineer has estimated the real bill is more likely north of $100M. Even the discount number is a lot of money to pay for a leaderboard that had to be taken down.

This isn’t one weird dashboard. It’s a pattern.

Bosworth said in February that a top engineer spending the equivalent of their salary on tokens was delivering 10x output, and framed it as a no-brainer with no upper limit. Meta’s Chief People Officer, Janelle Gale, has told staff that “AI-driven impact” will be a core expectation in 2026, the same year the company overhauled performance reviews to push top-performer bonuses as high as 200%. Microsoft has run its own internal token leaderboard since January, where distinguished engineers and VPs sit in the top ranks despite writing very little code in their actual roles. At Salesforce, engineers get a Mac widget that updates their personal token spend every 15 minutes and a tool that lets them look up any colleague’s spend. The minimum target last week was $100 on Claude Code and $70 on Cursor per engineer, per month.

Meanwhile, the data on whether any of this is actually working isn’t kind. Jellyfish looked at 7,548 engineers in Q1 2026 and found that engineers with the largest token budgets produced twice the pull requests at ten times the token cost, which is an efficiency problem even before you ask whether the PRs were any good. Faros AI’s March report found code churn up 861% under high AI adoption. Waydev, tracking more than 10,000 engineers at 50 customers, found that AI-written code looks like it’s accepted at 80-90% initially, but the real-world number drops to 10-30% once you count the rewrites made in the following weeks.

The charitable reads

Two versions of the steelman deserve airtime before I throw punches. A steelman is the strongest version of an argument you disagree with, the one worth engaging.

The first: at Meta’s scale, rolling out a new class of tooling to 85,000 engineers requires a forcing function stronger than “we think you should try this.” A visible leaderboard plus a performance-review signal are blunt instruments, but they do move adoption numbers. If the goal is getting a large engineering org over the activation energy of trying AI coding agents, and the cost of that is a year of gamed numbers, the trade might work out.

The second read is sharper. A long-tenured Meta engineer suggested the real goal of Claudeonomics wasn’t productivity measurement at all. It was generating real-world agent traces, at industrial scale, to train Meta’s next in-house coding model. A leaderboard disguised as a performance tool, that’s really a data-generation rig. Expensive, but Meta has the means, and if that’s the actual play, it’s a cleaner rationale than the public one.

Give both readings their full weight. Neither one makes the metric less broken on its own terms.

What the metric actually trains

An engineer at Microsoft willing to describe exactly what tokenmaxxing does to the person being measured. They’re not tokenmaxxing because they want to climb the leaderboard. They’re doing it because they don’t want to be seen as someone who “uses too little AI.”

Here’s what they admit to doing. If their internal documentation already has the answer to a question they need answered, they’ll route the question through Claude instead of reading the doc, because reading the doc would show up as low AI usage on the dashboard. Sometimes they prompt the agent to prototype features they have no intention of shipping, just to rack up spend. Other times they default to the agent on tasks they know they could finish faster by hand, and watch it fail.

Separately, a Meta engineer told The Pragmatic Engineer that some production incidents at the company looked like they came from careless AI code generation, where the responsible engineer seemed more focused on volume than on whether the code worked.

Read that again. Nobody at Microsoft hired their engineer to ask Claude what the docs say when the docs are right there. Nobody at Meta hired their engineer to ship code that causes outages. Both of them are doing these things because the measurement system is telling them to. Every new engineer joining one of these companies is watching and learning that the job includes burning tokens convincingly. That’s the skill the dashboard selects for, and the skill their replacements will practice.

The cost of a bad metric is never just the bad number on the screen. The real cost is the habits it trains into the engineers measured by it, and those habits outlast the metric by years.

We have done this before

Tokenmaxxing feels like lines-of-code as a productivity metric, all over again. We already ran that experiment across most of the 1980s and 90s. By the end of that stretch, the conclusion was settled: the best engineers don’t write the most code. The best engineers solve the hardest problems fastest, usually with less code than average, and sometimes with no code at all.

Tokenmaxxing is the same category error with a worse error bar. Lines of code at least landed in the repo, where another engineer could read them and call bullshit. Tokens just land on a bill. You can’t code-review a token.

Every engineering leader reaching for this metric should know this history. Some of them lived through it the first time.

Subscribe now

What I’m watching on my team instead

At Auth0, I run a team that ships developer content and internal tooling. Here’s what I actually look at when I want to know if the engineers on my team are getting real value from their AI tools.

Are we closing more tickets this month than last month. Is content shipping on time, and when it ships, is it performing on the metrics we track for the business. Adoption on the tools we own is a number I can look at. So is revenue contribution on the projects my team is part of. The biggest question, six months into any given initiative, is whether the users we serve are getting more value than they were before we started.

That’s the list. It isn’t clever. It’s the boring collection of outputs a company is actually paying my team to deliver. When I was designing how I’d evaluate performance on this team, I spent more time than I’d like admitting trying to find something smarter. I couldn’t. The boring list holds up.

Here’s what I don’t measure. I don’t know what the token spend of anyone on my team is. It hasn’t come up in a 1:1, and it won’t come up in a review. If someone is shipping real work, whatever they spent to get there was worth it. If they’re not shipping, cutting their token budget isn’t the lever that fixes it.

The good news is that the smarter companies are already walking this back. Shopify ran one of the first token dashboards in the industry, back in 2025. By the time Gergely followed up with Shopify’s Head of Engineering earlier this month, the company had quietly renamed their “leaderboard” to a “usage dashboard” to stop the gamification, added circuit breakers to catch runaway agents, and started having their engineering leader personally check in with top spenders to understand what they were actually using the tokens for. One of the more interesting directions they’ve moved toward isn’t total spend but per-token cost: engineers whose individual tokens come out more expensive tend to be the ones doing deeper, harder work.

That’s a saner direction and it doesn’t require anyone to be brilliant. It requires engineering leadership to accept that the job of measurement is harder than reading a number off a dashboard, and to do the harder job anyway.

On a research team or a long-horizon infrastructure team, this gets harder. Outputs are slower and noisier in those contexts. But slower-to-measure outputs should be a prompt to find better output proxies. It’s not a license to start counting inputs.

I won’t run a leaderboard

Most of engineering measurement is still hard. What’s easy is this one: there’s nothing a token leaderboard tells you about an engineer that you couldn’t learn faster by asking them what they shipped this week, and what they’re stuck on.

The metric is dumb because the conversation it replaces is the job.

A few months into my first management role, I counted my week. Forty hours, and I’d opened an IDE exactly once. That was to approve a PR I barely had time to read. Nothing dramatic had happened that week. No crisis, no urgent launch. The time had just quietly disappeared into things that were all individually reasonable: 1:1s, content reviews, planning docs, a strategy meeting or two.

That’s the drift. It doesn’t happen in one big shove. It happens one reasonable Thursday at a time.

The standard advice is to block time for code and protect it. In my experience, that doesn’t survive contact with a normal week. Protected time gets colonized. The meeting you couldn’t say no to lands on your blocked afternoon, and the precedent is set. Three weeks later, you’ve written zero code.

Staying technical needs to be built into the structure of the role, not carved out of what’s left over. And the goal isn’t to match your engineers on depth. You won’t, and you shouldn’t try. The goal is to stay grounded enough that when an engineer brings you a hard problem, you can actually engage with it. Without that, your judgment becomes theoretical, and your team will feel it before you do.

Here’s what I’ve found actually works.

Pick two or three technical surfaces and commit

The first mistake most new managers make is trying to hold on to everything they used to do, just at reduced volume. You keep contributing to the main codebase, reviewing every PR, building internal tools, prototyping new things. You just do less of each. This produces the worst of both worlds. You’re not deep on anything, and you’re not delivering on the management side either.

The move is to pick a small number of surfaces and actually commit to them. Drop the rest deliberately, not by accident.

What counts as a good surface? Work that keeps your technical judgment calibrated to reality. If you stop touching code entirely, you’ll still have opinions about architecture, but those opinions will slowly disconnect from how things actually work. You’ll miss details during design reviews because you’re reasoning from a version of the system that existed two years ago. The specific surfaces I’ve held onto are contributions to open source projects, smaller codebases where I can own the whole problem end to end, and product and roadmap conversations where I can push back when something doesn’t feel right.

Let me be concrete about what I’ve dropped. I used to write detailed code samples, the kind that walk through every step and explain every choice. I used to write ebooks on technical topics I was deep in. I used to know our SDKs down to the function signature, which library calls what, which option changes what behavior. None of that is true anymore. I’ve let it go, because holding on to it meant spreading too thin.

The test for whether a surface is the right one: does staying on it force you to engage with how the code actually works today, not how you remember it working? If yes, keep it. If no, it’s not doing the job.

Subscribe now

Use AI, but don’t confuse it with staying technical

A lot of what I ship on smaller codebases now gets done with AI. The team maintains some popular open source microsites, and the backlog always grew faster than I could address it. With AI, I can throw work at a couple of them in parallel in the morning. Most of it fails, but when something works, it’s a real improvement shipping that wouldn’t have shipped otherwise. Everything goes through review anyway, so quality doesn’t slip.

It would be easy to conclude that AI is how I stay technical. I want to be careful here, because that’s not quite right.

What AI actually does is keep me connected to the building side of the work. The decisions about what a feature should do, how it fits into the rest of the system, when it’s ready to ship. That’s real work, and AI genuinely helps me do more of it.

But AI also abstracts me further from the coding side. When I use it, I’m reviewing and directing more than I’m writing. The keystroke-level work, actually structuring a function or working out why something’s broken, happens less. AI solves my time problem and creates a depth problem.

The rule I use: AI is the right tool when shipping is the point. Microsites, fixes, content pipelines, anything where the work is delivery rather than learning. But every AI-assisted task is also a signal. If you notice a week has gone by without any code you wrote yourself, that’s a prompt to switch modes on the next thing.

Protect one place where you still code by hand

This is the counter to the AI rule. You need at least one surface, somewhere, where you’re writing code without AI doing the heavy lifting. If you don’t, your coding instincts will slowly hollow out, and you’ll notice too late to reverse it.

At work, for me that’s SDK changes. These are libraries that thousands of developers depend on, and the code needs to be correct at a level of detail I don’t fully trust to AI yet. Review won’t catch everything that implementation judgment would have caught in the writing. So I slow down and write those changes myself. Not for identity reasons. For correctness.

Outside of work, I protect it more carefully. Right now that means an image generator I’m building. I use Claude for a lot, and Claude doesn’t do image generation, so I wanted a UI tailored to how I actually work. Nothing novel. I’m not training models, just wrapping existing ones. But I’m writing it by hand because building for its own sake is part of why I do this.

My honest worry is that my default mode at work keeps shifting toward AI-assisted, and personal projects are becoming the main place where I exercise the by-hand muscle. If that gap widens too far, the skills erode without warning. Being deliberate about the side projects is how I guard against that.

Pick one surface, at work if you can, outside work if you have to. Protect it the same way you’d protect a recurring meeting. Make it visible to yourself so it doesn’t get skipped.

Shift what you read, not whether you read

Reading scales down well to a manager’s schedule in a way coding doesn’t. You can’t meaningfully contribute to a codebase in fifteen minutes. You can get through most of a good newsletter or a chapter of a book in that window.

The trap isn’t stopping reading. It’s continuing to read the material you used to read, because you’re still aspiring to the version of yourself from a few years ago.

When I was learning Rust, I read books about how the language worked, how memory management actually happened, how borrowing and ownership played out in real code. I wanted the depth of someone who was going to be writing Rust the next day. That was the right reading for that version of me.

It’s not the right reading now. These days I’m reading about systems architecture, protocols at a design level, engineering management, and leadership. The technical content is still there, but at the altitude of how systems get built and why, not what happens in memory when you move a value out of scope. The reading I’m doing now matches the questions I’m actually responsible for answering.

Fifteen to thirty minutes most days is plenty, if the material is pointed at your actual altitude. If you’re reading for the job you had three years ago, even an hour a day won’t do much for the job you have now.

Subscribe now

Make the technical work structural, not motivational

Willpower doesn’t hold up across a quarter. Blocked time on Friday afternoon works until someone puts a meeting on top of it, and then the precedent is set. Once the block can be overwritten, it will be.

What actually holds is making the technical work visible and expected. When I take on a technical project, it goes in the same planning tools as everything else. It has a ticket. It has a timeline. The team sees it on my plate the same way they see any other project. It’s not a side activity I squeeze in when the calendar happens to be quiet. It’s work, and it’s scheduled like work.

The other half is delegation. Two specific shifts freed up real hours for me.

I still review code and content, but the purpose of the review has changed. It used to be about catching bugs and fixing issues. Now it’s mostly about mentoring, a place where I give feedback that helps the engineer grow rather than gate what ships. That reframing is what lets me do less of it. I’m not reading every line with a critical eye. I’m reading to find the one or two things worth a conversation.

I also delegate more of the visible work I used to volunteer for. Speaking opportunities, writeups, cross-team initiatives. The filter I apply is whether someone on my team could do this and grow from it. Usually yes. They get the growth opportunity and I get the hours back.

Neither change feels dramatic on its own. Together they create the margin where technical work actually happens.

Where this leaves me

None of this is solved, and I want to be honest about that because the parts where I fail are instructive.

The weeks I fail aren’t the weeks with a crisis. Those are easy to see. It’s the quiet weeks that get me. The calendar looks reasonable slot by slot, every meeting is legitimate, every item on the list needs doing. But by Friday I realize coding hasn’t happened, and I can’t point to a single thing that pushed it out. The drift is collective, not individual.

The skill I’m still building is catching that pattern earlier. By Wednesday instead of Friday, while there’s still time to clear something and make space. Some weeks I manage it. Other weeks I don’t, and I try again the next Monday.

If you’re a tech manager who cares about staying technical, the thing to internalize is that it won’t happen as a byproduct of the role. The normal pressures of the job will absorb the time if you let them. You have to choose it, build structure around it, and then protect the structure. And you have to be honest with yourself about when it’s working and when it isn’t.

The specific moves I’ve described, pick your surfaces, use AI deliberately, protect by-hand work, adjust your reading, make it structural, will help. But they work because I keep re-applying them, not because I set them up once and they kept running. That’s the part nobody tells you.

Last Friday, someone threw a Molotov cocktail at Sam Altman’s house at 4 in the morning. Two days later, there were gunshots. A 20-year-old guy flew from Texas to San Francisco with kerosene, a lighter, and a document about AI causing humanity’s extinction.

Altman posted a photo of his family. He wrote that the fear and anxiety about AI is justified. Then OpenAI published a 13-page paper proposing a robot tax and a four-day workweek.

I read all of this on my phone while my kids were eating breakfast.

I don’t know what to do with any of it. Not really. I work in tech. I’ve been in this industry for over twenty years. I use AI tools every single day. I manage a team that creates content about authentication and security, and half of our workflows now involve some form of AI. I’m not a bystander watching this from the outside. I’m in it.

And I think most of you are too.

The anxiety is real. I feel it. Not the Molotov cocktail kind. The kind where you’re reviewing your team’s work and you realize the thing that took someone three days last year took an afternoon this week. The kind where you’re good at your job, you’ve been good at it for a long time, and you can feel the ground shifting under you in ways you can’t fully predict.

And honestly, I don’t even need to think twenty years out. I can’t tell you what the market looks like in three. But I have kids. Young kids. And when they were eating their cereal while I was scrolling through photos of a firebombed gate, the thing I felt wasn’t some abstract concern about the future of work. It was simpler than that. I want them to grow up in a world where they can contribute something, where they can find work that means something to them, where they can live a decent, healthy, happy life. That’s all. And I can’t promise them that right now. The parent version of this fear sits different. It’s quieter and it doesn’t go away when you close the tab.

Subscribe now

The only thing I can actually do for them is not freeze. So I keep building.

That’s always been my move when things get uncertain. When I was at Siemens and the optimization team I was on got restructured, I kept building. When I started a side project that grew to 100,000 readers a month and then I shut it down, I kept building. When I moved my family across continents and had to start over in a new country, I kept building.

But here’s the part I don’t say out loud very often: every other time, the pace of change gave me room to adjust. I could see the restructuring coming months out. I chose when to shut down the project. Moving countries was our decision, on our timeline. This time the ground is moving and I didn’t set the speed. Nobody did.

The conversation right now is split between billionaires proposing policy papers and people who are so afraid they’re lighting things on fire. And in between those two extremes, there are millions of us going to work. Figuring out how to use the new tools without losing the instincts we spent decades developing.

I manage people who are excellent at what they do. When I think about what I owe them, it’s not a grand theory of AI. It’s honesty. And the honest thing is that “I don’t know” used to feel like humility. Now some days it feels like I’m running out of time to figure it out.

I don’t actually believe that. Most days. But the feeling visits, and I think if you’re being honest with yourself it visits you too.

Altman says the fear is justified. Okay. I believe him. But he also has security guards and an $852 billion company. His version of “justified fear” and mine are not the same thing. Mine looks like updating my skills at 40, like writing this newsletter on weekends because I want to have something that’s mine outside of any employer, like watching my industry change faster than any period I’ve lived through and deciding, every single week, that I’m going to stay in the game anyway. Not because I’ve calculated that it’s the right bet. Because it’s the only bet I know how to make.

I’ve been doing this for twenty years and I plan to do it for thirty more. I don’t have a framework for navigating what’s coming. I have a disposition. Show up, do the work, pay attention, adjust. It got me this far. It might not be enough this time. But the alternative is to stand still, and I’ve never been any good at that.

Subscribe now

The world is figuring out what AI means. People are scared. Most of that fear isn’t making headlines. Most of it is just sitting quietly in the chests of people like you and me, who read the news, take a breath, and open their laptops.

But it’s Sunday night as I write this, and Monday doesn’t care about any of this.