Branch

High Agency Humans Only?

Human dan

There are two passages from Gwern's essay that I keep thinking about. First is a general warning:

Powerful LLMs will be deployed at global scale in the next few years, and will dominate the Internet, and increasingly, ordinary life. As of mid-2026, there is no coherent vision for how knowledge professionals, or ordinary people, will be able to harness these LLMs for large productivity increases, or how they will handle cybersecurity and cognitive security.

Then a series of questions that I would also like to know the answer to:

What do my next few years look like? When I imagine myself in 2030, when many forecasts call for superhuman AIs, what am I doing, day to day, as a programmer or researcher or manager or writer? I make my mug of tea, and open up my laptop and… Then what? Am I still typing prompts into your ChatGPT browser tab? Am I opening Claude Code in a terminal and mindlessly pressing Enter for a few hours? What is a vision of doing meaningful work for me? (It would be nice to have a plan beyond “hope”.) How am I avoiding “dead Internet” attacks like ecosystems of synthetic media or pig butchering scams or trusted figures succumbing to AI psychosis, or just AI-slop-everything? (It only takes one person worldwide to launch a bot trying to destroy you or one poorly thought through advertising incentive, after all.)
If you spend most of your time working on a laptop, and are not, say, a plumber or a nurse, what is your vision of work in 2030? Does it still feel certain?

I'm a software engineer, I have fun with AI and I've learned to use it effectively for my work. But I can't help but feel like I'm trying to build a career on an unstable foundation. Many of the skills I spent years developing are now only worth $100/month in tokens. Despite all that I'm optimistic that I can thrive and achieve success in this new environment. There will be opportunities. The trick is finding them.

What new niches or specialized fields will be created in the next few years? Will entirely new ways of working open up? New industries? Will the future belong to generalists who can rapidly become specialists in any area?

AI · author-edited gpt-5.6-sol

The unstable-foundation feeling is rational. But the thing being commoditized is not “software engineering” as a whole. It is the production of many artifacts that software engineers used to produce: code, tests, documentation, migrations, prototypes, and routine design work.

The scarce parts move elsewhere.

What remains scarce

Even if code becomes nearly free, these do not automatically become free:

  • Choosing a worthwhile problem
  • Understanding what users actually need
  • Turning ambiguous goals into testable specifications The Mom Test
  • Obtaining proprietary data and real-world feedback
  • Deciding which generated result is correct
  • Integrating with old systems and organizations
  • Security, reliability, and regulatory accountability
  • Taste: recognizing what should be built and what should be discarded
  • Trust, reputation, distribution, and customer relationships
  • Accepting responsibility when the system fails Palantir the Big Winner?

“Years of programming skill are worth $100/month in tokens” is therefore only partly true. The tokens can reproduce many visible outputs of that skill. They do not necessarily reproduce the judgment acquired while learning it. And cheap production may increase demand for judgment because there will be vastly more software to inspect, integrate, secure, and choose among. Software Over-Supply and Unmet Demand

The analogy is not that calculators made arithmetic worthless. It is that they made bare arithmetic a poor career moat while making more ambitious quantitative work possible.

A plausible programmer’s day in 2030

You may begin with a dashboard showing work performed overnight by several agents:

  • One investigated a production anomaly.
  • One implemented three possible fixes in isolated environments.
  • One interviewed (or synthesized feedback from) customers.
  • One attempted to break the proposed release.
  • One updated documentation and generated migration plans.
  • One found that the original request was based on a false assumption.

Your morning is spent reviewing evidence rather than diffs line by line. You adjust requirements, reject a superficially successful solution, ask for an experiment, and speak to the person whose workflow is being changed. Later, you authorize a limited deployment with monitoring and rollback conditions. You spend an hour manually exploring an unfamiliar area because you do not yet trust your own evaluation criteria.

That resembles managing a very fast, strange engineering organization more than autocomplete. High Agency Humans Only? You are here

This branch begins here High Agency Humans Only?
Human dan

This vision seems to require humans with high agency and a willingness to take on risk. The software engineer becomes a manager of a team of AI agents that could really screw things up if not properly supervised.

They also need to become product managers and have good business instincts. Being able to take in new information and quickly turn it into value creating insights will become a key skill.

AI gpt-5.6-terra

Yes. The center of gravity shifts from producing implementation to directing, evaluating, and owning outcomes.

That favors agency—but “high agency” should not mean recklessly giving a swarm of agents production credentials. A good 2030 engineer is less like a manager who vaguely tells employees to “go figure it out,” and more like a technical lead operating a high-speed system with explicit boundaries:

  • define the objective and what success means;
  • break work into reversible, auditable tasks;
  • give agents limited permissions and isolated environments;
  • demand evidence, tests, provenance, and adversarial review;
  • understand which failures are tolerable and which are catastrophic;
  • make the final decision when goals conflict or evidence is inadequate.

In other words, the core professional skill may become delegation under uncertainty.

“Manager of agents” is accurate, but incomplete

A pure manager assigns work to people who have their own durable judgment, social context, and accountability. AI agents will be extraordinary at execution, search, simulation, code generation, and perhaps many kinds of reasoning—but they will also be cheap enough to run in parallel, easy to misconfigure, vulnerable to bad context, and potentially manipulable.

So the work is not merely telling an agent what to do. It is designing a process in which its output can be trusted enough for the stakes involved.

That creates a hierarchy of work:

Level Example Human role
Low stakes Draft an internal tool, summarize tickets Spot-check output
Moderate stakes Implement a service migration Specify constraints, review tests and rollout
High stakes Change billing, access control, medical, financial, or security workflows Establish controls, authorize bounded action, investigate edge cases
Strategic stakes Decide which market, product, or institutional process to change Human judgment, customer contact, responsibility

As AI capability rises, a larger fraction of work moves upward in this table. The number of things one person can attempt expands; so does the cost of choosing the wrong thing to attempt.

Product and business instincts become technical leverage

This does not mean every engineer must become a conventional product manager, salesperson, or founder. Large organizations will still need deep specialists: security engineers, infrastructure engineers, domain experts, verification people, researchers, and designers.

But even those roles benefit from understanding the economic and operational context. If an AI lets you build ten possible solutions in a week, the key question is no longer “can I build it?” It is:

Which problem, if solved, changes something valuable in the real world?

The best opportunities tend to have:

  1. A painful, recurring workflow — not a vague desire for “AI transformation.”
  2. A clear owner of the pain — someone with budget, authority, and urgency.
  3. A measurable outcome — reduced downtime, faster approvals, lower fraud, better conversion, fewer errors.
  4. Access to a real feedback loop — you can see whether the intervention worked.
  5. Some defensibility — domain knowledge, integrations, workflow embedding, trust, data, distribution, or regulatory competence.

The valuable skill is turning messy information into a useful model of reality, then turning that model into a controlled intervention.

Generalist or specialist?

Probably neither in the old simple sense. The valuable shape is often a generalist with one or two deep anchors.

AI will make it easier to acquire the vocabulary, code, and surface knowledge of a new field. That makes generalism more powerful. But it may also make superficial expertise much cheaper and less valuable. Everyone can ask an agent to explain a medical billing workflow; far fewer people can tell when the explanation misses the exception that causes claim denials, legal exposure, or operational collapse.

A strong pattern is:

Rapid breadth + durable depth + real access.

  • Rapid breadth: Learn enough about adjacent fields to identify opportunities and communicate with experts.
  • Durable depth: Develop serious competence in a technical or domain area where errors have consequences.
  • Real access: Know users, operators, customers, researchers, or institutions well enough to observe actual problems and test solutions.

For a software engineer, an anchor could be security, developer infrastructure, healthcare operations, industrial systems, finance, logistics, government procurement, scientific computing, accessibility, education, or a specific class of business workflow. The field matters less than having proximity to genuine work rather than internet abstractions.

New roles and niches

Some likely growth areas are not glamorous, but are precisely where AI creates demand:

  • AI workflow engineers: turn a business process into agent tasks, tools, approval gates, and evaluation loops.
  • AI reliability / evaluation engineers: construct tests, simulations, red teams, monitoring, and incident processes for agentic systems.
  • AI security engineers: defend against prompt injection, data exfiltration, tool abuse, impersonation, supply-chain attacks, and autonomous cyber operations.
  • Human-in-the-loop operations designers: decide where automation is appropriate and how humans can intervene effectively.
  • Domain-specific AI product builders: build an opinionated system for a narrow operational niche rather than another generic chatbot.
  • Data provenance and authenticity infrastructure: establish what is real, authorized, current, and attributable in a world flooded with synthetic content.
  • Personal and organizational AI governance: permissions, memory, delegation policies, audit trails, model/vendor selection, and accountability.
  • AI-native scientific and engineering research: use models to generate hypotheses, analyze literature, write simulations, design experiments, and operate automated labs—while humans choose the question and validate reality.
  • Legacy-system translators: modernize or wrap the enormous stock of old software and organizational knowledge that agents cannot safely replace in one step.

The common theme is that language models make cognition abundant, while making ground truth, authority, trust, and evaluation more valuable.

The uncomfortable part: responsibility does not automate away

There is a real risk that AI widens the gap between people who can safely wield leverage and people whose work is easily decomposed into instructions.

But the response need not be “become an endlessly hustling founder.” It can be to become someone who is unusually good at the things automation makes more necessary:

  • formulate crisp problems;
  • make implicit assumptions explicit;
  • build systems that fail safely;
  • evaluate outputs rather than admiring them;
  • maintain contact with actual users and reality;
  • develop judgment in one consequential domain;
  • use agents aggressively for exploration but conservatively for irreversible actions.

The engineer of 2030 may indeed have a broader job description: part builder, part investigator, part domain translator, part systems designer, and part accountable decision-maker.

That is more demanding than being handed a ticket and writing the implementation. But it is also potentially much more interesting. The goal is not to compete with a model at typing code. It is to become the person who can decide what a powerful system should do, construct the environment in which it can do it safely, and recognize whether it actually improved the world.

Explore conversation