Thoughts on Gwern's "Guardian Angels" and the Future of Software Engineering

There are two passages from Gwern's essay that I keep thinking about. First is a general warning:

Powerful LLMs will be deployed at global scale in the next few years, and will dominate the Internet, and increasingly, ordinary life. As of mid-2026, there is no coherent vision for how knowledge professionals, or ordinary people, will be able to harness these LLMs for large productivity increases, or how they will handle cybersecurity and cognitive security.

Then a series of questions that I would also like to know the answer to:

What do my next few years look like? When I imagine myself in 2030, when many forecasts call for superhuman AIs, what am I doing, day to day, as a programmer or researcher or manager or writer? I make my mug of tea, and open up my laptop and… Then what? Am I still typing prompts into your ChatGPT browser tab? Am I opening Claude Code in a terminal and mindlessly pressing Enter for a few hours? What is a vision of doing meaningful work for me? (It would be nice to have a plan beyond “hope”.) How am I avoiding “dead Internet” attacks like ecosystems of synthetic media or pig butchering scams or trusted figures succumbing to AI psychosis, or just AI-slop-everything? (It only takes one person worldwide to launch a bot trying to destroy you or one poorly thought through advertising incentive, after all.)
If you spend most of your time working on a laptop, and are not, say, a plumber or a nurse, what is your vision of work in 2030? Does it still feel certain?

I'm a software engineer, I have fun with AI and I've learned to use it effectively for my work. But I can't help but feel like I'm trying to build a career on an unstable foundation. Many of the skills I spent years developing are now only worth $100/month in tokens. Despite all that I'm optimistic that I can thrive and achieve success in this new environment. There will be opportunities. The trick is finding them.

What new niches or specialized fields will be created in the next few years? Will entirely new ways of working open up? New industries? Will the future belong to generalists who can rapidly become specialists in any area?

The unstable-foundation feeling is rational. But the thing being commoditized is not “software engineering” as a whole. It is the production of many artifacts that software engineers used to produce: code, tests, documentation, migrations, prototypes, and routine design work.

The scarce parts move elsewhere.

What remains scarce

Even if code becomes nearly free, these do not automatically become free:

• Choosing a worthwhile problem
• Understanding what users actually need
• Turning ambiguous goals into testable specifications↳ The Mom Test
• Obtaining proprietary data and real-world feedback
• Deciding which generated result is correct
• Integrating with old systems and organizations
• Security, reliability, and regulatory accountability
• Taste: recognizing what should be built and what should be discarded
• Trust, reputation, distribution, and customer relationships
• Accepting responsibility when the system fails↳ Palantir the Big Winner?

“Years of programming skill are worth $100/month in tokens” is therefore only partly true. The tokens can reproduce many visible outputs of that skill. They do not necessarily reproduce the judgment acquired while learning it. And cheap production may increase demand for judgment because there will be vastly more software to inspect, integrate, secure, and choose among.↳ Software Over-Supply and Unmet Demand

The analogy is not that calculators made arithmetic worthless. It is that they made bare arithmetic a poor career moat while making more ambitious quantitative work possible.

A plausible programmer’s day in 2030

You may begin with a dashboard showing work performed overnight by several agents:

• One investigated a production anomaly.
• One implemented three possible fixes in isolated environments.
• One interviewed (or synthesized feedback from) customers.
• One attempted to break the proposed release.
• One updated documentation and generated migration plans.
• One found that the original request was based on a false assumption.

Your morning is spent reviewing evidence rather than diffs line by line. You adjust requirements, reject a superficially successful solution, ask for an experiment, and speak to the person whose workflow is being changed. Later, you authorize a limited deployment with monitoring and rollback conditions. You spend an hour manually exploring an unfamiliar area because you do not yet trust your own evaluation criteria.

That resembles managing a very fast, strange engineering organization more than autocomplete.↳ High Agency Humans Only? The important interface may not be a chat box. It may be a mixture of:

• Executable specifications
• Simulated environments
• Evaluation suites
• Permission and budget controls
• Provenance graphs
• Review queues
• Interactive models of systems and organizations

Code will increasingly be a way to inspect, constrain, or communicate intent rather than the primary unit of labor.

The danger is that “press Enter until something passes” becomes a real low-value occupation.↳ The Future of Vibe Coding The valuable role is designing the environment in which apparent success is hard to fake.

Likely new specialties

Some will be recognizable extensions of existing work.

Agent reliability and evaluation

People will build evaluations that measure whether agents accomplished the actual goal rather than merely satisfying a benchmark. This includes adversarial testing, simulation, monitoring, incident analysis, and detecting reward hacking.

A good evaluator may become more valuable than a fast implementer.↳ Evals Evals Evals

AI security and permission architecture

Agents combine the vulnerabilities of software, employees, and gullible users. Important work will include:

• Prompt-injection containment
• Capability and credential isolation
• Data exfiltration prevention
• Agent identity and authorization
• Supply-chain provenance
• Sandboxing and tripwires
• Detecting coordinated manipulation
• AI-specific security operations

“Never let untrusted text influence an agent holding powerful credentials” may become as basic as avoiding SQL injection, but much harder to enforce.

High-assurance and verifiable systems

Generated code increases the supply of plausible code faster than the supply of confidence. Formal methods, property testing, model checking, reproducible builds, typed interfaces, and machine-checkable evidence↳ Machine Checkable Evidence? may become much more important.

Vertical AI integration

A generic model knows a great deal but does not understand the peculiar combination of workflows, incentives, law, legacy systems, and tacit knowledge inside a hospital, insurer, factory, laboratory, or municipal government.↳ The Push to Codify Everything

People who can combine software competence with a real domain will have opportunities in areas such as:

• Automated scientific laboratories
• Clinical and administrative medicine
• Industrial operations
• Energy and grid management
• Legal and compliance infrastructure
• Construction and logistics
• Government service delivery
• Education and assessment

Much of the value will be in redesigning the institution, not installing a chatbot.

Human–AI organizational design

Companies will need to answer questions such as:

• What may an agent decide by itself?↳ Should Agents Make Any Decisions?
• What requires two independent checks?
• How is responsibility assigned?
• How should humans remain capable of intervening?
• Which measurements become corrupt once agents optimize them?
• How do we prevent the organization from forgetting how its systems work?

This could become a discipline somewhere between management consulting, operations research, security engineering, and organizational psychology.

Provenance, identity, and reputation

A dead Internet creates demand for scarce authenticity:

• Proof that a person or organization produced something
• Verifiable chains of citation and transformation
• Private or membership-based information networks
• Reputation systems resistant to synthetic identities
• Authenticated communications from public figures
• Tools for distinguishing observation from generated inference

The winning products may not perfectly detect AI content. Detection is intrinsically fragile. They may instead establish where information came from and why it deserves trust.

Personalized and ephemeral software

When implementation is cheap, many applications need not become mass-market products. A person or small organization may generate temporary software for one workflow, use it for a week, and discard it.

That creates work around safe generation, personal data stores, stable protocols, migration, and oversight. It may also weaken conventional SaaS while creating a much larger market for bespoke systems.↳ Journeyman Software Engineers

Robotics and the physical economy

The physical world supplies inconvenient ground truth. Models still need sensors, machines, maintenance, permits, supply chains, and safe deployment. Software engineers who can bridge models and physical systems may be better insulated than those working entirely within easily simulated digital environments.

Generalists or specialists?

Probably neither in isolation. The strongest position is a generalist with one or two sources of genuine depth.

AI makes it easier to become temporarily competent in a new subject. That favors people who can cross boundaries, learn quickly, and synthesize. But it also floods every field with convincing superficiality. That increases the value of people who know when the generated answer is subtly wrong.

A useful distinction is:

• Codified expertise becomes easier to rent from models.
• Tacit expertise remains harder: experience with failures, institutions, people, physical constraints, and unusual edge cases.↳ Make it Explicit
• Frontier expertise remains scarce because the answers are not yet in the training data.
• Accountable expertise remains scarce because someone must be trusted to sign off.

So the future may favor “deep generalists”: broad enough to direct models across fields, but deep enough somewhere to calibrate what good work looks like.

How to position yourself now

A reasonable strategy is to move upward and outward from code without abandoning technical depth.

1. Become excellent at specifying and evaluating work.
   Write acceptance criteria, invariants, threat models, experiments, and failure taxonomies. Treat prompting as the least interesting part of agent use.

2. Own outcomes rather than artifacts.
   “I wrote the service” is less defensible than “I reduced settlement failures by 60% while satisfying these constraints.”

3. Acquire a domain.
   Learn the economics, regulations, workflows, and failure modes of something outside generic software. Repeated contact with real users matters more than another framework.

4. Learn to build safe agent environments.
   Sandboxes, permissions, observability, reproducibility, evals, and rollback mechanisms are likely to compound in value.

5. Preserve your ability to work without the model.
   Not from nostalgia, but because independent reasoning is necessary to audit the model. If you cannot form your own hypothesis, you cannot tell whether an assistant is helping or steering you.

6. Build relationships and reputation.
   In a world of limitless generated claims, trusted collaborators and demonstrated judgment become stronger filters.

7. Prefer projects with real feedback.
   Production systems, paying customers, physical experiments, and measurable operational results provide resistance to self-deception.

8. Seek leverage or ownership.
   If AI greatly increases output per person, some gains will accrue to the owners of products, data, distribution, and capital rather than employees paid for producing artifacts. Small products, equity, and direct customer access can provide optionality.

Cognitive and cybersecurity may become part of ordinary professionalism

The same system that helps you work can also manipulate you, leak credentials, fabricate consensus, or gradually distort your beliefs. Defensive habits will matter:

• Give agents the least privilege necessary.
• Separate untrusted content from actions and credentials.
• Require independent confirmation for irreversible steps.
• Prefer authenticated sources and primary evidence.
• Maintain high-trust human communication channels.
• Do not treat conversational fluency as evidence.
• Keep logs of what information and instructions influenced important decisions.
• Periodically make major decisions away from the model.
• Be especially suspicious of urgency, intimacy, flattery, and claims of secret knowledge.

The social defense is as important as the technical one: a small network of grounded people who can tell one another, “No, this is not real,” or “You are becoming obsessed.”

Much has been said about the collapse in hiring of entry level and junior engineers.

It may be a “tragedy of the commons” effect where every company is making a decision that is good for them now, but potentially disastrous for the industry as a whole later.

In five years there may be a painful shortage of senior level engineers since new ones are not being trained up right now.

What could the consequences of this look like?

What opportunities would emerge from those consequences?

People who are good at mentoring and training juniors could become much more valuable. How would I train that skill set?

The most likely shortage would not be “people with five years on their résumé.” It would be a shortage of people with calibrated judgment acquired through repeated contact with real failures.

AI may compress parts of the path to seniority, but it can also conceal missing understanding. A junior can now produce senior-looking code without knowing why it works, where it will fail, or how to recover when it does. That makes the pipeline problem less visible until something goes wrong.

What a hollowed-out engineering pipeline could produce

1. A missing middle, not merely fewer seniors

Five years from now, companies may have:

• A small group of experienced staff engineers
• Many AI-amplified beginners
• Too few engineers capable of independently owning ambiguous production systems

The missing group will be people who can take a vaguely defined problem, discover the actual constraints, make reasonable tradeoffs, deploy a solution, and remain responsible for it.

This could make current senior engineers extremely overloaded. They would be asked to supervise both humans and agents while handling every difficult incident and architectural decision.

2. Senior-title inflation

Companies may respond by promoting people faster. The number of “senior engineers” may not fall, but the experience represented by the title will.

Interviews will consequently shift away from titles and puzzle solving toward evidence such as:

• Systems operated over time
• Incidents handled
• Decisions made under uncertainty
• Security and reliability judgment
• Ability to explain failures
• Ability to improve other engineers

Verified operational history could become more valuable than credentials.

3. Accumulating systems that nobody understands

AI makes it cheap to add code, integrations, dependencies, and automations. It does not make complexity disappear.

Organizations could accumulate enormous “comprehension debt”: systems that pass tests and work under normal conditions, but whose behavior no employee can adequately explain. Symptoms would include:

• Small changes causing surprising failures
• Agents repeatedly patching symptoms rather than causes
• Security boundaries that exist only implicitly
• Inability to migrate away from vendors or models
• Long outages because no one can construct a coherent system model
• Generated documentation that is extensive but untrustworthy

Maintenance, archaeology, simplification, and recovery would become premium skills.

4. Greater concentration of technical power

If competent technical supervision is scarce, smaller organizations may be unable to safely maintain complex systems. They may outsource more of their operations to model providers, cloud platforms, consultancies, and packaged vertical systems.

This could create a split between:

• Organizations able to sustain internal engineering judgment
• Organizations operating largely through opaque vendor-controlled agents

The former may gain unusual strategic independence.

5. Higher wages for some seniors—but worse working conditions

Scarcity does not automatically make senior engineering pleasant. A scarce senior might become the person who must approve dozens of agent-generated changes, mentor ten nominally independent developers, and remain on call for systems they did not design.

Compensation may rise for engineers with proven judgment, but so may liability, interruption, and burnout. The valuable skill will not simply be “knows a lot.” It will be creating structures that distribute judgment without becoming a bottleneck.

6. More spectacular failures

Missing apprenticeship means fewer people have gradually encountered low-stakes versions of important mistakes. Their first serious lesson may happen in a high-leverage environment.

Possible outcomes include:

• Large security incidents caused by poorly supervised agents
• Financial or operational failures from misunderstood generated systems
• Organizations losing critical knowledge when one senior leaves
• Compliance failures because nobody can explain how a decision was made
• “Normal accident” cascades across many interacting automations

After enough such failures, regulation and insurance requirements may force companies to demonstrate that qualified humans meaningfully supervise critical systems.

Why the market might not correct itself cleanly

Companies can poach experienced engineers instead of training beginners, so each company has an incentive to free-ride on everyone else’s apprenticeship. But once the supply contracts, several things can happen:

• Salaries rise enough that training becomes attractive again.
• Consultancies and specialized academies sell trained talent.
• Large firms build internal apprenticeship programs and retain graduates.
• Professional certification appears in regulated or high-assurance sectors.
• AI makes apprenticeship faster, partly offsetting the missing cohorts.
• Work is redesigned so fewer conventionally senior engineers are needed.

The last two matter. We should not assume that a 2031 senior must be produced through the exact 2021 career path. AI can provide unlimited explanations, exercises, simulations, and immediate feedback. What it cannot automatically provide is trustworthy exposure to real consequences.

The emerging training problem is therefore: How do we give people dense, safe, authentic experience?

Opportunities created by the shortage

Apprenticeship-as-a-service

A company could employ a mixture of experienced and junior engineers, deliver real client work, and explicitly operate as a talent refinery. Clients receive software; juniors receive supervised production experience; graduates become valuable hires.

The hard part—and the moat—would be quality control. A credible program would track what participants actually owned, which incidents they handled, and what decisions they could defend.

AI-native engineering academies

Traditional boot camps taught enough syntax and framework knowledge to obtain an entry-level job. That becomes less useful when models can generate the artifacts.

A more valuable academy would teach:

• Problem decomposition
• Debugging unfamiliar systems
• Reading generated code critically
• Testing and evaluation
• Security boundaries
• Production operations
• Incident response
• Requirements discovery
• Technical communication
• Responsible use of agent permissions

Students would inherit broken systems, investigate synthetic incidents, defend designs orally, and operate services over time. The product would be demonstrated judgment, not course completion.↳ AI-Native Engineering Curriculum

Simulation and engineering flight schools

There is room for realistic environments in which engineers can experience compressed years of operational events:

• Gradually degrading databases
• Ambiguous alerts
• Supply-chain compromises
• Partial network failures
• Misleading dashboards
• Agent-generated patches that fix the benchmark but violate an invariant
• Organizational pressure to deploy an unsafe change

Medicine, aviation, and cybersecurity already use simulation. Software apprenticeship has relied unusually heavily on production accidents happening naturally.

Mentorship infrastructure

Most engineering tools optimize implementation, not learning. New tools could help a mentor see:

• Where the learner relied on AI
• Whether they understood generated changes
• Which misconceptions recur
• Which tasks are just beyond their current ability
• Whether they can transfer knowledge to a new situation
• How much intervention was required

A useful system would not merely score final code. It would capture the trajectory: hypotheses, experiments, revisions, and reasoning.

Fractional technical leadership

Smaller companies may need experienced supervision but be unable to hire a full-time staff engineer. Fractional staff engineers, reliability reviewers, AI-governance leads, and architecture stewards could oversee several organizations.

The high-value version would establish systems and develop internal people, rather than becoming a permanent approval bottleneck.

Comprehension-debt reduction

There may be a large business in making generated systems understandable and governable:

• Dependency and data-flow reconstruction
• Invariant extraction
• Executable documentation
• Architecture simplification
• Security-boundary audits
• Disaster-recovery design
• Replacing agent-generated patch layers with coherent systems

This may resemble a combination of software archaeology, audit, and turnaround consulting.

Evidence-based technical credentials

If résumés and portfolios can be generated, trusted evidence becomes scarce. New credentialing systems could verify that someone:

• Operated a system for a meaningful period
• Diagnosed particular classes of failures
• Made and defended consequential tradeoffs
• Improved another engineer’s independence
• Worked within security and reliability constraints

A cryptographically verified Git history would not be enough. The credential must represent accountable participation in real or carefully proctored work.

Mentoring will become valuable—but not all mentoring

The scarce mentor is not the person who answers questions fastest or rewrites a junior’s code. It is the person who converts novices into autonomous engineers while keeping production safe.

That requires several separate skills:

1. Diagnosis: determining what the learner misunderstands.
2. Task design: selecting work that is difficult enough to teach but safe enough to attempt.
3. Scaffolding: providing only enough help for progress.
4. Feedback: making criticism specific, timely, and actionable.
5. Mental-model instruction: explaining principles rather than local fixes.
6. Calibration: gradually expanding autonomy as demonstrated competence grows.
7. Psychological safety: making uncertainty and mistakes discussable.
8. System design: structuring work so mentorship does not consume the mentor.

The key metric is not “How much did my mentee produce?” It is “What can my mentee now do correctly without me?”

How to train that skill set

Start mentoring one or two people now

Do not wait for formal management authority. Possible settings include:

• New hires on your team
• Interns
• Open-source contributors
• Internal study groups
• Volunteer programming organizations
• Colleagues moving into your specialty

Keep the group small enough that you can observe learning rather than merely answer questions.

Practice diagnosis before explanation

When someone is stuck, resist immediately telling them the answer. Ask:

• What do you currently believe is happening?
• What evidence would distinguish your hypotheses?
• Where does your confidence come from?
• What did you expect this command or component to do?
• What would you inspect next if I were unavailable?

This reveals whether the problem is missing knowledge, a false mental model, weak debugging process, or fear of acting.

Then give the smallest intervention likely to unblock them.

Design a progression of ownership

A useful sequence is:

1. Observe a task being performed.
2. Perform a bounded task with a checklist.
3. Perform it independently with review.
4. Handle an unexpected complication.
5. Explain the task to someone else.
6. Improve the process or checklist.
7. Own the outcome, including monitoring and follow-up.

Do not restrict juniors to endlessly generated low-risk tickets. Give them a narrow system or workflow they can genuinely own. Ownership is where judgment develops.

Turn real work into a curriculum

Maintain a list of experiences engineers need, such as:

• Deploying and rolling back a change
• Investigating a production alert
• Performing a schema migration
• Responding to a security report
• Interviewing a user
• Estimating an ambiguous project
• Removing a dependency
• Writing a postmortem
• Disagreeing constructively in a design review

When such opportunities arise, assign them deliberately instead of defaulting to the most experienced person. The senior can supervise while the learner drives.

Improve your feedback quality

Good feedback distinguishes among:

• Outcome: what happened
• Behavior: what the person did
• Reasoning: why they chose it
• Principle: what general lesson transfers
• Next attempt: what they should do differently

For example:

“The migration succeeded, but you did not estimate lock duration or define an abort condition. On the next migration, write down the expected resource impact, the signal that invalidates your estimate, and the exact rollback trigger.”

That is more useful than either “looks good” or taking over the migration.

Run postmortems as teaching exercises

After an incident or difficult project, ask the learner to reconstruct:

• What they knew at each point
• What they believed
• Which signals they overweighted
• Which hypothesis they failed to consider
• What would have made the error easier to detect
• Which system change would prevent recurrence

Avoid hindsight theater. The purpose is to improve future recognition, not demonstrate that the mentor would have known better.

Learn to supervise AI-assisted work

An AI-era mentor must distinguish model output from learner capability. Useful practices include:

• Ask for a prediction before the learner queries the model.
• Ask them to explain generated code and identify its weakest assumption.
• Change a requirement and see whether they can adapt the solution.
• Ask what evidence would falsify the model’s recommendation.
• Have them debug some tasks without AI when independent understanding matters.
• Review prompts and agent traces occasionally, not just the final diff.
• Require explicit invariants and tests before implementation.
• Let them use AI freely for routine work while assessing them through novel failures.

Blanket AI bans teach an obsolete workflow. Uncritical AI use teaches dependency. The goal is disciplined delegation.

Study adjacent disciplines

Useful bodies of knowledge include:

• Cognitive apprenticeship
• Deliberate practice
• Instructional design
• Coaching and motivational interviewing
• Incident command
• Human factors and safety engineering
• Giving and receiving feedback
• Engineering management

You do not need to become an educational theorist. Learn enough to design practice, detect misconceptions, and avoid making the learner passive.

Request feedback on your mentoring

Ask mentees questions such as:

• When did I intervene too early?
• When did I leave you blocked too long?
• Which explanation changed your mental model?
• What are you still unable to do without me?
• Do you feel safe telling me that you do not understand?
• Which reviews felt like learning, and which felt like compliance?

Also ask another experienced engineer to observe one of your design reviews or pairing sessions.

Build evidence that you can develop people

If you want this to become a career advantage, document outcomes without claiming ownership of another person’s success.

Keep records of:

• Competencies a mentee gained
• Areas of ownership transferred to them
• Runbooks or curricula you created
• Reduction in review or support required
• Incidents or projects they became able to lead
• Improvements in onboarding time
• Mentees who subsequently mentored others

The strongest evidence is a mentorship multiplier: people you trained can now safely train additional people.

Make this work formally visible. Otherwise, organizations often reward the engineer who produces the most individual output while treating the engineer who develops everyone else as merely “helpful.” Ask for mentoring to be included in role expectations, project allocation, and promotion criteria.

A strong career position

A particularly robust specialty would be:

I can take a group of AI-amplified but inexperienced engineers, put them inside a safe technical and organizational environment, and turn them into people capable of independently owning important systems.

That combines technical depth, education, evaluation, organizational design, and risk management. It remains useful whether AI progress slows, continues gradually, or accelerates sharply.

In a world where producing code is cheap, producing trustworthy independent judgment in other people could become one of the highest-leverage forms of engineering.

Based on our conversation and the branching paths it's taken, come up with a series of questions to ask me that are interesting and informative. Aim for at least 20 questions. Then for each question, imagine three possible answers I might give, and what your follow-up question would be. Then score them by how interesting or informative they are and use that to pick the top 5 questions.↳ This is Gwern's "Interview Prompt"

Below is a question set designed to reveal your comparative advantage, risk tolerance, interests, and possible paths through the AI transition. The hypothetical answers are deliberately distinct so the follow-up has somewhere useful to go.

20 candidate questions

Scoring criteria

I scored each question on two dimensions:

• Interesting: likely to open a non-obvious, personally meaningful line of discussion.
• Informative: likely to constrain the space of plausible career paths and next actions.

Scores are out of 10.

The top five questions

These are not necessarily the five most philosophically interesting. They are the five most likely to identify an actionable direction.

1. What real-world domain do you have unusual access to?

This is the most important question because access is often more valuable than abstract ability. The best AI opportunities are likely to come from observing a real workflow, repeatedly, with enough trust to see its exceptions and consequences.

Question: What real-world domain do you have unusual access to—through work, family, friends, location, or personal experience?

A useful answer identifies not only an industry but a place where you can actually talk to operators, inspect artifacts, and test improvements.

2. If you could spend the next year becoming unusually good at one meta-skill, which would it be?

This forces a choice among the skills likely to compound despite model progress: discovery, evals, security, operations, teaching, and sales.

Question: If you could spend the next year becoming unusually good at one meta-skill, which would it be: discovery, evals, security, operations, teaching, or sales?

The answer can become a concrete curriculum, portfolio strategy, and project-selection rule.

3. What do you think your strongest non-AI-replicable asset is today?

This avoids the unhelpful question, “What can AI never do?” Almost everything may become partially automatable. The practical question is what you already possess that is difficult to acquire instantly through tokens.

Question: What do you think your strongest non-AI-replicable asset is today?

Examples might be hard-won operational intuition, unusual taste, trust with a community, an ability to teach, domain familiarity, persistence with difficult systems, or willingness to be accountable.

4. Do you see yourself as more interested in Workvan or an AI-native academy?

This draws out a potentially deep fork in your interests: using AI to multiply your ability to deliver systems, or using it to multiply your ability to develop people.

Question: Do you see yourself as more interested in the “Workvan” vision—shipping bounded outcomes—or the “AI-native academy” vision—creating competent people?

The hybrid answer is particularly interesting: a deliberately apprenticeship-oriented consultancy could both solve real customer problems and rebuild the junior-engineer pipeline.

5. If AI gave you the productive capacity of a ten-person team tomorrow, what would you attempt?

This is a useful anti-abstraction test. It reveals what you actually want, where you see unmet demand, and whether your instinct is toward product creation, service work, research, institutional reform, or education.

Question: If AI gave you the productive capacity of a ten-person team tomorrow, what would you attempt that is currently impossible?

The critical follow-up is not “how would you build it?” but:

Who would use it first, what evidence would show that it matters, and what would they be willing to commit in order to test it?

That turns a compelling vision into a possible next experiment.

1. What real-world domain do I have unusual access to?

One domain is the AEC (Architecture, Engineering, and Construction) industry. I used to be a project manager building custom homes and I still know many people in the industry. I'm very familiar with the way construction projects are planned and executed.

Another is the EDA (Electronic Design Automation) world. I've spent the last two years working on the technical side of the Microelectronics Commons program. I've developed a strong familiarity with the problems faced by chip designers. I have strong opinions on what a next generation EDA development environment should look like.

2. If I could spend the next year becoming unusually good at one meta-skill, which would it be?

It would have to Evals. A good eval seems to be all that's required to teach new capabilities to AI models.

3. What do I think my strongest non-AI-replicable asset is today?

It probably is taste. I do think my opinions on what makes something "good" are strong. I think I also have a knack for seeing possibilities that aren't obvious.

4. Do I see myself as more interested in Workvan or an AI-native academy?

Training the next generation of engineers is something I'm very interested in. I like people and I like teaching I'd love to work on a project allows me to develop people.

5. If AI gave me the productive capacity of a ten-person team tomorrow, what would I attempt?

This might be the hardest question to answer. Building the academy and operations simulation environment would be fun. But I would also love to work with robotics and AI in the physical world. Also augmented reality systems. I want to put on glasses and get a video game HUD but for the real world.

You have three potentially powerful but somewhat divergent threads:

1. Deep domain access: AEC and chip design / EDA.
2. A leverage thesis: evaluation is a core bottleneck for capable AI.
3. A personal motivation: develop people, and build embodied/physical-world tools.

The challenge is not finding interesting things to do. It is finding a path where these reinforce one another rather than becoming five separate hobbies. Here are the questions I would use to pressure-test and connect them.

AEC: access, pain, and willingness to pay

1. In custom-home construction, what decision repeatedly went wrong despite everyone involved being competent and well-intentioned?
   Not “communication is bad” in general; what was the recurring decision: scope changes, sequencing, estimates, inspections, subcontractor coordination, procurement, client expectation-setting, or something else?

2. Which construction artifact is most misleading?
   Schedules, budgets, drawings, RFIs, change orders, daily logs, punch lists, bids, or progress reports can all be formally present while failing to represent reality. Which one did you trust least, and why?

3. Who felt the pain most intensely, and who could actually buy a solution?
   The homeowner, GC, superintendent, architect, trade contractor, owner’s representative, or insurer may all have different incentives. A useful product needs a buyer, not merely a victim.

4. What information did you wish you had at 7 a.m. on a job site that was unavailable, stale, or politically distorted?
   This could be a very concrete starting point for an AI/AR system.

5. Would people in your AEC network allow you to observe their work closely enough to see the ugly exceptions?
   “I know people in construction” is much less valuable than “three GCs will let me sit in on project meetings, inspect their workflows, and run a six-week experiment.” Which is true for you?

6. What is the smallest job or project type on which you could test an idea without needing permission from a national construction company?
   Renovations, custom homes, small commercial projects, inspections, or a single trade may provide a better experimental environment than an enterprise-wide platform.

7. What should not be automated in construction, even if AI becomes very capable?
   This gets at your taste and ethical instincts. Is it customer trust, safety signoff, trade judgment, negotiation, design authorship, site leadership, or responsibility for a change?

EDA: a narrower but potentially much stronger technical wedge

1. When you say “next-generation EDA development environment,” what does a chip designer do differently at 10:00 a.m.?
   Describe a specific before-and-after workflow. What do they see, ask, simulate, modify, verify, and hand off?

2. What is the real bottleneck in chip design today: generating designs, understanding existing designs, verifying them, integrating IP, running tools, managing data, or coordinating humans?
   “AI for EDA” could attack any of these, but they lead to very different businesses and evaluation problems.

3. Which EDA problem has a clean enough ground truth to build a serious eval around?
   For example: equivalence checking, bug localization, RTL generation under constraints, testbench generation, timing closure, PPA optimization, constraint generation, documentation, or debugging failed flows.

4. Which important EDA task looks easy on a benchmark but fails catastrophically in real use?
   This may be your best eval opportunity. The biggest opportunity is often not demonstrating that a model can produce plausible Verilog, but detecting when plausible Verilog is subtly unacceptable.

5. What proprietary inputs, tool licenses, fabrication constraints, and verification infrastructure would a serious EDA product need?
   Is there a path that starts with open-source silicon tooling and open PDKs, or is the value necessarily locked behind incumbent ecosystems?

6. Do you want to compete with EDA incumbents, sell tooling to them, or build an entirely different interface around their tools?
   “Replace Cadence/Synopsys” is a vast ambition. “Make a particular verification or debug workflow dramatically more legible” may be a practical wedge.

7. Who would trust you enough to give you a real failed design flow, not a sanitized demo problem?
   This is the EDA equivalent of the AEC access question. Without authentic failure cases, an eval or agent can easily optimize for theater.

Your eval thesis

I agree that evals are unusually important. But I would challenge the formulation:

“A good eval seems to be all that's required to teach new capabilities to AI models.”

A good eval is often necessary. It is rarely all that is required.

An eval can enable search, reinforcement learning, selection among candidate solutions, regression detection, and human oversight. But it may also be gamed, cover only what is easily measured, encode a distorted proxy, or make a system legible only within the test environment. In high-stakes work, the central problem is frequently creating an eval that remains connected to reality.

1. What do you mean by “teach new capabilities”: train a frontier model, build an agent loop, or reliably get useful behavior from an existing model?
   The required skills, budget, and path to impact differ greatly.

2. What is an example of a capability you believe becomes tractable once an eval exists?
   Pick one from your own world: a construction planner, an EDA debugging agent, an AR site assistant, or an engineering-training simulator.

3. Who decides whether the eval measures the real objective rather than a convenient proxy?
   In AEC, a schedule that looks feasible may be unsafe or impossible in the actual subcontractor market. In EDA, a generated design may pass a narrow test suite but be unmanufacturable or unmaintainable. How would your system expose that gap?

4. What would an adversarially robust eval look like for an agent that can use tools, alter code, generate documentation, and potentially hide its own mistakes?
   Would it require hidden test cases, independent checkers, sandboxed environments, audit trails, human review, long-horizon monitoring, or all of these?

5. Would you rather become an eval specialist who supplies infrastructure to many fields, or a domain specialist who builds unusually good evals for one field?
   The first has breadth and technical generality; the second has a much stronger moat through domain judgment and access.

6. What failure would convince you that your own eval was wrong?
   This is an important personal question. The best evaluators are unusually willing to identify where their measurement regime fails.

Academy and developing engineers

1. What kind of engineer do you want to produce?
   “AI-native” can mean a prompt operator, an independent system owner, a safety-conscious operator, a researcher, a product engineer, or a domain-embedded builder. Which one matters most to you?

2. What experience did you have early in your career that changed your judgment—and how could it be safely reproduced?
   Was it owning a failure, working with a demanding mentor, dealing with a customer, resolving ambiguity, being responsible for a deadline, or shipping under real constraints?

3. What would your academy teach that a motivated engineer with Claude Code, online courses, and an open-source project cannot teach themselves?
   If the answer is mostly content, it is vulnerable. If the answer is accountability, feedback, team dynamics, authentic operations, calibrated responsibility, and real consequences, that is much more defensible.

4. Would students work on real client systems, realistic simulations, open source, internal products, or all four?
   Each has tradeoffs:

5. Client work provides real stakes but limits safe experimentation.
6. Simulations enable deliberate practice but can become artificial.
7. Open source provides authenticity but uneven mentorship.

8. Internal products give control but risk becoming make-work.

9. How would you know a graduate is genuinely independent rather than merely fluent with agents?
   What assessment would require them to form hypotheses, inspect evidence, make tradeoffs, recover from a failure, and explain their reasoning?

10. What is your willingness to be responsible for a trainee’s career outcome?
    An academy becomes much more credible if it has a strong placement model, paid apprenticeship, transparent outcome data, or even some form of tuition tied to employment. But that changes it from an educational project into a serious institution.

11. Would you be happier spending most of your week teaching and coaching, or designing a system through which other mentors can reliably teach?
    These are different callings: excellent mentor versus founder of a mentorship institution.

Robotics, AR, and the physical world

“Video-game HUD for the real world” is evocative, but it contains several different products: spatial computing, personal memory augmentation, task guidance, expert telepresence, construction-site coordination, industrial inspection, accessibility, and robotics control.

1. Whose real-world perception or action would your HUD improve first?
   A superintendent walking a site, an electrician installing a component, a chip-fab technician, a warehouse worker, a trainee engineer, a homeowner, or you personally?

2. What does the user see through the glasses that they cannot get more cheaply from a phone, tablet, paper plan, or laptop?
   AR has repeatedly struggled with this. The answer needs to be more concrete than “contextual information”: hands-free work, centimeter-scale spatial alignment, safety-critical timing, 3D inspection, remote expertise, or persistent world-anchored memory.

3. How accurate must the system be before it is useful—and how dangerous is an incorrect overlay?
   In a game, a misaligned marker is annoying. On a construction site or in an industrial environment, it can cause costly rework or injury. This is an eval question in physical form.

4. Are you more interested in the hardware/robotics problem, the spatial-interface problem, or the workflow-reinvention problem?
   You can build an AR company without designing hardware; a robotics company without building a new foundation model; or a workflow company that uses existing devices. Which layer excites you enough to endure years of unglamorous constraints?

5. Can you name one task in AEC or EDA-adjacent physical work where an expert could teach a novice through an AR overlay?
   That could unite your interests: embodied work, expert judgment, apprenticeship, evaluation, and domain access.

The forcing questions

Finally, I would make you choose among some uncomfortable tradeoffs.

1. If you had to spend the next two years in only one domain—AEC, EDA, or engineering education—which loss would you regret least? Why?

2. Would you rather found something that could become a large company but requires years of selling into conservative institutions, or a smaller profitable business that lets you work closely with trainees and users?

3. Which is more motivating to you: being right about the future, building something people use, or helping particular people become more capable?
   Ideally all three overlap. In practice, one usually wins when the work becomes difficult.

4. What are you willing to do repeatedly that other technically capable people tend to avoid?
   Customer interviews? On-site observation? Teaching beginners? Sales? Writing? Debugging ugly legacy systems? Handling incidents? This is often a better career compass than a list of interests.

5. What would a meaningful failed attempt look like by the end of the next year?
   For example: “I ran a small, paid engineering apprenticeship with five participants and learned which simulations actually predict on-the-job performance,” or “I built an EDA-agent benchmark against real workflow failures and discovered the task is not yet commercially viable.” A good failure creates durable assets and clearer judgment.

The most promising unifying hypothesis I see is not “build an academy,” “build evals,” or “build AR” separately. It is something like:

Build environments in which people and AI can safely develop real technical judgment through interaction with realistic systems and measurable consequences.

That could begin narrowly: an EDA debugging and verification simulation, an AI-assisted apprenticeship program for engineers, or an AR-guided construction workflow. The immediate question is which setting gives you the best combination of real access, sharp feedback, personal energy, and a problem small enough to test within months.