Branch

Personal Quality/Taste Score

Human Duke

Everyone agrees that LLMs are pretty terrible at Essay writing. They use a repetitive and easy to spot style. They don't usually say anything interesting either.

We've seen that agent harnesses like Claude Code and Codex have given software engineers super powers, but there's nothing like this yet for essay writers (or writers in general).

What sort of tooling would give writers super powers?

How does it look different from the work software engineers do?

How can writers retain their unique style and voice? Or how can the tooling help them develop one?

AI · author-edited OpenAI

The equivalent of Claude Code for writers is probably not “a model that writes the essay.” It is a research, argument, and revision environment in which the writer remains the source of judgment.

Coding agents became useful when they gained a harness: access to the repository, search, tests, version control, execution, and a loop for checking their work. A writing agent needs analogous machinery—but prose has no compiler, and “good” is not a single measurable state.

What a writer’s harness would contain

1. A living model of the project

Not just a chat transcript or a folder of documents. The system should maintain:

  • the intended reader
  • the central question
  • candidate theses
  • claims and supporting evidence
  • objections and unresolved tensions
  • sources, quotations, and provenance
  • definitions of important terms
  • examples, scenes, and anecdotes
  • structural outlines
  • stylistic constraints
  • decisions the writer has made and rejected

This would be closer to a graph than a document. An essay is merely one rendering of that graph. WRITER.md

2. An argument debugger

This may be the closest analogue to tests and static analysis.

It could identify:

  • hidden assumptions
  • equivocation between two meanings of a term
  • causal claims supported only by correlation
  • conclusions stronger than the evidence permits
  • unaddressed obvious objections
  • false dichotomies
  • sections that are individually plausible but do not advance the thesis
  • abstractions that need examples
  • examples that are vivid but unrepresentative

It should also be able to “run” the argument against several readers: an expert, a hostile critic, an intelligent newcomer, or someone directly affected by the topic.

This is not an automatic truth machine. It is a way to generate pressure against the writer’s reasoning. Logical Fallacy Linter

3. Structural editing

Most writing assistance operates one sentence at a time, which is often the wrong level.

A useful system would let the writer manipulate the essay at several scales:

  • argument: claim, evidence, objection, reply
  • section: purpose, tension, payoff
  • paragraph: assertion, development, transition
  • sentence: rhythm, syntax, diction
  • word: precision and connotation

It could display a “reverse outline” generated from the actual draft, then compare that with the intended outline. It might label each paragraph by function and expose six consecutive paragraphs all doing background exposition.

It should support operations like:

  • “Show me three structures for this material without drafting them.”
  • “Move the central complication earlier.”
  • “What can be removed if this must lose 30% of its length?”
  • “Find the point at which the essay stops asking a question and starts pretending it has answered it.”
  • “Preserve the conclusion, but propose a route to it that does not depend on chronological organization.” Structural Editing UX

4. Revision instruments, not a rewrite button

A generic “improve this” button almost inevitably pulls prose toward the model’s average style.

Better tools would expose specific transformations:

  • compress repetition
  • replace abstract claims with concrete instances
  • vary sentence architecture
  • make uncertainty explicit
  • separate observation from inference
  • remove throat-clearing
  • identify clichés and inherited metaphors
  • locate unexplained jumps
  • estimate where reader attention is likely to drop
  • compare two revisions and describe what each gains and loses

The key is that suggestions should arrive as diagnoses, alternatives, or patches, not as a silently regenerated document.

How writing differs from software engineering

There are important analogies:

Software Writing
Repository Notes, sources, drafts, archive
Dependency graph Claim/evidence graph
Compiler errors Grammar and consistency failures
Unit tests Factual and citation checks
Integration tests Argument coherence
Runtime profiling Reader-response and pacing analysis
Version control Draft history and decision log
Code review Editorial and adversarial review

But the differences matter more.

Writing has no complete specification

In software, the desired behavior can often be stated externally. In an essay, discovering what the writer really thinks is part of the work. The specification changes because composition changes the author. Modeling the Author

A writing agent therefore cannot simply optimize against the initial brief. It must help the writer notice that the original brief was shallow or mistaken.

Quality is plural and contested

A paragraph can become clearer but less suggestive, shorter but less musical, more rigorous but less humane. These are not bugs with unambiguous fixes.

The tool should surface trade-offs rather than collapse them into a scalar “writing quality” score. Personal Quality/Taste Score You are here

This branch begins here Personal Quality/Taste Score
Human Duke

What if authors could store embeddings of writing's that they like and dislike. Then we could compare their writing with those embeddings. This is not a well-formed idea and we don't want the writer to just parrot their own favorite authors/writers.

AI OpenAI

There is a strong idea here, but “make my draft closer to the liked embeddings” is probably the wrong objective. That would produce imitation, and ordinary embeddings often capture topic more strongly than style anyway.

The more useful goal is:

Use admired and disliked writing to infer the writer’s editorial taste, then use that taste to diagnose drafts—not generate prose toward a centroid.

A contrastive taste model

The corpus should contain more than binary likes and dislikes. Writers could highlight passages and annotate why they react to them:

  • “Concrete without over-explaining”
  • “Earns this aphorism through the preceding example”
  • “Too polished; sounds airless”
  • “False profundity”
  • “Good alternation of long and short sentences”
  • “Abrupt transition, but productively so”
  • “Too many abstract nouns”
  • “I admire this but do not want to sound like it”
  • “This is already a tendency of mine; don’t reinforce it”

That last distinction is crucial. Taste, aspiration, and identity are not the same thing. I can admire Borges without wanting my product review rewritten as Borges.

Rather than one “style embedding,” the system could build several partial models:

  • sentence rhythm
  • degree of abstraction
  • explicitness versus implication
  • use of examples
  • rhetorical temperature
  • humor and irony
  • structural density
  • tolerance for digression
  • strength and frequency of claims
  • relationship to the reader
  • patterns the writer finds mannered or clichéd

The writer could activate different profiles by project: “reported feature,” “personal essay,” “technical explanation,” or “correspondence.”

Comparison should produce questions, not grades

A system should not say:

Your draft is 63% similar to writers you like.

That number would be nearly meaningless and dangerously seductive.

It could instead say:

You tend to favor passages that introduce abstractions through concrete cases. In this draft, the first concrete case appears after 900 words.

Or:

In passages you dislike, emphatic one-sentence paragraphs often state conclusions already implied by the previous paragraph. This draft contains five such cases.

Or:

Your reference corpus usually alternates exposition with complication. Sections 2–4 only accumulate evidence, so the argument may feel predetermined.

These are inspectable claims. The system should show the passages that led it to infer a preference, and let the writer correct the inference.

Preventing parroting

Several design choices would help.

1. Separate content from stylistic features

Before comparing passages, control for subject matter. Otherwise, a writer who saves many essays about cities may be told that an essay sounds “right” merely because it discusses streets and buildings.

Comparisons should operate on explicit features, syntactic representations, rhetorical roles, and structural patterns—not only generic semantic embeddings.

2. Use references for discrimination, not continuation

The system can learn to answer “Which of these two paragraphs would this writer probably prefer, and why?” without being trained to complete prose in the manner of a named author.

That makes it an editorial model rather than an imitation model.

3. Retrieve counterexamples

For every suggestion, show both:

  • an admired passage exhibiting the relevant property
  • an admired passage succeeding by violating it

This prevents preferences from hardening into rules. Good writers often have taste for multiple, incompatible effects.

4. Protect productive deviation

The system should identify when the draft differs from the taste profile without assuming that difference is bad:

This ending is more explicit than endings you usually admire. Is that deliberate?

A distinctive voice often emerges from the gap between what someone admires and what they naturally produce. Automatically closing that gap could erase the interesting part.

5. Detect mimicry directly

It could flag unusually close phrase-level, syntactic, or rhetorical resemblance to a source. “You appear to have absorbed this author’s cadence” is useful feedback, especially if the exact parallel is shown.

The writer’s own archive matters more

A corpus of favorite authors models taste. A writer’s drafts, revisions, journals, emails, published work, and rejected sentences model actual voice.

Version history is especially valuable. Every accepted and rejected edit is a preference signal:

  • The model suggested a cleaner sentence; the writer restored the awkward one.
  • The editor asked for a clearer thesis; the writer made it more qualified.
  • The writer repeatedly removed summary sentences.
  • The writer preserved certain fragments and irregular rhythms.

Those decisions reveal voice more accurately than finished prose alone. The harness could learn:

You routinely reject edits that replace physical verbs with abstract summaries.

That is both specific and grounded in the writer’s behavior.

A possible workflow

  1. The writer imports a private reference library.
  2. They highlight and annotate a small number of passages; the system proposes additional inferred dimensions.
  3. The system presents pairs—“Which do you prefer, and in what context?”—to refine the profile.
  4. It separately analyzes the writer’s own revision history.
  5. On a draft, it produces a taste-aware editorial report:
  6. where the draft aligns with recurring preferences
  7. where it deviates
  8. where it resembles a source too closely
  9. where the writer’s habitual weaknesses are being mistaken for voice
  10. which trade-offs each proposed revision would make
  11. The writer requests local alternatives, but the tool does not automatically normalize the whole document.

The deepest opportunity is not “write like the library.” It is make tacit taste explicit. Skilled editors do this: they gradually learn what a writer is trying to become, then point out where the draft betrays or exceeds that intention. Embeddings could provide part of the memory and pattern recognition, but annotations, contrastive judgments, revision history, and interpretable features would make the system genuinely useful.

Explore conversation