Literature · 7 min read
What Stylometry Can Infer About Literary Authorship
How authorship models work, why the Federalist case succeeded, and where textual confounds limit inference.

Authorship becomes a comparison among texts
Stylometry measures features of language in order to compare writing. In an authorship study, the usual question is not whether a passage “sounds like” a candidate to one reader. It is whether patterns in a disputed text resemble patterns in securely attributed samples closely enough, under a stated model, to support one attribution over alternatives.
That qualification contains most of the method's power and its limits. A model needs comparison texts, choices about which features count, and assumptions about possible authors. It can perform well when the candidates are known, the samples are comparable, and there is enough text. It can fail when topic, genre, editing, collaboration, translation, or historical change masquerades as an author's signature.
The evidence therefore supports a conditional title claim: stylometry can infer relative authorship under specified conditions. It does not read identity directly from prose, and a probability produced inside a closed set of candidates is not the probability that the true author is in that set.
The Federalist study showed what favorable evidence looks like
The best-known demonstration concerns The Federalist, 85 essays published in 1787 and 1788 under the shared name “Publius.” Historical records attributed most essays to Alexander Hamilton, James Madison, or John Jay, but twelve were disputed principally between Hamilton and Madison. Frederick Mosteller and David Wallace analyzed the problem in the early 1960s and in their 1964 book Inference and Disputed Authorship.
They focused on function words such as “upon,” “whilst,” and “by.” These words are useful because writers use them frequently and may attend to them less consciously than topical nouns. Mosteller and Wallace estimated how often candidate authors used selected words in known essays, then compared those distributions with the disputed papers. Several discriminators pointed strongly toward Madison.
This was not a machine finding an author without context. The historical problem supplied a bounded candidate set and many known texts by the leading candidates in a similar genre, period, and publication setting. The researchers tested different statistical approaches and could compare results. Later stylometric methods have generally agreed with the Madison attribution, while historians also use documentary evidence.
The case is important precisely because its conditions were unusually favorable. Applying the same confidence to one unsigned poem, a co-written play, or a short social-media post would not reproduce those conditions. A successful precedent validates a kind of inference, not every use of it.
Modern methods turn style into features and decisions
A contemporary pipeline begins with a corpus. Researchers identify texts believed to be by candidate authors, divide data into training and test sets, and represent each sample with measurable features. Features may include word and character n-grams, function-word frequencies, sentence lengths, punctuation, vocabulary distributions, part-of-speech sequences, or syntactic constructions.
A classifier learns boundaries among known authors from the training data. Its performance is then evaluated on held-out texts whose labels were not used to fit the model. Cross-validation can make efficient use of a limited corpus by rotating which samples are held out. Accuracy, confusion matrices, calibration, and error analysis reveal different properties; one overall percentage can hide systematic failure on a particular author or genre.
Authorship attribution and authorship verification are different tasks. Attribution selects among candidates. Verification asks whether a text is compatible with a claimed author, possibly when the actual author is absent from the reference set. The second is an open-set problem and is generally harder. A classifier forced to pick Hamilton or Madison will pick one even if John Jay wrote the sample, unless the system includes a meaningful rejection option.
Unsupervised methods pose another question: which texts cluster together without known author labels? Clusters can suggest structure, but naming a cluster requires external evidence. A group may reflect subject matter, date, publisher, narrator, or genre rather than authorship.
Large language models and embeddings add features learned from vast corpora, but they do not eliminate study design. A 2025 reanalysis of the Federalist problem found that modern embeddings can contribute to classification, while also treating the case as a statistical experiment. New tools change representation; they do not repeal the need for held-out evaluation or comparable samples.
Topic and transmission can imitate a writer
Topic is a central confound. If one candidate's training texts concern war and another's concern domestic life, a model may learn subject vocabulary. Function words and character patterns can reduce that risk, not remove it. Syntax and punctuation can also vary with genre. Letters, speeches, fiction, and legal briefs by one person may differ more than two authors working under the same editorial rules.
Text length affects stability. Frequent features can be estimated more reliably from long samples. Short texts provide fewer observations, so random variation has greater influence. Combining many short texts may help, but only if aggregation does not hide changes over time or mix collaborators.
Editing and transmission pose historical problems. A compositor may regularize spelling; an editor may alter punctuation; a surviving edition may differ from manuscript copy. Translation substitutes another person's lexical and syntactic decisions. Optical character recognition introduces errors into digitized sources, and those errors may correlate with a particular archive or typeface. A model can learn the archive instead of the author.
Collaboration challenges the assumption that one text has one generating style. A play may include scenes by different writers, and a political essay may be revised by colleagues. Rolling-window analysis can look for internal changes, but the window size itself affects results. Detecting a boundary statistically does not identify the social process that produced it.
Writers also change. Age, audience, illness, deadlines, medium, and deliberate imitation can shift measurable habits. “Style” is not a fixed biometric equivalent to DNA. It is a probability distribution over linguistic choices made in circumstances.
Validation must match the historical claim
The strongest stylometric studies test whether a method can distinguish the relevant authors under realistic conditions. If the disputed work is a novel, validation only on newspaper columns is weak. If candidates wrote across decades, a random train-test split may leak time-specific signals. Holding out entire works, periods, or genres is often more demanding and more informative.
Researchers can also perform sensitivity analysis. Does the attribution persist when spelling is normalized, when content words are removed, when features or sample lengths change, and when plausible additional candidates are included? Agreement across reasonably justified models is more persuasive than one optimized result. Disagreement is evidence about fragility, not an invitation to report only the preferred run.
Independent evidence matters. Manuscripts, correspondence, contracts, publication records, revisions, and testimony can corroborate or contradict a statistical result. Stylometry is especially useful for ranking hypotheses or revealing unexpected textual groupings. It is less secure when used as a solitary verdict in a historically open field.
Ethical stakes also change the standard. Attributing an old anonymous pamphlet may affect scholarship; identifying a living pseudonymous writer may expose a person to danger. Accuracy measured on a benchmark does not settle whether disclosure is justified. The possibility of adversarial rewriting further weakens claims that style is inescapable.
What the evidence permits
Stylometry has established that relatively unobtrusive linguistic patterns can carry authorship information. The Federalist analysis remains persuasive because documentary context, comparable known samples, multiple features, and later replication converge. Modern classifiers can exploit far larger feature sets and test performance systematically.
What is contested is how far that success travels. Scholars disagree about appropriate features and validation for sparse, collaborative, ancient, translated, or heavily edited texts. No universal accuracy rate applies across corpora. A method that distinguishes ten modern novelists cannot inherit that accuracy for Pauline epistles or Shakespearean collaboration.
A responsible conclusion names the candidate set, corpus, features, validation design, error rate, and sensitivity to alternatives. It distinguishes “most similar among these candidates” from “written by this person.” It integrates non-stylometric evidence and leaves room for an unknown author. Under those conditions, stylometry can make authorship judgments more explicit and testable. Outside them, numerical precision may conceal uncertainty rather than reduce it.
Sources and review note
2 sources, every one with a link you can follow.
- The foundational case is Mosteller and Wallace's Inference and Disputed Authorship, contextualized by the Oxford chapter “Authors and Writing”.
- Eder's replication of Delta on the disputed Federalist papers tests a later method, while Maciej Eder and colleagues' recent theory of linguistic individuality supplies current methodological context.
The review risk is turning a closed-set classifier's output into an open-world identity claim.
