Text Analysis Software

This page provides guidance on the use of the use of open-source proprietary text analysis software and Author Attribution Methodology (AAM) designed to determine authorship of text. The software was used in the Thomas Paine Collected Writings, managed by managed by the Thomas Paine Historical Association and published in 2026 by Princeton University Press.

An Overview

Using computer methods of analyzing text to determine authorship is not a matter of opinion. Calculations are objective, there is no room to introduce prejudices. 

This process of analysis began in the early 1960s with Mosteller and Wallace, using function word use (“and”, “but”, “as”, etc.), they achieved 50% accuracy. The “authorships” of the Federalist Papers that you read on the Internet were based on them, and thus only half correct.

Java Graphic Author Attribution (JGAAP) is a free source of programmed style features that can be used to test texts using style(s) features. The Text Analysis Project at the Institute for Thomas Paine Studies at Iona University developed this computer-based author attribution procedure to help in the problem of identifying authorship of texts attributed to Paine, or texts previously not attributed to Paine that should be. By deploying the new features (17 of them now, compared to the one used by Mosteller and Wallace) ITPS was able to achieve 90% accuracy.

Stylistic features identified in JGAAP are often used unconsciously and consistently by authors, and, if correctly identified, will correctly reveal identity of the author. It is much like fingerprints, which are hardwired in the author’s head, who can only explain, argue, or express themselves with their unique brain patterns. We use machine learning where special algorithms use documents of known authorship as training examples to train the computer to recognize each author’s writing style, or syntax, based on the use of 17 accepted author attribution features. Once the computer is trained, the completed model is tested against the disputed document to assess the nearest fit to the author training sets.

The number of authors used was twenty-nine here, including Paine, Franklin, Benezet, Jefferson, Hopkinson, Price, Priestley, John and Sam Adams, Hopkins, Witherspoon, Madison, Monroe, Young, Cassandra, Matlack, G. Morris, Peale, Rittenhouse, and others. Authors are selected by time period and physical availability. A “leave-one-out” method of testing each author’s file is done: one of the works in an author’s file of works is tested against the remaining works, and so on through all the works.

For example, when using this method of testing the integrity of Paine’s file, we noticed that “African Slavery in America” stood out with a very low percentage. By removing it, the remaining works tested at 100%, assuring that Paine’s file was accurate. The same tests were done on all the author files to be certain of their purity. In all we use over 100 author files that we have covered French, English, and American authors in testing. Non-English texts are Google-translated, which has proved to be extremely accurate so that all tests are done in English.

The focus is on the style of the documents in the computer analysis, not the content, but then the results must be confirmed through thorough analysis, for example, by Paine scholars familiar with the philosophy and approach to political issues of Paine, and the forensic clues to geographical, personal content, and internal evidence in the document.

Some of the features used: 

  • Character NGrams (Cg) – uses a sequence of n (2 or 3) characters to compare. For example, “Character NGrams” has these 2-grams: Ch, ha,ar,rc,ct,te,er,r_, _N, NG, Gr,ra,am,ms. This has proven very reliable in text mining applications. 
  • First Word In Sentence (Fwis) – compiles the first words used in all sentences and compares. 
  • MW Function Words (Mwfw) – from Mosteller-Wallace “Federalist Papers” work. Function words are the most common words, like prepositions, pronouns, articles, etc. They are content neutral and are used in a subconscious manner, and are most reliable in author attribution works. 
  • Prepositions (Prep) 
  • Special Words 2 (Sw2) – developed by our Text Analysis Project, these include use of period words like “hath”, “juster”, “wilfull”. 
  • Suffices (Suf) – looks at the last three letters of each word. 
  • SW French Origin (Swfo) – words are compared to an English words of French origin compilation. 
  • Vowel-initial Words (Viw) 
  • Word NGrams (Wg) – uses a sequence of words for comparison 

Learning methods used and abbreviations 

  • Centroid Driver, Cosine Distance (CdCosd) – nearest-neighbor approach using normalized product distance 
  • Centroid Driver, Histogram Distance (CdHisd) – nearest-neighbor approach using Euclidian distance 
  • Linear SVM (Lsvm) – generates a linear separator to divide the feature space into regions, each corresponding to a specific author 
  • Multilayer Perceptron (Mp) – an artificial neural network that maps sets of input data onto appropriate outputs. 

Anyone can do this, so don’t be intimidated. But there are a lot of basics you should know:

  • If it’s an historical text, you would need some historicity background.
  • If you are interested in an historical figure, you’ll need other authors in that period, especially if your author is known to others.
  • If you have an unknown author of a work, you need all authors in that time, place, ideology, etc.
  • Broad Author Files are crucial and should contain 2000 to 3000 words from definitive works of each author.

Video Overviews: Basic Intro to the Text Analysis Software

Watch these six videos in order for an overview:

1. Creating Author Files

2. Generating Author Packages

3. Testing a Document

4. Analyzing Graphs

5. Conclusion

Scroll to Top