Background

Topic modeling is a concept that is often used to describe a set of algorithms that are used to describe a large number of documents that often share a common theme (see Blei 2012). In other words. topic modeling algorithms are statistical methods that analyze the words in texts to discover the themes that run through them, how those themes are connected to each other, and how they change over time. More importantly, topic modeling as a technique makes no previous assumptions about the text and does not require any prior annotations or labeling of the documents —which is one of the reasons why we favored its use. As a consequence, the topics themselves emerge from the analysis of the original texts. Topic modeling enables us (as researchers) to organize and summarize documents at a scale that would be impossible by human annotation —or that would require large efforts to do so.

Latent Dirichlet Allocation (commonly known as LDA) is the simplest form of topic modeling (see Blei and Jordan 2003). The main idea behind LDA is that the text in documents do not belong to a single topic exclusively, but can belong to multiple topics at the same time instead. Formally, a topic can be defined as a probability distribution over a fixed vocabulary. For example, a topic about English literature has words about literary works with high probability, whereas a topic about computer science has words about engineering with a high probability.

LDA can be explained using two steps. First, we randomly choose a distribution of words over topics. And second, for each word in the document we randomly choose a topic from the distribution over topics in step #1. As part of this second step, we also randomly choose a word from the corresponding distribution over the vocabulary. This process is usually carried out in multiple iterations. Therefore, a topic model with greater iterations is usually considered to provide a better representation of the data (see “methodology” below where we explain choices specific to this analysis). Consequently, the output of an LDA model usually consists of the probability distribution of documents and terms over different topics.

In our case, we specifically chose to use LDA because of its simplicity and, more importantly, because it makes no previous assumptions over the contents of the documents. Our intention was to fully take advantage of the affordances that topic modeling provides as a machine learning technique and discover the relationships within a collection of documents, which falls right under the scope of LDA. Since we are dealing with a collection of Shakespeare’s plays these relations translate to the hidden connections between characters expressed in their dialogue.

Lisa M. Rhody (2012) has shown how LDA topic models can fail for figurative language. Indeed, despite the popularity of topic modeling and LDA models for critical inquiry, the “black box” nature of topic models and algorithms warrants attention: as Andrew Goldstone and Ted Underwood (2014) explain of their topic models, “No matter how carefully prepared the topic model may be, it is still important to remember that is a model, a statistical simplification.” These caveats hold true for the research presented here. Rather than viewing a topic model as a critical endpoint, we position the topic model as a means to a critical end: the models do not offer the final analysis, but rather, they provide a lens through which we can approach the text. Benjamin J. Schmidt (2012) compellingly argues that “humanists need to ground the analysis of topic models in the words they are built from” (see also Schmidt 2016). By using a relatively small corpus (Shakespeare’s plays) and paying specific attention to characters and the words they speak, we follow Schmidt’s directive. Indeed, this research embodies the spirit of digital humanities because it uses a digital tool as a way of sparking humanist engagement with the texts.

Turning to Shakespeare

Our analysis shows that topic modeling is able to highlight the hidden relationships between characters in different plays. More specifically, the topic modeling analysis that we conducted sheds some insights about how characters evolve across different plays. For instance, topic modeling Prince Hal’s speeches show how his character changes over the course of 1 Henry IV, 2 Henry IV, and Henry V by analyzing the words he speaks. Shakespeare famously included one of his most popular characters, Falstaff, in The Merry Wives of Windsor (a comedy) after he was a hit in the Henry IV history plays, leading centuries of scholars to decry “the Falstaff of Merry Wives” as “a big-bellied imposter” (Coleridge 1851) who is only a pale imitator. As Phyllis Rackin and Evelyn Gajowski put it, “By the beginning of the twentieth century the authenticity of Falstaff in Merry Wives had been fully discredited” (2015, 6)—he was even called “pseudo-Falstaff” by Harold Bloom (1998), as Rackin and Gajowski point out. Topic modeling Falstaff’s language shows that Falstaff of the comedy is not an imposter: rather, to the statistical model, he is indistinguishable from Falstaff of the histories. Furthermore, though our topic modeling analysis we were able to find unexpected relationships between characters: for instance, King Henry V and Petruchio (from Taming of the Shrew) draw on similar lexicon even though they appear in plays of different genres, that is, history play and comedy, respectively. Future work in this area could focus on unexpected parallels between characters: topic modeling Shakespeare’s plays by character leads to hitherto unexplored sets of literary foils, that is, perhaps unexpected pairs of characters to compare.

Some of the topics that our model extracted showed the thematic unity of individual plays. For instance, some clusters appeared focused on the characters of one play, such as ‘Angelo,’ ‘Claudio,’ ‘Duke,’ ‘prison,’ ‘brother,’ and ‘good,’ a topic primarily taken from Measure for Measure; the topic ‘Hamlet,’ ‘king,’ ‘Horatio,’ ‘Laertes,’ ‘Rosencrantz,’ ‘Guildenstern,’ ‘queen,’ ‘Polonius,’ ‘Ophelia,’ ‘ghost,’ and ‘Denmark’ clearly refers to the play Hamlet and actually offers little avenue for exploration as it only tells us that these characters often interact and belong in a group, which, of course, we already know from our dramatis personae.⁴ Others, however, showed how some topics extend beyond a particular play. One of our topics, for instance, had ‘Macbeth,’ ‘Banquo,’ ‘hail,’ ‘sir,’ ‘king,’ ‘murder,’ ‘witch,’ and ‘Caesar’ as high ranking terms, which suggests that the model found the similarities between Macbeth and Julius Caesar—not least of which, of course, is prophecy by witches (see Appendix 1 for a list of terms in these topics). The sample models in Appendix A show that the topic model algorithms were, indeed, picking up thematic similarities between plays and shared preoccupations of characters.

This paper focuses on Shakespeare’s second tetralogy, the Henriad. The Henriad is a group of four plays (Richard II, 1 & 2 Henry IV, and Henry V) primarily focused on one person: King Henry V, who appears in the final three plays. Scholars have long discussed the trajectory of Henry’s character (see, for instance, Dickinson 1961, Schell 1970, Frazer 2013): as Henry himself explains, he will pretend to be debauched and irresponsible—his ascension to the throne and reformation will be even more powerful than if he had been good all along: “My reformation, glittering over my fault,/Shall show more goodly and attract more eyes/Than that which hath no foil to set it off” (1 Henry IV 1.2.212-215).⁵ At the end of 2 Henry IV, Prince Hal becomes King Henry, and announces his new self: “Presume not that I am the thing that I was / For God doth know, so shall the world perceive, / That I have turn’d away my former self” (5.5.56-58). Even as a carousing young prince, Hal has planned his transformation into responsible king, which he puts in motion at his coronation.

Falstaff, a drinking buddy and father figure for young Prince Hal, also appears in multiple plays: in 1 and 2 Henry IV, he traces a trajectory opposite of Hal’s: he begins as a boisterous drunk, a dishonourable yet happy knight content with his beer and his women, and ends by being rejected by Hal upon his accession to the throne. Falstaff is also mentioned in Henry V, as the other characters reminisce fondly about his drinking and womanizing after his death. Falstaff was so popular that after Shakespeare killed him at the end of 2 Henry IV, he was brought back for a comedy, The Merry Wives of Windsor, set in a different reality than Shakespeare’s histories. As the old story goes, Queen Elizabeth enjoyed Falstaff’s character so much, it was she who requested he return to the stage (Crane 1997, 3).

Our topic modeling demonstrates that Prince Hal (Henry V) is a qualitatively different character in the plays in which he appears, whereas Falstaff remains the same. This might be counter-intuitive, as Prince Hal/Henry V is indeed the same historical figure that Shakespeare dramatizes in multiple plays, yet Falstaff is a fictitious character Shakespeare created to suit two different genres. Indeed, while Shakespearean scholars might be willing to accept a reformed Henry V who is different from his younger self, stating that Falstaff of Merry Wives is the same Falstaff from the history plays is a more contentious claim.

From Prince Hal to Henry V

Henry V appeared prominently in three topics that our model created. For instance, in a topic that included words such as king, lord, duke, and God as the most probable words, which we’ll call the “kingly topic,” Henry in the play Henry V had a 34% chance of belonging (see Appendix A for a list of words in the “kingly topic”). A few other characters from this play had much higher probabilities of being in this topic—for instance, the Duke of Bedford or the Duke of Gloucester. This, however, seems to be expected: these characters speak far fewer lines than the hero of the play: in this case, six and five lines respectively. If you speak only fifty words or so, then you are more likely to be on point or speaking about the same subject. For characters who speak more lines, such as Henry, they are more likely to discuss varying ideas. In Henry V, the title character speaks over a thousand lines: an amount of speech exceeded only by Iago and Hamlet in all of Shakespeare’s plays.⁶ In those over-a-thousand lines, Henry is likely to be talking about kings, gods, and lords in Henry V. However, in the prequels, the Henry IV plays, his younger self, Prince Hal, is nowhere near as likely to talk about such noble and kingly terms. For this topic, Prince Hal when he first appears (in 1 Henry IV) is only 5% as likely to discuss these terms, whereas by 2 Henry IV, his likelihood of discussing these ideas increases marginally to 8%, which might be accounted for by his reformation at the end of the play. Upon hearing that he has inherited the throne, Henry begins to use more formal diction immediately: “The tide of blood in me / Hath proudly flowed in vanity till now; / Now it doth turn and ebb back to the sea / Where it shall mingle with the state of floods, / And flow henceforth in formal majesty” (2 Henry IV 5.2.129-33). Henry at once calls the parliament to session so he can be crowned and begin the business of governing.

When Henry becomes king, he starts talking more like his father, King Henry IV. When King Henry IV is on the throne, he uses terms in the “kingly topic” at almost the same rate as his son does when he is crowned. In 1 Henry IV, the titular king has a 34% probability of belonging to this topic, just like his son, King Henry V, in his title play. In 2 Henry IV, however, Henry IV’s probability of belonging to the “kingly” topic drops, although not precipitously: he is only 21% likely to belong. In 2 Henry IV, he’s more concerned with the sin of how he took the crown from the previous king, Richard II. As King Henry IV says to his son, “God knows, my son, / What by-paths and indirect crooked ways / I met this crown, and I myself know well / How troublesome it sat upon my head” (2 Henry IV 4.5.183-86). The change in this topic model could reflect how before his death, King Henry IV begins to turn away from the business of actually ruling, and instead thinks of past regrets.

We are not arguing that having a high probability of being associated with the “kingly topic” makes a character more likely to be a ruler. Often, however, kings are around the 30–40% mark of probability. For instance, in Richard II, King Richard II’s speeches have a 38% chance of belonging to this topic. In plays beyond the Henriad, Richard III and Henry VI fall into this range. Aspirants to the throne do at times, too: Jack Cade, leader of Cade rebellion, exhibits similar traits. Nobles can also speak in this register: Queen Margaret in the Henry VI plays is an example of this. Having a higher percentage in this category, however, does not indicate that a character is necessarily more kingly—as we discussed, characters with only a few lines can rank disproportionately high in any given topic. That said, these data clearly show that Prince Hal is not interested in the language of rulership before he comes to the throne, which is exactly what he tells the audience.

Prince Hal tells his audience that we should consider his youth a “foil” to his mature age (1 Henry IV 1.2.215). Shakespeare’s dramaturgy puts Hal in natural comparison to a number of other characters: to his father, the previous king; to his rival, Hotspur (who was also named Henry); to his brother, Prince John; we can even compare him to his son, Henry VI, the star of Shakespeare’s first tetralogy. Shakespeare wrote the Henry VI group of plays first—they were big hits that prompted him to write more history plays. Then, like George Lucas did with Star Wars, he went back to write the prequels, the Henriad (also called the second tetralogy). At the end of Henry V, Shakespeare invites the audience to think of Henry V in relation to his son—a story early modern audiences would have already known from Shakespeare’s earlier plays. As the Epilogue relates, King Henry V was “This star of England. Fortune made his sword; / By which the world’s best garden he achieved, / And of it left his son imperial lord. / Henry the Sixth, in infant bands crowned king / Of France and England, did this king succeed; / Whose state so many had the managing, / That they lost France, and made his England bleed” (Henry V Epilogue.6-12). Henry V won England great glory (and part of France); his son lost it all.

Topic modeling prompts us to find foils for Henry beyond those set up by the structure of Shakespeare’s plays. In our model of Henry V, the character who speaks most like Henry on the “kingly topic” is not actually a historical character at all: it is the chorus. In Henry V, the chorus serves to introduce the play, to tell the audience about offstage events, and to interpret the events presented in a way that manipulates the audience response. What this topic model suggests is that the chorus is equally invested in kingship and how it functions as Henry V is when he becomes king.

Up to this point, we’ve been focusing on the words most likely to be in a given topic: for instance, in the “kingly topic,” the terms ‘king’ and ‘lord’ have two percent probability of appearing in this topic. Although this might seem low, in our model, that suggests that those are among the most popular words in this topic. Both ‘king’ and ‘lord’ can be used as a form of address, which means that they might be among the most frequent words in the history plays. However, this model is based not on the frequency of terms but on the probability of a given word appearing. And while some of the other words in this topic are indeed terms of address, such as queen and majesty, this topic is not entirely based on titles (for instance, ‘friar,’ which appears prominently in other topics, does not appear in this one). Some of the other words in this topic spell out the importance of kingship: ‘war,’ ‘sword,’ and ‘soldier’ suggest the martial aspects of this topic, whereas ‘good,’ ‘noble,’ and ‘peace’ suggest the qualities that a good king should hold. The appearance of terms such as ‘crown,’ ‘sovereign,’ and ‘royal’ reinforce our calling this the “kingly topic.” ‘England’ and ‘English’ appear in this topic—unsurprising for plays about the Kings of England—but ‘France’ appears too, which highlights the importance of England’s greatest rival and the English holdings on French soil.

Naturally, a number of the kings’ names appear in the “kingly topic,” including Henry, Gloucester, and Richard. What we might not expect is for ‘Edward’ to appear on this list. The name “Edward” appears in only two topics. The “kingly topic” (discussed here) and another topic derived primarily, if not entirely, from stage directions (see Appendix A). Although King Edward IV and his grandson prince Edward both appear briefly in Richard III and could be responsible for the appearance of this word in this topic, it is equally likely that the king of a century earlier, Edward III, is responsible for evoking this name. Edward III was the last king before the War of Roses: before the era in which eight of Shakespeare’s history plays take place.⁷ All of the kings in the Henriad proudly trace their ancestry back to Edward: they can claim kingship only because they share “Edward’s sacred blood” (Richard II 1.2.17). Henry V’s legal claim to the throne is based on this heritage: when the ambassador to France tries to flatter him, he calls Edward Henry’s “great predecessor, King Edward the Third” (Henry V 1.2.248). What this topic suggests is that kingship in Shakespeare’s War of the Roses plays is as much about tracing inheritance to a now-legendary past king as it is being a noble warrior. Using a topic model based on probability instead of frequency allows us to find areas of comparison that might otherwise be overlooked, such as the importance of Edward III (a king who never appears onstage) to Shakespeare’s history plays. Of course, the importance of Edward III to the Henriad has already been explored in scholarship, to the point where the Riverside Shakespeare gives the title “Edward’s Sacred Blood” to its family tree (Evans 1997, 630–631).

We are not, of course, the first to discuss kingship in Henry V, nor are we the first to suggest that Prince Hal’s character changes. Our methods, however, have led us to make new connections and draw new conclusions: these topic models quantify Hal’s reformation, lead us to draw comparisons between Henry V and the Chorus of that play, and induce us to consider the importance of characters such as Edward III who do not even appear in the Henriad.

Is Falstaff Falstaff?

In 1 & 2 Henry IV, Falstaff is a boisterous, debauched, drunk. Hal must reject Falstaff in order to achieve his reformation and become king: the climactic moment of 2 Henry IV is Henry V’s cold proclamation to Falstaff, “I know thee not, old man” (5.5.47). With his speech during his coronation procession, Henry V rejects his ill-spent youth and repudiates Falstaff as the personification of his earlier days.

Although he dies offstage between 2 Henry IV and Henry V, Falstaff’s reputation extends beyond his death. Fluellen, a soldier, recounts Falstaff’s rejection by Prince Hal: “Harry Monmouth [Prince Hal], being in his right wits and his good judgments, turned away the fat knight with the great belly-doublet: he was full of jests, and gipes, and knaveries, and mocks; I have forgot his name” (Henry V 4.7.46-50). His interlocutor, Gower, supplies the missing name and names the ghost that metaphorically haunts Henry V: “Sir John Falstaff” (4.7.51). Falstaff is a larger-than-life character whose presence looms beyond those plays in which he actually appears.

Although Falstaff is a fictional character, not taken from Shakespeare’s history book sources, he is based on historical figures and literary stock characters. Falstaff combines the classical stock figure of the Miles Gloriosus (boastful yet cowardly knight) with the stock character of the glutton and the fool. Shakespeare takes great pains to disavow that Falstaff is connected to Sir John Oldcastle, a historical friend of Prince Hal’s: as the epilogue to 2 Henry IV states, “Falstaff shall die of a sweat, unless already a’ be killed with your hard opinions; for Oldcastle died a martyr, and this is not the man” (Epilogue.30-32). Shakespeare’s claim that Falstaff is not Oldcastle, of course, only serves to draw the comparison for the audience (see, for instance, Taylor 1985 and Melchiori 1994).

As an amalgam of historical and literary sources who Shakespeare invented to suit his dramatic needs, Falstaff’s character might be expected to be adaptable depending on the contexts in which he appears. Our topic model, however, suggests that Falstaff is Falstaff, whether he be in the history plays in which he originally appeared, or in the comedy spin-off, Merry Wives of Windsor, in which he starred. Merry Wives occurs in a timeline that seems irreconcilable to the Henriad: scholars debate if Merry Wives is a prequel or sequel (see, for instance, Melchiori 1994). Most recently, Kay Stanton (2015) has argued that Merry Wives takes place in a feminist parallel universe. Our topic model suggests that there is something ultimately Falstaffian that an algorithm might capture: Falstaff is, indeed, Falstaff: across plays, across places, and across genres.

Falstaff was primarily written by the same author, Shakespeare. Of course, despite Shakespeare’s predominant hand in these plays, every early modern play is, to a degree, an act of collaborative authorship. Will Kempe, the actor who played Falstaff, would have contributed to each performance and improvised certain elements, which may or not be captured in the printed version. Plays could be revised by other playwrights working for the companies or altered, to varying degrees, by printers or compositors. Falstaff, then, can be considered a collaborative creation, and we cannot know how much the printed version reflects this. Topic modeling, however, allows us to see how much the published (and later, encoded) version of Falstaff in each play is the same or different.

Of the 49 topics that our model generated, only two of our topics featured Falstaff from all three plays: the first was a list that favored general terms of address (“Sir,” “master,” ‘come,’ ‘good,’ ‘well,’ ‘man,’ ‘mistress’…); the second was a grab-bag of words (‘love,’ ‘thou,’ ‘shall,’ ‘man,’ ‘come,’ ‘would’) that didn’t reflect a particularly theme or play. Turning to “topics” that seem, at first, not to be topic-based can be fruitful: the field of stylometrics has shown the value of looking at often-overlooked words to determine authorship (Juola 2013).⁸ The general nature of these words doesn’t mean the results for this topic should be discarded: indeed, these non-thematic topics suggest that there the statistical model sees a similarity between Falstaff that goes beyond his love of women and sack—that is to say, although a humanist close reader might not associate these words with each other or Falstaff, this algorithm does.

The fact that Falstaff from all three plays appears in only two of our forty-nine topics could seem like a relatively low occurrence: however, Falstaff (in all plays) occurs only in four topics created by our model. Table 1 demonstrates that Falstaff from 1 Henry IV is 75% likely to share a topic with his later self in 2 Henry IV (unlike Prince Hal, who does not show the same similarity with his later self). But perhaps most surprisingly, Falstaff from The Merry Wives of Windsor is just as likely to share a topic from the same character from the history play. There is no topic where Falstaff from one play appears alone without Falstaff from another. Even though The Merry Wives of Windsor takes place in a different universe from the history plays, he is basically the same Falstaff. The Falstaff of the history plays is a fictionalized character living in a true historical setting, whereas Falstaff of Merry Wives is part of a realistic and not historicized English town. Despite the genre-based differences between these plays, Falstaff is Falstaff.

Conclusions

Topic modeling, when paired with thoughtfully encoded texts, can lead us to ask questions that we might not otherwise have asked. However, without an understanding of these texts, topic modeling can reveal precious little. Even productive topic models, such as this one that led us to reconsider the nature of Prince Hal and Falstaff, will not always offer fruitful comparisons. For instance, one of the topics that this model generated had “shall, would, make, well, sir, us, and yet” as some of the most probable words. While we did not find this an interesting avenue for further research, though, perhaps, a linguist working on modals would. The topic model that we created did not provide the questions to us: it gave us comparisons, themes, and hints that led us to our research questions. To put it another way: this topic model provides a distant reading (following Jockers 2013 and Moretti 2013) of Shakespeare’s plays, which can be supported by more traditional literary analysis and textual evidence and can lead to new research questions.

Ultimately, our research demonstrates the fruitfulness not just of topic modeling, but of topic modeling a dramatic corpus by character. The first major conclusion of this research, that Prince Hal is not the same as Henry V, is one that the character himself tells us: yet our model shows that Hal is categorically and quantifiably different (not just claiming to be so). Our second major conclusion—that Falstaff’s character does not change over multiple plays—adds a new form of evidence to be considered in a centuries-long debate in Shakespeare studies. Indeed, it is our hope that this algorithmically-informed analysis of Shakespeare’s plays is not taken as a final answer, but, instead, as a prompt. As this research demonstrates, topic modeling plays with attention to each speaker opens the door for new comparisons that expand on and reassess previous critical interpretations.

Competing Interests

The authors have no competing interests to declare.