Presidential Candidates and Verbal Honesty

As a physicist, I am fascinated by trying to quantify the world – to find the numbers that can represent what is going on in nature. People are hard to quantify most of the time, but trying to do so can be informative. Organizations like PolitiFact [1] offer a set of data about people – specifically, about politicians – that can allow us to create numbers to represent and understand those people. In this post, I discuss my own recent attempt to quantify the data collected by PolitiFact by constructing an “Verbal Honesty Score” for the remaining Democrat and Republican Presidential Candidates. I find that Donald Trump is, by far, the most verbally dishonest candidate… by a long shot, and by two measures of the same data. I find that Hillary Clinton, Bernie Sanders, and John Kasich are the most verbally honest of the group, though who is more verbally honest differs by which of the two measures you want to use. For now, I assign no errors to the numbers though errors of a statistical nature are possible. I comment on errors of a systematic nature.

Introduction

The organization, PolitiFact.com, assesses spoken and written statements (claims) made by the current Presidential Candidates from both major parties. They rank the claims into one of 6 categories: True, Mostly True, Half-True, Mostly False, False, and “Pants on Fire.” This categorization can be considered subjective near the boundaries of categories, and so while it should be possible to assign a systematic error based on mis-categorization rates I do not attempt to do that because I have no access to the internal decision-making process for these rankings. Since there is a count of statements that fall in each category, it should be possible to assign statistical uncertainties to anything done with these rankings, but I do not pursue that further in this particular post. I will comment later on statistical uncertainties.

The “Verbal Honesty Score”

I made the following definition, then, of a “Verbal Honesty Score” that assigns point values to each category:

• 5 points for “True”
• 4 points for “Mostly True”
• 3 points for “Half-True”
• 2 points for “Mostly False”
• 1 point for “False”
• -1 point for “Pants on Fire”

“Pants on Fire” is such an egregious category – the “big lie,” akin to dangerous nonsense or propaganda – that I felt it deserved to DETRACT from a candidate’s Verbal Honesty Score. You could easily change this to a 1-6 scale and recompute, but the results you will find will not significantly change the conclusions drawn from the definition I have chosen.

I then construct a weighted mean of statements using this:

where is the score assigned to category and is the number of statements that fall into category . The quantity, , is then what I henceforth refer to as the “Verbal Honesty Score”. If a candidate is ALWAYS verbally true, then they get a perfect score of 5.00. Of course, there is no perfectly honest person, so all we can do is rank candidates against one another. We expect that even the most relatively honest candidates will not achieve a 5.00, or even a score between 4.00 – 5.00, as candidates inevitably stretch the truth or spin numbers to try to achieve a viewpoint that suits their message.

The results are tabulated and graphed in the figure above. I find the following scores for these candidates, based on statements assessed as of March 12, 2016:

• Clinton: 3.30
• Cruz: 2.10
• Kasich: 3.17
• Rubio: 2.84
• Sanders: 3.18
• Trump: 1.33

ranked alphabetically. Ranked instead by the Verbal Honesty Score:

• Clinton: 3.30
• Sanders: 3.18
• Kasich: 3.17
• Rubio: 2.84
• Cruz: 2.18
• Trump: 1.33

Trump is dragged far toward the bottom by the sheer number of “Pants on Fire” statements he has made. Clinton has made 2 of these and, amazingly, Sanders has made NONE of these so far.

The “Pants on Fire” Percentage

I also decided to look at these candidates based on the percentage of all assessed statements that were rated “Pants on Fire” as an alternative way of using the data. Ranking the candidates from smallest to largest “Pants on Fire” percentage, I find:

• Sanders: 0%
• Clinton: 1%
• Rubio: 3%
• Kasich: 5%
• Cruz: 7%
• Trump: 19%

At the top of the pile, things have reshuffled. Ranked in this way, Sanders and Clinton change places. Rubio and Kasich also change places, with Rubio proving slightly less “Pants on Fire” than Kasich. Cruz is second-to-highest, and comfortably in the highest place – with more than twice the percentage of statements that are classed as “Pants on Fire” than the next-closest candidate – is Trump.

Conclusions

The broad conclusions from this quantitative assessment of candidate statement categorization are as follows. Clinton and Sanders are, across both major parties, the most verbally honest of the group. Among Republicans, Rubio and Kasich are comfortably at the top of the rankings, though neither of them ranks higher than their Democratic counterparts. At the bottom of the rankings, regardless of metric used, are Cruz and Trump. Candidate Donald Trump consistently ranks as the least verbally honest candidate in the race. To put things in perspective, based on the claims he has made that have also been assessed by PolitiFact, it suggests that every claim from Trump that is requested or selected for assessment by PolitiFact has a 1-in-5 chance of being a complete and utter fabrication. You could roll a 6-sided die and have a better chance of rolling a 1-5 than you do of getting a non-“Pants on Fire” statement from Candidate Trump. On the other end of the spectrum, Clinton and Sanders have proven to be the most verbally honest candidates in either party – only about 1-in-100 of their statements are going to be “Pants on Fire”. Based on the two measurements used here, either Clinton or Sanders could be considered “most honest”. Given the small differences between their scores, it is really a small difference that separates them in this category.

A final comment can be made on statistical uncertainty. Propagating the statistical uncertainty into either of the two scores calculated here suggests that the maximum statistical uncertainty on these numbers is at the level of 10% (on Trump’s numbers). This is not enough to put a candidate like Trump in a statistically favorable position compared to other candidates with nearby scores. He consistently ranks many standard deviations lower than his closest peer in the ranking system. The statistical uncertainties on these measures will, however, essentially put Clinton and Sanders, or Kasich and Rubio, in statistical dead heats. While Clinton/Sanders are still far more verbally honest than Kasich/Rubio, within their parties they are essentially indistinguishable in their verbal honesty scores.

[1] http://www.politifact.com/