A friend of mine today informed me that McCain’s answers to the 14 ScienceDebate2008 science policy questions were in [1]. When another friend informed me of the availability of Obama’s answers many weeks ago [2], I gave my personal reactions to some key issues that are of importance to me. This time around, it was suggested I apply a content analysis, similar to the one I applied to the candidates’ and VP candidates’ speeches [3] [4], and see what I got. I thought this was a neat idea, though I had to adapt my method to this kind of forum.
First, let me explain the problem in applying the “Substance Quotient” analysis. First, the substance quotient is measured against a set of issues the candidates state are important to them. Since a given speech is not necessarily arranged around a specified framework, the issues list becomes the denominator. Here, we have 14 questions with interconnecting themes that define the framework. Second, it was nearly impossible to disentangle “Wills” and “Bys” in these answers. For example, asked about what policies he would enact to ensure America remains a world leader in innovation, Obama responds,
My administration will increase funding for basic research in physical and life sciences, mathematics, and engineering at a rate that would double basic research budgets over the next decade.
This sounds like a “Will” – ” . . . will increase funding . . . ” – but then you have to wonder, what would a “by” be for this? The answer is almost tautological. The President has the power to increase funding by requesting more – it’s a power of the President. Therefore, the line between “Will” and “By” blurs.
Instead, I came up with a different approach. I consider the content of the response to each question. I avoid measuring whether the candidate actually answers the question – that’s harder to tease out of the answer. Instead, I ask the following: given the question, how many concrete and how many vague proposals does the candidate give in response? A concrete is something that is within the power of the President to do – appoint people, form panels, invest in specific areas of research, increase funding (or, at least, REQUEST an increase), request changes to the tax code, and set agency prioritizations.
A vagary is something that is . . . well . . . vague. For instance, I consider vagaries to be statements like, “I will invest in the breakthough research we need to meet our energy challenges and to transform our defense programs.” While the word “invest” appears here, the investment is directed toward a vague achievement with no attempt to be specific about how one identifies those areas of research. “Expanding access to education” without a statement of how this can be done is considered vague. “Ensuring the national space workforce is maintained and fully utilized” is also vague – there is no plan for how to actually do this. But, I do count vagaries as, at least, a response to the question that implies some unstated policy action.
Based on these assessments of the problem at hand, I define two quantities that measure the content of answers to each question. The first is the Concrete Purity, which is just the ratio of the number of concrete statements over the sum of concrete and vague statements (C/(C+V)). The second is the Policy Score, the sum of C + 0.5V. I count vagaries as only half what I count concrete statements. You could penalize vagaries more, but this seemed a reasonable start.
Why do I define two quantities? Well, a candidate could have just one concrete statement to a question, which gives a Concrete Purity of 1.0. However, their Policy Score would be really low, also 1.0. It’s therefore better to get the purity close to 1.0 and get the policy score as big as possible.
With that analysis framework in mind, let’s take a look at the questions and each candidate’s purity and score. The question is printed, and below that the Concrete Purity, followed by the Policy Score, for each candidate. I list McCain first, then Obama.
- Science and technology have been responsible for half of the growth of the American economy since WWII. But several recent reports question America’s continued leadership in these vital areas. What policies will you support to ensure that America remains the world leader in innovation?
McCain: 0.13, 8.5
Obama: 0.73, 9.5 - The Earth’s climate is changing and there is concern about the potentially adverse effects of these changes on life on the planet. What is your position on the following measures that have been proposed to address global climate change—a cap-and-trade system, a carbon tax, increased fuel-economy standards, or research? Are there other policies you would support?
McCain: 0.75, 7.0
Obama: 0.86, 6.5 - Many policymakers and scientists say energy security and sustainability are major problems facing the United States this century. What policies would you support to meet demand for energy while ensuring an economically and environmentally sustainable future?
McCain: 0.0, 1.5
Obama: 0.57, 5.5 - A comparison of 15-year-olds in 30 wealthy nations found that average science scores among U.S. students ranked 17th, while average U.S. math scores ranked 24th. What role do you think the federal government should play in preparing K-12 students for the science and technology driven 21st Century?
McCain: 0.6, 4.0
Obama: 0.44, 6.5 - Science and technology are at the core of national security like never before. What is your view of how science and technology can best be used to ensure national security and where should we put our focus?
McCain: 1.0, 1.0
Obama: 0.3, 6.5 - Some estimates suggest that if H5N1 Avian Flu becomes a pandemic it could kill more than 300 million people. In an era of constant and rapid international travel, what steps should the United States take to protect our population from global pandemics or deliberate biological attacks?
McCain: 0.0, 1.0
Obama: 0.44, 6.5 - The field of genetics has the potential to improve human health and nutrition, but many people are concerned about the effects of genetic modification both in humans and in agriculture. What is the right policy balance between the benefits of genetic advances and their potential risks?
McCain: 0.5, 1.5
Obama: 0.0, 0.5 - Stem cell research advocates say it may successfully lead to treatments for many chronic diseases and injuries, saving lives, but opponents argue that using embryos as a source for stem cells destroys human life. What is your position on government regulation and funding of stem cell research?
McCain: 1.0, 3.0
Obama: 0.33, 2.0 - Scientists estimate that some 75 percent of the world’s fisheries are in serious decline and habitats around the world like coral reefs are seriously threatened. What steps, if any, should the United States take during your presidency to protect ocean health?
McCain: 0.0, 0.5
Obama: 0.5, 4.5 - Thirty-nine states expect some level of water shortage over the next decade, and scientific studies suggest that a majority of our water resources are at risk. What policies would you support to meet demand for water resources?
McCain: 0.0, 0.5
Obama: 0.0, 2.0 - The study of Earth from space can yield important information about climate change; focus on the cosmos can advance our understanding of the universe; and manned space travel can help us inspire new generations of youth to go into science. Can we afford all of them? How would you prioritize space in your administration?
McCain: 0.5, 7.5
Obama: 0.33, 2.0 - Many government scientists report political interference in their job. Is it acceptable for elected officials to hold back or alter scientific reports if they conflict with their own views, and how will you balance scientific information with politics and personal beliefs in your decision-making?
McCain: 0.75, 3.5
Obama: 0.67, 5.0 - For many years, Congress has recognized the importance of science and engineering research to realizing our national goals. Given that the next Congress will likely face spending constraints, what priority would you give to investment in basic research in upcoming budgets?
McCain: 1.0, 2.0
Obama: 1.0, 1.0 - Americans are increasingly concerned with the cost, quality and availability of health care. How do you see science, research and technology contributing to improved health and quality of life?
McCain: 0.0, 0.5
Obama: 1.0, 2.0
There is a lot of information here. We can draw some broad conclusions from this analysis, and then drill down into some of the specific questions. Out of the 14 questions, McCain has a higher purity than Obama in 6, putting the two candidates in nearly a dead-heat of concreteness for the statements that they make for each issue. However, for the issues where McCain scored higher in Concrete Purity than Obama, he scored lower on the Policy Score half the time. That means he was concrete, but shallow in the enumeration of his policy actions.
Looking just at the Policy Scores, McCain out-scored Obama in 5/14 questions. If we ask how often McCain beat Obama in both the Concrete Purity and Policy Score, we find 3 questions. If we ask the same of Obama, we find 6 questions.
It’s hard to draw an exact conclusion from the above, although quite broadly in either the individual measures, or in the combination of measures, Obama always beats McCain in his content.
Let’s drill down to specific questions. For instance, it is interesting to look at a question relating to an issue on which the candidates see themselves as “expert”. McCain and national security are the most obvious combination. While his response suggested he understood the complexity of national security, his almost complete lack of policy proposals relating to science policy is a surprise. In fact, while he takes three paragraphs to provide exposition on the subject, the one thing he proposes is to continue to encourage the development of spinoff technologies with spending on advanced R&D. His lack of detail on this matter suggests a total lack of understanding of the critical role of technology in national security.
Obama, on the other hand, has about 30% of what he says as concrete policy implementations, with a score of 6.5 compared to McCain’s score of 1.0. McCain’s record may imply a better understanding of national security issues, but when it comes to vision for science policy in this area it’s clear that Obama has given this a lot more thought.
Related to this issue is that of biological weapons and flu pandemics. Again, you’d expect McCain to dominate this. However, he again comes in with a Policy Score of 1.0, based on two vague statements. McCain spends most of his ink setting up the context of a pandemic or attack, then suggesting how people other than himself can mitigate the damage (members of the communities affected) or outlining broad strategies that would be needed to address the outbreak. However, in all of this he barely talks about what a President can do to set policies and achieve goals. He boils it down to funding more things in research and automated detection/analysis of samples.
On the other hand, McCain blows Obama out of the water on space policy. McCain gets a purity of 50% and a score of 7.5, compared to Obama’s 33% and score of 2. I recall commenting on how odd it was that McCain included Space as an issue on his issue list [5], but it does seem a thing he takes super-seriously.
Other areas I was surprised McCain didn’t do better were water policy and health. As an Arizona Senator, water rights and water policy are so critical. How he could be so shallow on this is beyond me. As a member of America’s senior ranks, I would expect his depth of detail on health technology to be as deep as he is old. Sadly, this is also not the case.
For both of them, I was disappointed that the stem cell question didn’t lead to higher policy scores. While McCain stated his position on the issue, morally and governmentally, in absolute terms, he did not at all describe a policy path to making embryonic stem cells a thing for purely “academic debate”. Obama, who also speaks in absolutes about his support for all stem cell research, with ethical boundaries, really failed to dig into policies to promote this field of research. Each of them got a score of 3, disappointing compared to their scores on space, education, national security, and innovation.
Overall, I found this an enlightening exercise. You can find my analysis and calculations here:
and the raw data here:
The text highlighted in green is considered “concrete”, and in yellow is considered “vague”.
[1] http://www.sciencedebate2008.com/www/index.php?id=42
[2] http://steve.cooleysekula.net/blog/2008/08/31/science-policy-answers-from-obama/
[3] http://steve.cooleysekula.net/blog/2008/09/08/substance-quotients-on-mccain-and-obama/
[4] http://steve.cooleysekula.net/blog/2008/09/04/the-substance-quotient-vp-speech-analysis/
[5] http://steve.cooleysekula.net/blog/2008/08/23/presidential-candidates-an-out-in-front-analysis/
