Beliefs Deserve Better Science

When religious beliefs are placed into the sphere of scientific test, scientific criticism of those beliefs is fair game. All questions put under the framework of the scientific method enjoy the same scrutiny, and it is imperative to approach the question with the same critical toolkit as one would approach any other question. When my issue of The Economist showed up on my Kindle on Friday, I was at first intrigued to see an article suggesting that prayer makes couples more faithful to one another (as opposed, for instance, to just thinking positive thoughts). As I realized that the reporting on this study was done with some carelessness, however, my interest turned to anger at both the incomplete reporting of The Economist and the questionable conclusions of the study’s authors.

The study in question was recently published in The Journal of Personality and Social Psychology [1]. The Economist [2] reported that the study, of 83  undergraduates from Florida State University, showed that praying for your partner reduced the rate of infidelity by a “significant” amount over the 4 weeks of the study. The group of 83 was initially tested to determine their level of infidelity, scored on a scale of 9 points; the average of the entire group was 3.5, with ” . . . [considerable variation] between the four groups.” (as reported by the Economist). A higher score indicates higher infidelity. The students were then broken into four groups randomly, and each was tasked with a different mental exercise. One of the groups was tasked with prayer for their partner. At the end of four weeks, as the Economist reports, “People who had prayed for their partners averaged 2.4, significantly lower than their initial scores, whereas those who thought positively about their partners or considered their day both showed ratings of 3.9—significantly higher” [2].

What does the paper actually say? As a scientist, my first question is: “What does ‘significantly’ mean – that is, what is the variation in each group around these reported means?”

To answer that, I looked at the paper. The results from this study were reported as follows in the paper [1]:

Infidelity. . . . participants in the prayer for partner condition reported significantly lower infidelity scores (M = 2.44, SD = 1.04) than did those in the neutral condition (M = 3.91, SD = 2.16), F(1, 78) = 7.61, p < .01, eta_p^2 = .87, and the positive thoughts about partner condition (M = 3.90, SD = 2.37), F(1,78) = 6.70, p < .01, \eta_p^2 = .80, but not those in the undirected prayer condition (M = 3.19, SD = 2.11), F(1, 78) = 2.02, p < .16, \eta_p^2 .45. However, those in the undirected prayer condition did not significantly differ from the other two conditions ( ps > .05, ns). All the means reported in this and subsequent analyses were adjusted for the covariates (see Table 2).”

The notation of the statistical analysis used here, ANCOVA, is described elsewhere [3]. The F-value and p-value above are as determined from the standard Fisher Test. However, the key here is that the means have uncertainties – standard deviations, or “SD” – which result from the small statistics of the samples (four groups made from 83 students means roughly 21 students per group, which is not a lot of people per group). If I took four random samples of 21 students from a single group of 83 total students, whose total average scores were 3.5 and whose SD for those scores were about 1-2 overall, I could easy generate four samples with this trend. The reality is that there is no statistically significant difference between these four groups.

To review the science here, 83 undergrads were broken into 4 groups. One group had to pray for their romantic partner each day (using a prescribed prayer), 2 had to conduct undirected prayer, and the last simply had to reflect on the day. All groups performed their assigned activity for the same time, once per day. At the end of the study, the mean infidelity score for the directed prayer group was 2.44 +/- 1.04. For the two undirected prayer groups, it was 3.19 +/- 2.11 and 3.90 +/- 2.37. For the neutral group (daily reflection) it was 3.91 +/- 2.16. Nothing can be concluded from this limited data set – these numbers are, within the rules of statistics, all the same. In order to shed any meaningful light on this question, a sample size at least 3-5 times bigger is needed.

The worst criticism I have of this study is that in its present form it CANNOT be conducted blind. The participants were quizzed on their fidelity before being broken into groups. One of those groups was asked to pray for their partner. Those people could easily be consciously or unconsciously influenced to act more faithful over the 4 weeks of the study, since there is likely some sense that their fidelity is being directly tested. There is no way to control for that in this study, and by construction it is potentially biased. At the very least, that bias would need to be quantified.

It’s my opinion that it was irresponsible for The Economist to fail to report the standard deviations in this study and to simply regurgitate the conclusions of the study without thinking about the results. The Economist is a major, international news analysis of serious record. Regarding the study authors, the fact that the study uses a small sample and doesn’t include any attempt at blinding or bias correction suggests this area of research needs more maturity. Until then, nobody should take it seriously.

When people want aspects of religious belief to enter the realm of scientific scrutiny, two things need to happen. First, those conducting the test owe those of religious faith the best possible scientific practice; this study, in my opinion, falls woefully short of that responsibility. Second, those whose faith is, literally, being put to the test should demand better methods but be prepared to expect results at odds with their belief system. After all, nature does as nature does, no matter what you or I believe. At the very least, inadequate studies like this only serve the confirmation bias, rather than real scientific discovery.

[1] “Faith and unfaithfulness: Can praying for your partner reduce infidelity?” http://www.apa.org/pubs/journals/psp/

[2] http://www.economist.com/node/16886238

[3] ANCOVA: http://udel.edu/~mcdonald/statancova.html

Second year

The first month of being a faculty member was one of the most difficult months of my life. Changing jobs is always hard, but going from post-doc to faculty is a promotion without a well-defined manual. The federal government made things extra special by creating a new DOE young investigator award program, but placed the deadline on September 1, 2009. A number of us thought it a little odd, if not downright frustrating, that new faculty were expected to hand in a proposal for a lot of money (over many years) in such a short period of time. That first year, you’re trying to establish your research program and get ready to teach. The first month is all prep, in unfamiliar territory, for a journey that at first has no clear path.

After my summer of work at CERN (and a subsequent two-week coma), Jodi and I stepped back into a normal work day. We’re both teaching now, just one course each (since we have research to do as our primary effort). We buried ourselves in prep work during the week before classes. This year, classes started on the Monday after the students moved in, so the semester came fast and furious. We’ve been busy working on proposals, big and small, and getting up earlier to commute in for our morning teaching. Meetings are interspersed with office hours, and research is taking a small back seat this week until we get settled in classes. It’s the typical beginning of a semester.

The start of the second year has been much better than the start of the first. We know a lot more people than we did last year. We feel a lot more like part of a community, both in our department and at the University. We have footing, something we didn’t have a year ago. We have ambitions and goals. The DOE early career award program has shifted to a proposal due date in November. And most important of all, we better understand just what the real lives of faculty are like and are taking advantage of the benefits while managing the surprises.

I’m very happy to be teaching again. That said, I am also eager to stabilize my schedule so that I can turn a lot of attention back to my research. I can’t leave it for too long. I crave the classroom, but I have a deep hunger that rages inside when it comes to tearing at the mysteries of the cosmos.

Back from Vacation

Well . . . sort of. I took a long vacation from personal things this summer to work in Geneva, Switzerland, at the CERN laboratory. I posted lots of things in the SMU CERN blog [1] and mirrored those posts in my own professional blog [2]. If you missed those adventures, have a look.

When I returned from CERN in late July, I got to see Jodi for just one day before she flew to Canada for a meeting of the SuperCDMS Collaboration. I spent the week at home, working on small projects and recovering from my summer of ATLAS work. I needed to get my brain back into something resembling a shape, and time away from work was the best way to do this. This effort culminated, upon Jodi’s return to Texas, in our first vacation since last year (I don’t count Christmas. I draw a distinction between a “holiday,” forced and mandatory institution-sanctioned time off, and a “vacation,” the use of unpaid leave to remember why life is worth living).

We took off for Door County, WI, where mobile phone reception is almost nil and wifi is spotty, at best. We did this on purpose. My e-mail has been going into an electronic DMZ from which I will extract the actionable items when I return. I have avoided work, and instead focused on life. Of course, this is not devoid of physics. I make a distinction between obligations and passions. On vacation, my only obligation is to myself and to my passions. Jodi and I discussed course structure philosophies, the nature of large collaborations, and other bits and pieces. These took minutes over coffee. We spent hours biking and hiking and swimming, or napping, or reading.

Photos from our adventures in Door County are available in our photo album online [3]. We’re not back to Texas yet, but our time in Door County is drawing to a close and we have some last stops to make on our way back to Dallas.

[1] https://blog.smu.edu/smucern/

[2] http://steve.cooleysekula.net/goingupalleys/category/event/cern-summer-2010/

[3] http://snappy.cooleysekula.net/thumbnails.php?album=14