Baylor religion study biased

October 4, 2006

Baylor University has recently conducted a detailed study of religious belief and released preliminary results in September 2006. The study was funding by the Templeton Foundation, famous for giving grants to organizations and individuals who try to reconcile religious belief with science. The survey's results have been widely reported. A closer look at the results, however, reveals spin, bias, and important findings that have not been widely reported. The Baylor report is available here (in PDF).

One widely reported finding from the study was the number of "Nones," or people without religious affiliation, has allegedly been dropped from about 14% of the population (found in earlier studies) to around 10%. This finding was uncritically repeated, for example, in the Dallas News under the title "Study: Number without religion is overstated." The researchers attribute this to more targeted questions, but it is an unfortunate artifact of their biased methodology. Respondents were asked about both their religious belief and any affiliation with a church. The study's 10% finding is the result of effectively changing the answers of the people who claimed no belief affiliation but who did report a church affiliation. The findings confound the two issues. There are many reasons to attend a church beyond one's religious belief including involvement with a community, friends, or family. To confirm this problem, Baylor sociologist Kevin Dougherty says that "when we asked the same questions other surveys asked about identification or preference, we got the same roughly 13% to 14% they get." It seems their methodology was devised to reduce the number of "Nones."

Perhaps the next version of the study should ask the number of people who have ever visited a doctor or hospital and subtract those respondents from the number of reported believers. Seeking medical help is a clear demonstration that the seeker does not believe that appeal to an omnipotent god has hope of being effective. You'll never see questions concerning practical religious belief on religious study instruments, however, because most religious studies are designed to inflate the number of believers. Numbers, whether real or not, are a source of political power.

On a related topic, any conclusions reported about atheists, agnostics, freethinkers, brights, rationalists, or humanists are suspect. The study lumps all of these groups into either the "no religion" category or the "other" catch-all category, both of which may be problematic for characterizing non-believers who have the courage of their convictions. While the survey instrument may have some validity for believers, most non-believers would probably lose interest in the survey as they are not recognized as a group worthy of mention despite being a larger group than many of the religious beliefs individually listed. Given that and the low response rate for the survey (46%), you can bet that many non-believers were simply not counted after having been exposed to such and obviously biased survey instrument. The instrument is included in the last few pages of the above PDF file, should you care to examine it. The report even makes the claim that "Atheists are certain that God does not exist," which shows an amazing lack of understanding of their subject matter on the part of the Baylor researchers.

A surprisingly high number of respondents (43%) were reported to have admitted to having "prophetic dreams," or dreams that predict the future. The report lists this number among the various paranormal beliefs that the study looked into. The prophetic dreams results were even included in an overall measure of paranormal belief, used in later correlations. Upon further inspection, however, one discovers that the "prophetic dreams" category is based on two questions in the survey instrument in a category listed as "The New Age":

As an adult, have you ever had one of the following:... A dream that later came true?
To what extent do you agree or disagree with the following statements: ... Dreams sometimes foretell the future or reveal hidden truths.

There are certainly non-paranormal reasons for responding to both questions positively. Dreams are often about one's life and issues that concern a person. It's not at all remarkable that people would commonly dream about their own futures and their future actions--things that might later come true. For example, if you were having trouble at work and dreamt that you were fired. If you were in fact fired after the dream, it doesn't mean that your dream was prophetic or that you were psychic, even though you would likely answer the survey question "yes." As for "hidden truths," dreams can be a window into one's own subconscious. Such interpretation can be based on psychology--not parapsychology. Perhaps by making the finding more lurid, the researchers were able to get more publicity for their survey results.

The Christian biases of the surveyors show up in their reporting of beliefs about the personality a monotheistic god, presumably for the subset of the respondents that believe in a single god. This category includes Deists, who worship the God of Nature, Jewish believers who worship Yahweh, Muslims who worship Allah, Wiccans who might claim to worship the Goddess. All of these different beliefs were categorized under belief about "God," the proper name of the Christian god. Perhaps this complaint is a nit, but it does not show respect for the beliefs of non-Christians, which you would expect in a formal study of religious belief. The Austin American Statesman repeated this mistake in their uncritical articles on the study.

The final issue of spin is a finding about beliefs in the personality of their god that the report didn't fully explain. The survey divided the space of possibilities into two orthogonal dimensions that cover the space of possibilities. Either their god is engaged or not. Either their god is angry or not. From these orthogonal dimensions, the survey had a number of questions that grouped believer respondents into four categories to describe the personality of their god. These categories were benevolent, authoritarian, critical, or distant. The survey found that believers fell into these categories with roughly equal numbers. While the study focused on the demographics behind the different categories, it neglected to point out that the relatively equal numbers of believers in these categories is evidence that there is no objective reality behind the assertions. Presumably, if these believers were having regular conversations with this unique god in prayer and worshiping it in their churches, they would, in the aggregate, have some statistically significant agreement of what that god's personality is. By contrast, I'll bet the majority of believers could correctly name the color of the mid-day sky.

The Templeton Foundation did not spend their money wisely on Baylor's work, if they were looking for a serious study of religious belief. If, on the other hand, they were looking for

half baked "research" to downplay the number of Nones that would be picked up and uncritically reported by the press;
information that would help with marketing to believers; and
demographic information that would help politicians use religious belief to their advantage,

the Templeton Foundation got what it paid for. Unfortunately for them, their money gave another convincing piece of evidence against religious belief. As the reader may recall, the Templeton Foundation funded a large and well designed study that showed no positive effect for intercessory prayer.

Baylor University is home to Intelligent Design promoter William Dembski, who, despite years of work, has yet to make a claim in support of an "intelligent designer" (aka "God") that hasn't been conclusively proven false by the scientific community.

Don Baker

From the officers:

The audio and video from Donald Wright's June Lecture are now available.

The Annual ACA Bat Cruise will be Saturday, September 28th, 6-8pm. Get your tickets early to ensure seating.

Return to current news | News archive

Atheist Community of Austin ° 1507 W. Koenig Ln. ° Austin, TX 78756-1415

Voice Mail: 512-371-2911

ACA is affiliated with
Atheist Alliance, American Atheists, American Humanists Association,
the Council for Secular Humanism, and the Austin Coalition of Reason.

"Atheist Community of Austin" is a trademark of the Atheist Community of Austin.