Edward Rothman, professor of statistics, director, U-M Center for Statistical Consultation and Research

"There are all kinds of ways that surveys can be misleading. In a survey of crime statistics a few years ago, it was discovered that Grand Rapids led most cities in the rate of rape and Detroit was far behind. The reasons had to do with the definition of rape. Certain levels of assault were counted as rape in Grand Rapids and not in Detroit. So it's always important to understand the operational definition of what is being measured in a survey.

Photo by Marcia L. Ledford, U-M Photo Services Rothman

"Then there's the Hawthorn effect, named for a factory in Cicero, Illinois, just south of Chicago, where time studies were initiated. Researchers would turn the lights on a little brighter in parts of the factory, then measure the length of time it took workers to perform certain tasks. They discovered that people worked faster than they did when they weren't observed by the researchers. This kind of effect can also occur in survey research, when people respond in a way they think meets the expectations of the interviewer.

"People may also be motivated to provide information that meets their own needs. Years ago when water levels in the Great Lakes were high, I was working with the Army Corps of Engineers to determine the extent of shoreline property damage. We sent surveys to people who lived in homes along the Lake Michigan shoreline and found that property owners reported higher levels of damage than people who were renters, even when property owners and renters inhabited contiguous pieces of land. There are statistical techniques for determining the extent to which this kind of bias is going on, and obtaining answers that are probably closer to the truth than could be obtained by guessing.

"Another common source of survey bias comes from the interpretation of the results. Some studies, for example, show that the longer you live together prior to marriage, the more likely you are to divorce. Some people say, well, obviously you shouldn't live together before marriage. Others point out that people who live together tend to have lower incomes, and having a low income increases the chances of divorce. The data are just sitting there. People see them through their own eyes, their own perspectives. At the very least, we ought to try to see things from a variety of perspectives. I'm not sure how easy this is, though. There's a poem by Yeats called 'For Anne Gregory.' It begins:

Never shall a young man,
Thrown into despair…
Love you for yourself alone
And not your yellow hair.

"And it ends by observing that,
...only God, my dear,
Could love you for yourself alone
And not your yellow hair."