Michael Ranney and Students Explore Reasoning with Numbers

Can a person's point of view be changed just by hearing a new number? Associate Professor Michael Ranney has launched a research effort called Reasoning with Numbers to determine if exposure to accurate and critical statistics actually moves individuals to reexamine their ideas. "I'm interested in exploring whether simple information can alter a person's policy on an issue, and how people use numbers to formulate or justify their beliefs," Ranney explained, describing the study that he is conducting with graduate students Jennifer Garcia de Osuna and Janek Nelson, along with undergraduates Franz Cheng, Paul Winterbauer, Laura Germine, and Johnnie Chamberlain.

Ranney and his Reasoning with Numbers (RwN) research team began by questioning a sample of UC Berkeley undergraduates about what they think is the average SAT-I percentile of first-year undergraduates admitted to Cal. Ask yourself what you estimate it is. Now ask yourself what you would prefer it to be, if you had the power to change it.

Many are surprised to learn that Cal students score on average around the ninety-fourth percentile on the SAT-I. "You might think UC Berkeley students would know this statistic, since it relates to them, but on average their estimates were generally ten percentiles below the real figure," Ranney mused.

The interesting twist is that the students' views on what the percentile should be often changed when they heard the correct statistic. "Prior to hearing the actual statistic," Ranney said, "a third desired the mean percentile to stay what they estimated it to be, a third wanted it higher, and a third thought it should be lower. After the feedback where they found out the correct number, the students who were considerably surprised (all of them underestimated) were twice as likely to advocate relatively less stringency on SAT-I scores. On the other hand, the average preference jumped up from around the eighty-fourth percentile before the feedback to about the eighty-ninth percentile after the feedback," Ranney elaborated. "Given the current UC admissions controversy, we were struck that only two of ninety-three students stepped a bit out of the framework of the question and recommended abolishing the SAT-I."

Reasoning with Numbers is not limiting itself to sticker-shock on SAT scores. It's also asking people to look closely at over a dozen other topics, including such hot button issues as the U.S. rates of immigration and abortion. "In these cases, about 80% of the people are basing opinions on significantly inaccurate base rate information," Ranney said. "Very few people, even those who feel strongly one way or the other about immigration, for instance, realize that the annual increase in the total U.S. population as a result of legal immigration is roughly one-third of one percent. But what we're studying is not merely the degree to which people are informed or misinformed, but the extent to which the correct information makes them infer new conclusions and reevaluate even their own sometimes firmly held beliefs." Ranney and his team call this new research paradigm numerically driven inferencing.

Professor Ranney and his students are still working on analyzing the large set of data they've collected, including qualitative analyses of elicited rationales. For instance, people who are surprised by the rather high proportion of pregnancies that end in abortion are more likely to then reason about their new preferences in terms of birth control.

Some of the RwN Group's next steps will include discussing their methodology and initial results with interested colleagues and expanding their study beyond members of the campus community. The group plans to present a paper on its findings at the annual meeting of the Society for Judgement and Decision Making in November in Orlando. "Our ultimate goal," said Ranney, "is to improve both adults' and children's numerical literacy and rationality."

Laura Germine, Janek Nelson, Michael Ranney, Johnnie Chamberlain