Over the past several months I have read dozens and dozens of academic papers. These have been primarily in the disciplines of communications, educational and political psychology. Many proclaimed to report on empirical research, but their method was nothing more than a few interviews and great deal of theory-rationalisation. There is surely a double standard. While senior academics can proffer stylised facts, PhD students like me are justifiably compelled to complete coursework about rigorous research methods.
[For me that's a story in itself. The PhD coordinator is demanding that I fly weekly to Sydney to attend every single class, which I am simply unable to do. Without special dispensation from the Dean, my candidacy is currently under a cloud.]
I have read all of these papers to find out how other researchers have collected data about deliberative and collaborative processes. Leaving aside the works that border on empirical pretence, much of what is left is experimentally and quantitatively derived. These studies start with hypotheses and then use survey or questionnaire to prove or disprove them. They are theory-driven, which means they find what they are looking for.
In the area of psychology, much of this data collection takes the form of personal inventory. These would include judgment statements like "I tend to make decisions on evidence rather than intuitively", on Likert scales ranging from strongly disagree to strongly agree. I'm reminded of the USA entry forms after 9/11--"Are you a member of a terrorist organisation?" Silly really.
There are various levels of falseness. First there is the invitation for flagrant deceit--people will tell you precisely what they think you want to know. Second, and for academic researchers in particular, you'll get made-up answers because subjects will not have consciously thought about it. James Fishkin, who developed Deliberative Polling, calls these
phantom opinions. Lastly, and insidiously, you'll get what critical theorists call "false consciousness", a personal belief induced by hegemonic social pressure (eg. "autonomy is good").
I am interested in the willingness and capacity of citizens to participate in deliberative processes. As part of this investigation, I want to know what predispositions, traits or epistemological theories they bring to the table, regardless of validity. I'll be using qualitative methods to uncover these (eg interview, observation, subject narrative), and will probably have to break new ground in this methodological space considering the apparent paucity of giants to stand on. But I'll keep reading and maybe something will pop up....