Moreintelligentlife.com is the website of Intelligent Life magazine. Articles about Arts, Lifestyle, Travels, and Ideas

THE PERILS OF POPULARISING SCIENCE

THE ART AND SCIENCE OF COMMUNICATION | January 23rd 2008

Karmelize/Flickr

In appealing to the layperson, scientists often sacrifice accuracy in favour of simplicity, and the field suffers for it. Jason Zevin, a cognitive neuroscientist, takes those best-selling theorists to task ...

Special to MORE INTELLIGENT LIFE

I have to confess I enjoy telling people I'm a "brain scientist." It is fun to say out loud, and tends to draw people out in unpredictable ways. A friend of mine recently spent a night in jail (long story) and when it came up that he did cognitive neuroscience research, his cellmates began peppering him with questions about whether different types of brainwaves are related to different states of consciousness (they are, sort of), and whether it's true that we only use 2% of our brains (it isn't).

There are many motivations for talking about brain science with people outside of the field, above and beyond killing time while you wait for a public defender to show up. For starters, it's what we're most interested in, and what we feel most comfortable talking about. Furthermore, many people are genuinely interested in how their brains work, and it's very satisfying to oblige them with answers when we can.

One problem in these situations is that despite the vast amounts of knowledge the field has accumulated about, for example, the formation of new memories, the data rarely provide answers to first-order questions, such as "how are new memories formed?" It turns out that there are lots of different kinds of memory that are dependent on dissociable systems in the brain and have very different properties from one another. What we normally think of as "memory" is actually a confederation of related but distinct processes.

There are also all kinds of ongoing debates about the role of this or that protein in particular laboratory paradigms used to study memory. These often involve trying to change the baseline rate of electrochemical transmission between two neurons in a disembodied slice of tissue. It's not that there hasn't been tremendous progress in memory research, but that the progress has taken us into territory that is not even on the map as far as folk psychological notions of "memory" go. Much of this remains unexplored and poorly understood.

At best, most of what is known is more complicated than I'm able to understand--much less explain to a general audience. And at least some of what I know about any topic in neuroscience is liable to have been discredited by a recent article in Science or Nature. This makes me cautious whenever anyone turns to me for an authoritative opinion on anything regarding the brain.

This is why it is always so disorienting to talk to people who have just read or are reading anything by Steven Pinker (such as his recent piece "The Moral Instinct" in the New York Times Magazine). Often, these people know all kinds of amazing things--including things I'm pretty sure aren't true. This is not to say that Pinker is a charlatan (although some researchers might actually go this far; a colleague just vandalised my copy of "The Stuff of Thought", changing it to "The Stuff I Just Thought Up"). The problem is that our field is one with many open questions, many confusing and apparently mutually exclusive data points, not to mention a dizzying array of theoretical perspectives to consider.

As scientists, we learn to live with the fact that much of our work is highly subjective. There is actually very little that any two people who call themselves "cognitive neuroscientists" are guaranteed to agree on. Mostly we make progress by choosing the side of an argument that seems most plausible given our pre-theoretical commitments, and trying to provide data that would convince someone starting from the other side.

We depend on the people who disagree with us to keep us honest when our imagination or our capacity for due diligence fail us. Any published work has to survive a process of peer review in which researchers working on similar topics evaluate whether our data mean what we say they mean. Empirical evidence is always reported along with a description of the methods used, which should, in principle, be enough to replicate the result. In other words, there is a system in place that is designed to rein in our impulses to put our thumbs on the scale when weighing the merits of our arguments and the data that support them.

The pressures of writing for a general audience are different. Here, the goal is to present a clear, compelling narrative that will be attractive to an audience of non-experts, who want to come away feeling as though they've learned something. Who wants to buy a book called "How the Mind Works" only to discover that scientists don't really agree on the most fundamental issues regarding how the mind works?

I think there is probably an audience for a measured, nuanced account of the various internecine struggles among scientists who are ostensibly working together to understand the mind, but it is probably much smaller than the one for a book that "explains what the mind is, how it evolved, and how it allows us to see, think, feel, laugh, interact, enjoy the arts, and ponder the mysteries of life."

Along with the pressure to be clear, coherent and flashy, the usual bulwarks against inaccuracies don't really apply. The general audience is not in a position to try to replicate results, nor do they have the training to evaluate what you are saying. Basically, they must take the author at his word.

As a result, it is nearly impossible to provide the layperson with a fair account of the state of knowledge regarding, say, the role of word meanings in syntactic processing. Pinker is acutely aware of this. Chapter 3 of "The Stuff of Thought" begins with an extended riff on the notion of a straw man, meant to signal the fact that he is about to characterise a number of positions with which he disagrees, and he may, despite his best efforts at fairness, slip in a cheap shot or two.

The problem for the audience is being able to identify the cheap shots when they come. Here's an example: Pinker seems to undermine the theoretical framework of "radical pragmatics" by citing research Alan Kawamoto and Jay McClelland published in 1986. The citation given is of a somewhat preliminary research report that appeared in an edited volume, in which the authors present a computer simulation model that captures some interesting aspects of sentence processing.

Their model was able, among other things, to determine which of several meanings was intended for an ambiguous word, using a process of probabilistic constraint satisfaction rather than the hard and fast categorical rules favoured by most contemporary linguistic theories. But the model did make some embarrassing errors, for example, given the sentence "The window was broken by a bat", it produced an interpretation consistent with both the mammal and the sports equipment.

These errors are presented by Pinker as conclusive evidence against radical pragmatics, which is a very cheap shot for at least two reasons. The first is that it is essentially argument by anecdote. In Pinker's telling, the model is only as good as its most embarrassing error, and we can learn little from its successes. But in the decades since the initial research, constraint satisfaction mechanisms have become a fairly mainstream, if not entirely uncontroversial part of both linguistics and the psychology of language. This is at least in part because many people have found merit in the model despite its problems, and have done serious theoretical and empirical work to follow up on it.

Furthermore, in Pinker's example, the model is made to stand for something it was never meant to do. It was intended, implemented, and most widely interpreted as a demonstration of constraint satisfaction in grammatical processing. While it shares some assumptions with radical pragmatics (and cognitive linguistics in general), it is not an implementation of that theory at all, and so its various foibles can't really be taken as evidence for or against it.

It is not as if argument by anecdote and non sequitur were entirely absent from the peer-reviewed literature. What makes them troubling in Pinker's case is the questions they raise about the goals of his work and the consequences for the science. Why go "above the filter," so to speak, and publish the bulk of one's academic work in general-interest form?

When arguments are made with the goal of convincing people who lack the expertise to evaluate them, they are bound to fall short of what it would take to move the field forward. What Pinker ends up representing is an unfortunate fact of life in disciplines where so little is known with any certainty: often the rewards for winning arguments are much greater than the rewards for being right.

(Jason Zevin is an assistant professor of psychology at the Sackler Institute for Developmental Psychobiology, Weill-Cornell Medical College)

See The Economist's survey on The Brain.