I went to an interesting panel discussion about science communication the other day. The panelists were all accomplished scientists who did a lot of outreach and science communication and were talking about how important and rewarding it is, and how we should all do more of it.

To  which I agree, but more and more, I am also wondering: How do we know it’s working? And are there things that work better than others? And are we really spending our time and resources where they are most needed?

Because here is my problem: I love doing outreach activities. I had high school students visit the lab last summer as part of the Science Insights programme at the University of Edinburgh. I have volunteered at the Cambridge Science Festival, where I have made DNA sculptures out of jelly babies with young children. I have helped put on mini-symposia to discuss the impact of science on society (and vice versa).

But I am a bit worried that there is an aspect of preaching to the choir in all this. The people who come to those events are those interested in science in the first place (“Yeah,” a six-year old interrupted me, as I was launching into my spiel at the Science Festival, “I know what a DNA is.”) They will come to those events, they might learn something, we will probably learn something from them, everybody has a good time, then we all go home. But what changes? What really happens?

The people we really need to reach out to are the ones who don’t come to those events. Those who don’t trust science, and are therefore putting themselves and others at risk. The people who spend a lot of money on homeopathic remedies and other quackery. The parents who refuse to have their children immunised. The policy makers who deny that human-caused climate-change is real. This is where the real hard work is. And how do we get there? How do we talk to them? How do we do it so it’s a real dialogue and  not just us scientists lecturing people about what’s good for them (which isn’t really working that well, is it?)

In other words, how can we target effective science communication so that it has the highest impact? Or maybe the best impact-to-effort ratio? How do we measure whether a science commuication activity has been successful? (Beyond just number of visitors or clicks. Those are easy things to measure, but poor indicators of quality or effectiveness.)

Given that virtually all science communicators are scientists (either still active in research with a sci com sideline, or people with a solid science background who have moved into communicating science full time), it is astonishing how  little we have thought about actually figuring out how it works. And/or whether it works. We are hardly collecting data, we are not comparing strategies to find what works best, we are not coming up with new ideas that are demonstrably better (in the sense of more effective) than the old ones. We need to  do more to evaluate what’s working and to make science communication more evidence-based.