This content is locked. Please login or become a member.
Weighing Preliminary Findings
For any study, ideally, we’d like this to be published in a peer-reviewed scientific journal. Why? Because peer review means that it’s been vetted independently by some of the world’s experts to make sure that the methodology is sound, that the conclusions drawn are supported by the evidence. But perfect should not be the enemy of good. Because the peer review process is so stringent, it might take many, many years for a paper to go through peer review. Whereas as practitioners, we might need to make a decision immediately. So sometimes it may well be that we base a decision on some preliminary findings.
So what does that mean? It might mean that a paper is a working paper. It’s a draft which hasn’t yet gone through peer review. But while this might mean it’s not a hundred percent accurate, it’s not zero percent accurate either. So how much weight can we put on a non-peer-reviewed study? This is again where we want to go to the credentials of the authors. So is this a report written by a management consultancy? So often management consultants, they might be great at advising a company on how to address its problems, but they don’t have expertise in scientific research. Often, they are doing research to build their brand, so the research they put out will only give conclusions that the public wants to hear because releasing a paper with those conclusions improves its image.
What if the paper is done by some academic scientists? Well, look at whether they have expertise in this particular field. If I was to write a preliminary paper on responsible business or sustainable finance, you might put at least some weight on that because that is my expertise and I have a track record of publications in the top scientific journals on that topic.
Exploring the Body of Knowledge
Even if you have a well-credentialed author, it might be that he or she can sometimes make unintentional mistakes, or maybe he or she might have different incentives and might exaggerate the findings to give you a paper which is more likely to sell. So what this suggests for us as consumers of information is to avoid putting too much weight on one single study, but to look at scientific consensus. So that is the weight of evidence out there, which is produced by a whole body of researchers rather than being skewed by one particular study.
For example, there was a paper published in The Lancet, a respected medical journal, claiming that vaccination leads to autism. But there’s loads of other studies which claim the opposite, and indeed what a systematic review will do will be looking at the density of studies. It will be finding if sometimes papers get retracted, which was the case for that Lancet study on autism. And so it avoids being skewed by one particular study, which is a particular problem if we suffer from confirmation bias. We might try to find the one study that supports our viewpoint and ignore the rest of the scientific body of knowledge.