Ok, maybe I added the last four on and that's just me. I do also conform to many of the other aspects of that stereotype though. However I also feel the need to say "I am not a trekkie" of course all trekkies say that but it is true in my case. No really.
I've spoken before about my atheism and this is part of a larger scepticism about anything new age, psychics, alternative medicine etc. I have even been known to bend people's ears about this if they will stand still long enough to listen. Of course some people, well actually one in particular, enjoys baiting me for his entertainment with various outrageous statements like Dawkins is your pope etc. He should know better really given he is currently researching for his PhD in Neuropsychology however it was whilst at his studies that he came across an unrelated article by Professor Daryl Bem that purports to show evidence of psychic abilities, namely presentiment or precognition and my friend couldn't wait to share it with me to say "ha, what about your scepticism now?"
The article in question, all 61 pages of it, can be read on the author's website and is due for publication in a serious and respected journal from the American Psychological Assocation's Journal of Personality and Social Psychology, remember that title by the way. I'll come back to it later.
What does the paper say? Well, you can read summaries of it via this web article which also makes some good points about it.
I've read about half of the paper and skimmed the rest (I'm a geek but I have a life, and a job). It presents data from 9 different studies with over a 1000 participants in total and reports a statistically significant effect in several of them that cannot be explained by current scientific theories. They all work on the idea of a retroactive effect, i.e. the supposed cause actually takes place in the future. Here's an example of the description of one experiment from the paper itself.
This is an experiment that tests for ESP. It takes about 20 minutes and is run completely by computer. First you will answer a couple of brief questions. Then, on each trial of the experiment, pictures of two curtains will appear on the screen side by side. One of them has a picture behind it; the other has a blank wall behind it. Your task is to click on the curtain that you feel has the picture behind it. The curtain will then open, permitting you to see if you selected the correct curtain. There will be 36 trials in all. Several of the pictures contain explicit erotic images (e.g., couples engaged in nonviolent but explicit consensual sexual acts). If you object to seeing such images, you should not participate in this experiment.
At this point I would like to say that my psychologist friend who sent me the paper could learn a thing or two about designing interesting experiments. When I was a participant in one of his studies recently what did I spend two hours looking at and responding to? Coloured circles in an array. But I digress....
The clever bit in Bem's experiment is that the curtain behind which the picture will be placed is not actually chosen until after the subject has made a "guess". Only after that does the computer randomly select a curtain and puts the picture behind it (we'll come back to the word random as well).
From pure chance you'd expect to get 50% of the guesses right, but Bem says participants get closer to 53% for those cases where there is a naughty picture behind the curtain. Somehow participants can predict where porn will be in the future. (Explains a lot about the Internet maybe). That small difference is statistically significant according to various tests you can do for that sort of thing that I don't claim to understand. (My knowledge of stats and probability is something I'd really like to improve.) Other studies in the paper relate to different effects and get similar small but seemingly unexplainable results.
So there you have it. Rigorous scientific proof of psychic phenomena.
Err, well no not quite. As Ben Goldacre would say "I think you'll find it's a bit more complicated than that". (His book Bad Science is an excellent read for anyone who wants to know what to make of various supposed scientific pronouncements in the media). Just as one swallow doesn't make a summer so one paper however well written doesn't make a proof. (The paper does seem very thorough to my amateur eye though).
Repeatability is very important in science and there are already several attempts under way to repeat the experiments to see if they get the same result. Importantly there is also a place where scientists can register to say they are replicating the experiment. This means results can't be brushed aside or hidden, be they positive or negative. People will know the studies have been conducted. (This counters the so called bottom draw effect or publication bias. Positive results are more likely to be published that negative ones.) Of course when it comes to headlines on various newspapers and websites the results of these repeatability tests will almost definitely go unreported or get little coverage if they fail to confirm the headline.
Aside from psychic phenomena what could explain the results? Well one thing is the use of the term random. If there is any bias in the way picture's position is chosen this could have an influence on the results. That said, there is a significant discussion of this in the paper itself but again it needs an expert to read and understand it. Luckily such experts are looking over it in the psychology community.
Professor Bem, despite some vague words about quantum effects, doesn't himself have a way to explain how the psychic phenomena work and the experiments weren't designed to test any theory of psychic powers that he had. This in itself could be an issue when it comes to analysing data. This paper, which I haven't read fully yet, criticises such an approach as something of a fishing expedition. It also makes the point that for effects that would completely confound current theory and practise there needs to be a somewhat higher burden of proof or significance for sound statistical reasons.
Chris French who studies the psychology of anomalous experiences spoke about the study on the Guardian Weekly Science podcast. He made the point that one of the reasons this study is getting so much attention is because Daryl Bem is a respected name in social psychology and the journal is for a mainstream subject. Most psychologists don't study parapsychology and so don't read parapsychology journals otherwise they would see studies like these pop up now and again but they turn out to be unrepeatable. At the risk of being accused of playing the man not the ball a related point I would make is that Daryl Bem is known for his work in social psychology and the journal he's published in is one of social psychology. That doesn't denigrate his results but it perhaps gives another small reason to look very hard at them.
What all this shows is how science really operates and there has been some excellent work criticising Bem's results. Bem himself has tried to address many criticisms in his paper pre-emptively and he is making the software he used available for other to examine and try his experiments. This isn't like literary criticism where an author would get all huffy about a bad review. Scientists, good ones anyway, expect and welcome criticism of their work. It helps add to the body of scientific knowledge.
Finally, when it all comes out in the wash, if evidence is found of some psychic style effect then I'll change my views on it. Though the really interesting part will be finding out how the effect works, a whole new theory of time and space or multi-dimensional parallel universes, who knows.
That's the scientific method, evidence is examined, hypotheses and theories are formed and tested, new ones emerge for the greater understanding of all and we change our world view. How many god botherers or new age alternative medicine proponents would say the same?
Simon Stanford is a forty-something IT type based in Birmingham. He is an Open Source advocate with an interest in science and scepticism. He writes a blog at http://simonstanford.blogspot.com & http://raetsel.wordpress.com. He also writes short fiction which can be read online at http://abctales.com/user/raetsel
3 comments:
Small nitpicking point: it's 61 pages only because of the APA submission format. Once published, it will be much less. If ever published actually as given the controversy, I wouldn't be surprised seeing the journal retracting its decision (or am I just practicing wishful thinking here?).
More importantly, from a psychologist point of view,it's not even really well written. Bem took some liberty with the usual standards of scientific writing in the field.
And last but not least, the controversy has been now lasting for two months in Psychology. Though for it to be a controversy, we should argue, but anyone who would have been participating in the journal club we specially had for it in Birmingham Uni would know how much the consensus is wide: it's crap. Period. This digression finished, two months of gossip is already too much of attention given to something that doesn't deserve it (and too much of bad reputation thrown on a topic that already has its share of it).
I'm guessing you are a psychologist Amélie or involved in the field anyway so it is really interesting to hear an insider's view as it were.
I am very much an outsider but with an interest in skepticism so it is interesting to contrast our two views on the subject. Imagine the further contrast with members of the general public who only read a summary or press release of this report. What impression will they take away from it?
Yes, I am a psychologist in the broad meaning of it (i.e., I am doing a PhD in Experimental Psychology).
Inside the field, right now, it's pretty much just a matter of laughing for us. The journal club we held on the paper was in fact very boring as we couldn't find anybody to try and defend Bem's side.
This paper (http://www.ruudwetzels.com/articles/Wagenmakersetal_subm.pdf) already did an excellent job in dismissing Bem's methodology and results. After it, there is not really any need to try and destroy it further.
Now, it's pretty technical, and you have a point about the repercussions in the general audience. But those two New-York Times papers (http://www.nytimes.com/2011/01/11/science/11esp.html & http://www.nytimes.com/2011/01/06/science/06esp.html?_r=1&pagewanted=2) are accessible and pretty unambiguous about what message should be taken from Bem's paper.
On the other hand I have been monitoring a bit the web and Twitter to see if pro-ESP and alikes are jumping on it, and most of what you can find is actually seeing this as a joke and Bem as an idiot.
Which brings back to what is my take on this as a psychologist: I am embarassed and feel like we're going backwards once again in our transformation towards "proper" science; or to be precise in the representation of ourselves as a science. What am I supposed to answer when I said I am a researcher in Psychology and am being asked if I can read their mind...? ;-)
But this is my very own opinion. Some colleagues might be embarassed by other aspects of the situation (the peer-review flaw which occurred here, the poor stats, the unprofessional writing...).
Post a Comment