I am currently doing an Alaskan tour of six cities in seven days, working with educators in a state-wide, publicly-funded charter school system. Even though it is cold, it is a lot of fun. Alaska is beautiful, and the charter school system is excellent. It is great to see quality education occurring in such a novel way.
Because airplanes are the main way one gets from city to city in Alaska, I have been spending a lot of time sitting in airports, on tarmacs, and occasionally on an airplane that is actually flying. As a result, I have been doing a lot of reading. I came across an interesting article in Science Science News today1, and I think it is a great illustration of something I stress in most of my science books.
For several years now, scientists have been using functional magnetic resonance imaging (fMRI) to observe functioning brains. The main technique involves using a magnetic resonance imaging machine to look for small changes that occur within the blood vessels of a person’s brain. The very reasonable argument proposed is that the more active a neuron is, the more blood it needs. Thus, if the fMRI sees an increase in blood flow to a particular region of the brain, the neurons in that region must be more active. So…a subject is stimulated in some way, and the fMRI looks for increases in blood flow. Any region of the brain that “lights up” must be the region that is responsible for either processing whatever stimulus was provided or producing a response to it.
Several hundred papers have been published discussing the results of all manner of fMRI experiments, and they have made all sorts of definitive conclusions regarding what regions of the brain are responsible for processing various stimuli or producing various responses to those stimuli. Well, Craig Bennett wanted to see how reliable fMRI experiments are, so he decided to do a very simple baseline test. He used fMRI to study the way a dead fish’s brain responds to stimuli.
The results were quite interesting.
It turns out that when the dead fish was shown pictures of scenes that depicted emotion, a part of the dead fish’s brain lit up under the fMRI. If the fish had not been dead, the definitive conclusion would have been that the regions of the brain that were lit up were the regions responsible for how the fish processes the images it was seeing. Obviously, however, Bennett knew that his study said nothing about how a dead fish’s brain works. Instead, it says something about the reliability of fMRI studies.
In order to see what was going on, he tested the statistical methods used to take the volumes of raw data the fMRI collects and translate them into the result the experimenter actually reads – the region of the brain with increased bloodflow. Not surprisingly, he found that the way the raw data were processed (which no one ever really checks or questions) can take random noise and make it look like increased blood flow. What he as an experimenter read, therefore, was just nothing but random noise. The problem is, had he not been using a dead fish, he would never have questioned the result.
The paper goes on to list several other problems associated with fMRI experiments. For example, because of the nature of fMRI, the researcher cannot measure activity over the entire brain at once. He or she has to focus on a specific area. That, in itself, produces a bias in the results. Also, because of the nature of these kinds of experiments, they are rarely repeated. In some of the few cases where experiments have been repeated, the two trials did not produce consistent results. The paper even says that recent research casts doubt on the fundamental (and very reasonable) assumption that increased blood flow means increased neuron activity.
In short, then, an experimental technique that has been used for 20 years to make definitive statements about brain function is, to some extent, inherently flawed. Indeed, according to the article, a review of all papers using fMRI techniques published in Nature in the year 2008 found that almost half of them contain significant methodological errors.
So what’s the point? Is science flawed? Of course it is, but that’s not news to any scientist. Because it is flawed, is it useless? Of course not. Science makes lots of mistakes, but over time, it produces a lot of reliable, useful information. The point is simple: Scientific conclusions are based on assumptions. Sometimes those assumptions are testable, sometimes they are not. Sometimes, they are testable, but no one really thoroughly tests them. As any good scientist knows (but few in the general public understand), this makes scientific conclusions very tentative.
The fact that a whole lot of expert scientists believe something, therefore, means rather little. A whole lot of scientists who are experts in brain research believed the conclusions of fMRI studies, and now we see that a significant number of the studies that led to those conclusions are flawed. As a result, a significant number of the conclusions are probably incorrect. Because of such events, a good scientist continually checks assumptions against the data, regardless of how many scientists believe the assumption. As a frequent commenter on this blog (Norwegian Shooter) recently found out, for example, simply listening to the majority of scientists can often make you dead wrong!
Of course, being told by a bunch of scientists what to believe is much easier than actually thinking for yourself. Unfortunately, the easy way is rarely the best way.
REFERENCE
1. Laura Sanders, “Trawling the Brain,” Science Science News 176:16-20, 2009.
Return to Text
Thanks for the shout out! Right back at you! I would love it if you would comment on my own blog, I recently resumed after a mid-winter break.
I must say that I am very jealous of your travels. Nice work if you can get it.
I too have been quite skeptical about fMRI brain research. It has certainly been oversold by psychologists. However, I remain skeptical about your interpretations of science articles, so I look forward to reading the Science piece cited.
PS, when I saw the title and Alaska in the first line, I thought for sure you’d include a Sarah Palin reference. Oh well.
PPS, why do you suppose no other commenters have taken issue with my contributions here?
Perhaps I will comment on your blog when my life settles down. However, I am coming back from AK just to turn around and go to Thailand and then New Zealand.
Sorry about the lack of reference to AK’s most famous resident. I am actually not a fan of hers, so it didn’t even occur to me.
It’s hard for me to judge the actions of others. However, if I were to make a stab at it, I think most of my readers are a bit more gentle than I am. Perhaps they don’t want to make you feel “ganged up” on. Alternatively, they might just enjoy reading the exchange between us and feel no need to comment.
Well there aren’t exactly a large number of regular commenters on the blog. I know why I don’t respond, and it’s partly to avoid getting caught in a crossfire, or starting a crossfire against JLWile which he’d still probably win. Nice as it’d be to be called “gentle”, “chicken” is better in my case!
Josiah, I would call you many things, but “chicken” in not among them. On the blog about “Jesus the Revolutionary,” you were steadfast and very courageous. You were wrong :), but you were courageous. The same can be said about your comments on the way God creates. While I think you were quite wrong, the debate was very enjoyable and ANYTHING but timid!
Wow, New Zealand, I’m insanely jealous. Is Apologia Ministries going world-wide?
The article is in Science News, not Science, and is available online.
“Not surprisingly, he found that the way the raw data were processed (which no one ever really checks or questions) can take random noise and make it look like increased blood flow. What he as an experimenter read, therefore, was just nothing but random noise. The problem is, had he not been using a dead fish, he would never have questioned the result.”
Your parenthetical statement is absolutely false. From the article: “Choosing criteria to catch real and informative brain changes, and guarding against spurious results, is one of the most important parts of an fMRI experiment, and also one of the most opaque.”
And had he not been using a dead fish, he would not have looked at the raw data without “using a statistical check to prevent random results from accidentally seeming significant.” The point of the exercise was to show how important statistical analysis is to experimental results.
The problem with fMRI, and applicable to many other areas of research, is making definitive statements based on the results. That’s where it gets into trouble. The last two paragraphs of the article is a very good statement on the use and possible abuse of fMRI.
However, your last section of bolded text goes to far. Not all scientific conclusions are very tentative. Physical and chemical laws for example, are rock solid. Neuroscience and biology conclusions a bit less so. Psychological and economic have very significant assumptions made. But it is a range, traditionally called hard to soft, not a blanket statement. So you are not helping the general public to understand scientific conclusions any better.
Finally, “being told by a bunch of scientists what to believe is much easier than actually thinking for yourself” sounds like common sense, but the connotation that you can ignore scientific consensus and interpret raw data on your own is ridiculous. You have more scientific knowledge than 99% of the US, but you’re not doing any original research. If you don’t rely on scientific consensus to inform your view of the world, then you’re doing theology, not science.
Thanks for the correction, Shooter. You are right – it is Science News. I read Science, Science News, and Chemical and Engineering News as my main “general” science sources. It is very easy to get the first two mixed up.
My books (not necessarily Apologia) HAVE been worldwide for many years. They have been used by private schools and home schools in South Africa, Canada, Mexico, Australia, New Zealand, the UK, Japan, and South Korea for many years now. In fact, there are Korean translations of my books so that they can be used by those in South Korea who are not familiar with English. As far as I know, Apologia never did any marketing in any country other than the US. However, when educators find serious science that is not propaganda, they gravitate to it. That is why my books are so popular in many, many parts of the world. I think other science authors would find a lot of success if they stopped the propaganda and just concentrated on teaching science in light of the data that we currently know.
The parenthetical statement is quite true, and once again, YOU have provided the evidence by the quote. Perhaps you should get your dictionary out and see what “opaque” means…Indeed, the whole point of the article was the fact that most fMRI experimenters DO NOT do that check.
The bold text certainly does not go too far, and any reasonable scientist understands this. Physical and chemical laws are not rock solid. I agree that they are supported by more repeatable experiments than “psychological laws,” but since experiments can be flawed, they are tentative as well. I suggest you read Karl Popper on this subject, as you do seem to have a very basic misunderstanding of what science is and how it works. Popper would help you a great deal on this subject. By letting people know what all trained scientists understand, I am certainly helping to educate them on the proper nature of science. Science is, indeed, VERY tentative.
Actually, if you rely on scientific consensus to inform your views, THAT’S when you are essentially doing theology. You are asking the “high priests” of science to tell you what to believe. That’s what uninformed people do, but it is not what rational people do. Certainly most people cannot do original research. However, any rational person can read several different sources so as to be able to see the data and make conclusions for himself or herself. I agree, it’s hard. It’s much easier to look to your high priests and simply believe whatever they tell you. However, as you have already discovered, the things your high priests tell you to believe are often not supported by the data. As a rational person, you should do the hard work and actually think for yourself.
There must’ve been a sale.
I leave The O’Flainn’s LiveJournal analysis about this fMRI Salmon of Doubt and immediately run into the article again here while checking on your Pan-Alaskan tour.
JJB
Nice Douglas Adams reference, JJ! Your wit is matched only by your broad knowledge of all things interesting.
Dr.Wile,
Thank you for visting Alaska and inspiring a homeschool mom(family) with answers and hope.
It was my pleasure, RSchmidt. Not only is your state truly lovely, but the people I met and talked with were even more amazing.
You are doing the right thing for your children, even though it is not easy!
Greetings Norwegian Shooter! I’m a frequent reader of Dr. Wile’s blog, but not necessarily a frequent commenter (at least not recently). I haven’t “taken issue” with your contributions for multiple reasons.
1. I’m not really a science person (my dear friend Dr. Wile can attest to this) and I generally try to refrain from engaging in debates concerning subjects I know very little about (something I think more people should do).
2. While I don’t consider myself “more gentle” than Dr. Wile, and certainly wouldn’t have a problem “ganging up” on you if I felt I should, I do occasionally enjoy reading the exchange between you two.
3. This isn’t my blog and therefore I don’t really feel it’s my place. In this instance, you asked… so I’m responding.
Finally, your comments (maybe less so recently, but certainly when you started commenting on this blog) were rather condescending. In many instances it was if you were actually attacking Dr. Wile for his beliefs, rather attacking the belief itself (I hope you can see the difference). I found it to be rude and distasteful. Add to that the fact that you frequently offered little to no evidence to debate the issue… at that point, what do I have to comment on?
I will agree with you on two points however, I too am very jealous of Dr. Wile’s travels. And I too am a little disappointed that there were no Palin comments. Like Dr. Wile, I am not a fan, but all the more reason for a “I was in Alaska for a week and never did see Russia!” comment.
In response to another comment… I agree with Dr. Wile, Josiah certainly does not strike me as a chicken.
Greetings as well BS (sorry, couldn’t resist). Points taken. I can see how you could take my posts as condescending, but tone on the internet is awful hard to convey. I’m shooting for amusing or at worst teasing, but I’m sure I miss often. And I would think that you’d have to say Dr. Wile has been more condescending to me lately, especially the Seeking God in Science part 2 post.
Anyone, I would be interested in a link to the “Jesus the Revolutionary” blog post.
Actually, Shooter, I am not being condescending, I am just pointing out your blunders. Now…if you didn’t post as if you knew more about science than I do, I would be a LOT more gentle in pointing out your blunders. However, so long as you take the tone you are taking, I feel no need to be gentle with you.
It’s actually called Jesus the Sinner. He likened Jesus to Che Guevara, and that’s what stuck in my head. As you can see, Josiah held his own quite nicely against me.
I think you’re right, Dr. Wile. Sometimes I’ve thought you’ve been a bit too condescending to Shooter, but really never without cause. And I appreciate how kindly you answer all of our questions. That means a lot.
I don’t mind the condescension, the projection actually bothers me more. I’ve got plenty tough skin, I just wanted to call a spade a spade.
Kyle, I guess I’m not part of “our.” I’m also curious as to what you think is proper cause to justify condescension.
Shooter, you would be a part of “our” if you actually asked questions. Instead, you act as if you are correcting me, when in fact, you are either ignoring data or making statements that are ridiculously full of errors.
If it makes you feel better, please think that I am projecting. However, anyone who reads your posts sees otherwise.
It’s not that it’s condescending, it’s that it appears to be, and otherwise would be, were it not being initiated by the other side. And like you said, attitudes are hard to judge over the internet.
Thanks, Kyle. I enjoy answering questions, so please feel free to keep on asking!