Research Cycle

 Vol 5|No 1|October|2008
Please feel free to e-mail this article to a friend, a principal, a parent, a colleague, a teacher librarian, a college professor, a poet, a magician, a vendor, an artist, a juggler, a student, a news reporter or anyone you think might enjoy it. Other transmissions and duplications not permitted. (See copyright statement below).

The Evidence Gap, Truthiness and the Wicked Witch

By Jamie McKenzie

Many times our students will encounter claims and statements poorly substantiated by evidence.

They are contending with a broad social trend that Stephen Colbert has called "truthiness."

Merriam Webster's definitions:

1. truthiness (noun)
1 : "truth that comes from the gut, not books" (Stephen Colbert, Comedy Central's "The Colbert Report," October 2005)
2 : "the quality of preferring concepts or facts one wishes to be true, rather than concepts or facts known to be true" (American Dialect Society, January 2006)

We must develop their recognition skills so that an alarm goes off when they read or see an unsubstantiated claim, whether that claim be a biographical note, the ad for a drug or the platform of someone running for office.

Once the alarm sounds, they must learn to ask for the evidence or data that is missing.

A Biographical Example

A few years back while looking for Web evidence of Captain James Cook's navigational skill, I encountered a passage on the site of the Canadian National Library that I found astounding for its lack of evidence and its distorted history.

Many believe that Captain James Cook was the greatest ocean explorer ever to have lived, and that he mapped more of the world than any other man. It cannot be denied that he combined great qualities of seamanship, leadership and navigational skill. He mapped many parts of the Pacific, including New Zealand, Hawaii and the west coast of North America.

He also set an example of how to treat a ship, its crew, and the people he met on his explorations.
Passageways: True Tales of Adventure for Young Explorers

This passage has been altered since I lodged a complaint (removal of underlined segment above), but it remains a bold statement of opinion about a complicated man that oversimplifies his record, distorts the truth and relies upon heavy handed language to pressure us into accepting its questionable point of view. There is still an "evidence gap."

Cook routinely took hostages during his three voyages through the Pacific Islands when the native people "borrowed" his equipment or possessions. He preferred the chiefs as hostages and died in Hawaii on the day a chief refused to come along when summoned. Stabbed to death on the beach, Cook was far from a good example of how to treat the native people he encountered. His treatment of his crew is also open to question.

The previous comments are based on information from "The Death of Captain Cook" By Richard P. Aulie at and The Trial of the Cannibal Dog by Anne Salmond. One wonders if the person who wrote the passage about Cook above actually knew much about his encounters with his crew or the natives he encountered during these voyages.

A Pharmaceutical Example

The New York Times has been running a series of articles, "The Evidence Gap," examining the use of drugs that have been prescribed by thousands of doctors to millions of people despite the lack of evidence that these drugs will prolong life or reduce risks from heart attacks. In some cases, these drugs are now thought to increase the risk of cancer.

When the Food and Drug Administration approved a new type of cholesterol-lowering medicine in 2002, it did so on the basis of a handful of clinical trials covering a total of 3,900 patients. None of the patients took the medicine for more than 12 weeks, and the trials offered no evidence that it had reduced heart attacks or cardiovascular disease, the goal of any cholesterol drug. Source: "For Widely Used Drug, Question of Usefulness Is Still Lingering" By Alex Berenson Published: September 1, 2008 in the New York Times.

This series so far offers six examples of drugs that have been widely adopted without compelling evidence.

The willingness of doctors to prescribe such drugs is a serious issue, especially given the lavish attention and benefits provided to many of them by the drug companies. See article on these benefits "MEDICAL MARKETING -- Treatment by Incentive: As Doctor Writes Prescription, Drug Company Writes a Check." June 27, 2004 - By Gardiner Harris in the New York Times.

At the heart of the various investigations into drug industry marketing is the question of whether drug companies are persuading doctors -- -- often through payoffs -- to prescribe drugs that patients do not need or should not use or for which there may be cheaper alternatives.

But the willingness of patients to accept these drugs without checking doctors' advice is an important aspect of the problem. A patient, sadly, must understand that the old saying "Buyer Beware!" now applies to the medical field in which it must be restated, "Patient Beware!"

After a decade of urging seniors to take flu shots, new studies are challenging the design of the original studies that seemed to support the benefits of such shots. Evidently, the seniors in those studies who took shots were generally better off to begin with, were characterized by healthier habits and received care superior to that available to those not taking the shots, so their survival rate may have been explained by factors other than the shots.

The influenza vaccine, which has been strongly recommended for people over 65 for more than four decades, is losing its reputation as an effective way to ward off the virus in the elderly.

A growing number of immunologists and epidemiologists say the vaccine probably does not work very well for people over 70, the group that accounts for three-fourths of all flu deaths.
"Doubts Grow Over Flu Vaccine in Elderly." Published: September 1, 2008 By Brenda Goodman in the New York Times.

A Political Example

Modern campaigns often rely upon sound bites, mind bites, eye candy and mind candy to sell candidates, whether they be running on the left or the right. Much campaigning is done with 15 and 30 second campaign ads that compress complicated issues into simple one line solutions.

It seems that few citizens have the time or the inclination to visit the candidates' Web sites where they would usually find lengthy white papers outlining the candidates' positions on the issues of the day. The thirst for simple, brief answers to complicated problems is met by simplistic solutions that often make little sense. Combined with voters' general dislike of pain and sacrifice, the policy options offered by hopeful candidates rarely involve either.

The current issue of high gas prices is a good example. Voters see this as a crucial issue in most countries, but politicians have a difficult time selling conservation as a viable policy option. In the USA presidential election, both major party candidates who previously opposed offshore drilling have shifted their stances as polls have shown citizens want more oil now and want more drilling now, even though most experts have testified that it might take 6-8 years or longer before any of this new drilling would bring more oil to market and even though the oil involved might be a "drop in the bucket."

"Suppose the US produced all its oil domestically," said Robert Kaufmann, director of the Center for Energy and Environmental Studies at Boston University. "Do you think oil companies would sell oil to US consumers for one cent less than they could get from French consumers? No. Where oil comes from has no effect on price." Source: "New offshore drilling not a quick fix, analysts say." June 20, 2008 in the Boston Globe, by Lisa Wangsness.

According to the experts quoted in the above article, "about 86 billion barrels of additional oil may lie offshore, according to the US government's Energy Information Administration. Of that amount, about 18 billion barrels are subject to the moratorium. Much of the rest lies in areas that are too expensive to exploit or that oil companies have not yet tapped for technical reasons, fueling the industry's desire for fresh territory."

Furthermore, the article claims, "It would take at least a decade for oil companies to obtain permits, procure equipment, and do the exploration necessary to get the oil out of the ground, most industry analysts say."

Both Obama and McCain now have energy policies that seek independence from foreign sources over the next decade, trusting to new alternatives but suggesting little in the way of conservation or sacrifice. It is interesting to compare their comments about energy in their acceptance speeches:

Senator Obama's Comments about Energy Senator McCain's Comments about Energy

And for the sake of our economy, our security, and the future of our planet, I will set a clear goal as president: In 10 years, we will finally end our dependence on oil from the Middle East.


We will do this. Washington -- Washington has been talking about our oil addiction for the last 30 years. And, by the way, John McCain has been there for 26 of them.


And in that time, he has said no to higher fuel-efficiency standards for cars, no to investments in renewable energy, no to renewable fuels. And today, we import triple the amount of oil than we had on the day that Senator McCain took office.

Now is the time to end this addiction and to understand that drilling is a stop-gap measure, not a long-term solution, not even close.


As president, as president, I will tap our natural gas reserves, invest in clean coal technology, and find ways to safely harness nuclear power. I'll help our auto companies re-tool, so that the fuel-efficient cars of the future are built right here in America.

OBAMA: And I'll invest $150 billion over the next decade in affordable, renewable sources of energy -- wind power, and solar power, and the next generation of biofuels -- an investment that will lead to new industries and 5 million new jobs that pay well and can't be outsourced.

My fellow Americans, when I’m President, we’re going to embark on the most ambitious national project in decades. We are going to stop sending $700 billion a year to countries that don’t like us very much. We will attack the problem on every front. We will produce more energy at home. We will drill new wells offshore, and we’ll drill them now. We will build more nuclear power plants. We will develop clean coal technology. We will increase the use of wind, tide, solar and natural gas. We will encourage the development and use of flex fuel, hybrid and electric automobiles.

Senator Obama thinks we can achieve energy independence without more drilling and without more nuclear power. But Americans know better than that. We must use all resources and develop all technologies necessary to rescue our economy from the damage caused by rising oil prices and to restore the health of our planet. It’s an ambitious plan, but Americans are ambitious by nature, and we have faced greater challenges. It’s time for us to show the world again how Americans lead.

This great national cause will create millions of new jobs, many in industries that will be the engine of our future prosperity; jobs that will be there when your children enter the workforce.

When their ideas are placed side by side, it is difficult to distinguish between them, in part because their positions are thin on specifics, another version of the "evidence gap" that is the focus of this article. What is missing from their words are clear facts about their track records, other than the attacks made by their opponents.

Obama attacks McCain for his lack of support for higher fuel-efficiency standards for cars, investments in renewable energy, and renewable fuels.

McCain attacks Obama for his lack of support for offshore drilling.

Numbers Do Not Lie, But Some People Lie with Numbers

Obama promises to spend "$150 billion over the next decade in affordable, renewable sources of energy," but the number, while impressive, is abstract and difficult to grasp. McCain counters with talk about an ambitious project that will, among many other things, ". . . increase the use of wind, tide, solar and natural gas. We will encourage the development and use of flex fuel, hybrid and electric automobiles." He not does explain how these things will happen. He does not provide specifics or details, but neither does Obama.

To be fair to both candidates, an acceptance speech is expected to be inspirational - a dramatic and moving piece of oratory that cannot be weighted down with too much in the way of specifics or evidence without endangering the opportunity to "rally the base." Appeals to emotion seem strategically wise, while sharp edged specifics would seem risky.

Simple ideas, neatly packaged, have great appeal in such speeches. Thus we heard avid party loyalists chanting "Drill, Baby, Drill!" as the McCain outlined his position.

Big numbers like Obama's promise of a $150 billion investment in alternative energy sources are impressive but unnerving. They resonate. They impress. But they also inspire skepticism. Where will he find this money? If we go to his campaign Web site looking for specifics, we do not find them, even if we download the full text of "NEW ENERGY FOR AMERICA" at It is a big number, for sure, but there is no place in the four page document that clarifies how it will be spent or how it will be funded. There seems to be an evidence gap.

McCain also threw around some big numbers, promising that his plan would help us, "stop sending $700 billion a year to countries that don’t like us very much." Astute commentators quickly pointed out that the number he used is the total trade deficit, not the trade deficit for oil, and that much of the number was spent on goods produced by so-called friends.

There seems to be an evidence gap.

CBS News reported on September 11, 2008 that, "The big rise in oil prices, with the average barrel of imported crude jumping to a record $124.66, pushed overall imports up by 3.9 percent to a record $230.3 billion." "U.S. Trade Deficit Soars; Oil Imports Rise." "Countries that don’t like us very much," is a clever and emotionally appealing sentiment, but it clouds over the facts. Much of the oil we import comes from Canada and friends, as explained by the U.S. Government Web site, The Energy Information Administration.

About Half of U.S. Petroleum Imports Come from the Western Hemisphere

Some may be surprised to learn that almost 50% of U.S. crude oil and petroleum products imports came from the Western Hemisphere (North, South, and Central America and the Caribbean including U.S. territories) during 2006. We imported only 16% of our crude oil and petroleum products from the Persian Gulf countries of Bahrain, Iraq, Kuwait, Qatar, Saudi Arabia, and United Arab Emirates. During 2007, our five biggest suppliers of crude oil and petroleum products were:

* Canada (18.2%)
* Mexico (11.4%)
* Saudi Arabia (11.0%)
* Venezuela (10.1%)
* Nigeria (8.4%)

It turns out that the biggest problem is our dependence on oil, not our dependence on oil from "countries that don’t like us very much." Once we close the "evidence gap," a very different picture emerges than the one McCain hoped to evoke.


When politicians throw around large numbers or purport to believe in things they have never acted upon, when they wish to be seen green when they are really gray, or when they hope to be seen as well informed when they are really ignorant, they are flirting with what Stephen Colbert has called "truthiness."

Stephen Colbert defines truthiness as "The quality by which one purports to know something emotionally or instinctively, without regard to evidence or intellectual examination".

When students and citizens learn to poke below the surface claims and demand evidence to substantiate claims and positions, truthiness melts away like the wicked witch (image) in The Wonderful Wizard of Oz. Closing the evidence gap is like throwing water on the witch.


Copyright Policy: Materials published in The Question Mark may be duplicated in hard copy format if unchanged in format and content for educational, nonprofit school district and university use only and may also be sent from person to person by email. This copyright statement must be included. All other uses, transmissions and duplications are prohibited unless permission is granted expressly. Showing these pages remotely through frames is not permitted.

FNO Press is applying for formal copyright registration for articles. Unauthorized abridgements are illegal.