Friday, 15 August 2008

Unskilled and unaware of it

The paper Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments by Justin Kruger and David Dunning of Cornell University should be mandatory reading for all students at about year 10 level.

If more people understood that they didn't understand a particular subject then perhaps the church of global warming would have many fewer parishioners.
People tend to hold overly favorable views of their abilities in many social and intellectual domains. The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it. Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd. Several analyses linked this miscalibration to deficits in metacognitive skill, or the capacity to distinguish accuracy from error. Paradoxically, improving the skills of participants, and thus increasing their metacognitive competence, helped them recognize the limitations of their abilities.

It is one of the essential features of such incompetence that the person so afflicted is incapable of knowing that he is incompetent. To have such knowledge would already be to remedy a good portion of the offense. (Miller, 1993, p. 4) In 1995, McArthur Wheeler walked into two Pittsburgh banks and robbed them in broad daylight, with no visible attempt at disguise. He was arrested later that night, less than an hour after videotapes of him taken .from surveillance cameras were broadcast on the 11 o'clock news. When police later showed him the surveillance
tapes, Mr. Wheeler stared in incredulity. "But I wore the juice," he mumbled. Apparently, Mr. Wheeler was under the impression that rubbing one's face with lemon juice rendered it invisible to videotape cameras (Fuocco, 1996).

We bring up the unfortunate affairs of Mr. Wheeler to make three points. The first two are noncontroversial. First, in many domains in life, success and satisfaction depend on knowledge, wisdom, or savvy in knowing which rules to follow and which strategies to pursue. This is true not only for committing crimes, but also for many tasks in the social and intellectual domains, such as promoting effective leadership, raising children, constructing a solid logical argument, or designing a rigorous psychological study. Second, people differ widely in the knowledge and strategies they apply in these domains (Dunning, Meyerowitz, & Holzberg, 1989; Dunning, Perie, & Story, 1991; Story & Dunning, 1998), with varying levels of success. Some of the knowledge and theories that people apply to their actions are sound and meet with favorable results. Others, like the lemon juice hypothesis of McArthur Wheeler, are imperfect at best and wrong-headed, incompetent, or dysfunctional at worst.

Perhaps more controversial is the third point, the one that is the focus of this article. We argue that when people are incompetent in the strategies they adopt to achieve success and satisfaction, they suffer a dual burden: Not only do they reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the ability to realize it. Instead, like Mr. Wheeler, they are left with the mistaken impression that they are doing just fine. As Miller (1993) perceptively observed in the quote that opens this article, and as Charles Darwin (1871) sagely noted over a century ago, "ignorance more frequently begets confidence than does knowledge" (p. 3).

In essence, we argue that the skills that engender competence in a particular domain are often the very same skills necessary to evaluate competence in that domain—one's own or anyone else's. Because of this, incompetent individuals lack what cognitive psychologists variously term metacognition (Everson & Tobias, 1998), metamemory (Klin, Guizman, & Levine, 1997), metacomprehension (Maki, Jonas, & Kallod, 1994), or self-monitoring skills (Chi, Glaser, & Rees, 1982). These terms refer to the ability to know how well one is performing, when one is likely to be accurate in judgment, and when one is likely to be in error. For example, consider the ability to write grammatical English. The skills that enable one to construct a grammatical sentence are the same skills necessary to recognize a grammatical sentence, and thus are the same skills necessary to determine if a grammatical mistake has been made. In short, the same knowledge that underlies the ability to produce correct judgment is also the knowledge that underlies the ability to recognize correct judgment. To lack the former is to be deficient in the latter.
Do yourself a favour and read the whole thing.

(Nothing Follows)

8 comments:

Anonymous said...

If more people understood that they didn't understand a particular subject - yes, if only you would realise that you simply don't understand basic physics, let alone complex science. It would be a relief indeed if you'd shut up about things you obviously don't remotely understand how to understand.

Jack Lacton said...

Fudgie,

I have a double degree in advanced mathematics and computer science so I'm pretty sure that 1) understand the physics (which is quite basic, really) and 2) understand computer models given I work with them every day.

Anonymous said...

All your posts suggest otherwise, fucky.

Jack Lacton said...

Which posts, Fudgie?

Name one in which I have demonstrated a lack of understanding of the mathematics.

You point at Tamino as proof, quote the discredited Hansen etc and try and tell us that the Hokey Stick is statistically valid, which shows how little you understand.

Anonymous said...

Ignore the bloke Jack.

What he hasn't yet got is that the sheer nastiness and arrogance of him, and those of his persuasion, is the reason that the warmists are now losing an argument they had won two years ago.

People in general don't like arrogant w***kers. Every horrible post they make is another nail.

Best let them all keep digging, don't you think?

Jack Lacton said...

hoppers,

Fudgie appears to be a paid operative of Big Green, which his dishonesty and general obfuscation supports.

Their tactic is to spread their propaganda and paint those of us on the side of truth as the bad guys.

By letting him go unanswered the Sierra Clubs and Greenpeaces of this world achieve their goal.

Anonymous said...

In which posts have you not understood the mathematics, you ask. I presume you mean the physics. The answer is every single climate post you've ever made. Your beliefs about climate science can only be held by someone who doesn't understand the science. Obviously, the fact that you don't understand science means that you can't appreciate how badly you fail to understand the science.

Anonymous said...

So how does providing links to sites like realclimate, rehashing a few arguments you have read on the web, (anyone can do that) and being generally abusive prove that you have a greater grasp of the science than Lacton?