The Dunning–Kruger effect... how good do you think you are?

[Dan Hintz]

I came across an interesting paper in the psych journals back in early 2000ish (I'm an OCD info junkie, so I read a lot of varied stuff), shortly after it was published. Every year or two I'm reminded of this effect in my discourse with various people. I was recently reminded of it again while reading an article by Steven Johnson in Highland Woodworking's newsletter, and considering he does an excellent job summing up their research, I'll paste the bulk of the article here:


Last month's rant about the unhandyman that butchered dozens of projects in the house I am remodeling provoked some interesting responses from readers. Bruce, from Medway, MA suggested that I may have encountered the Dunning-Kruger effect, which was first identified in 1999, and which suggests that those who are unskilled often make poor decisions, but that their own incompetence denies them the ability to appreciate their mistakes, giving them the illusion of superiority and the feeling that they are "above average." Conversely, the Dunning-Kruger Effect postulates that those who are highly skilled often underrate their own abilities.

Justin Kruger and David Dunning, at the time working at Cornell University, made clear that the effect is not limited to the unskilled. Sometimes the skilled suffer from the same effect, which they identified as a "paradoxical defect in the perception of skills," in oneself and in others. They cited a 1994 study of college professors wherein 94% rated their own performance as above average when judged against their peers. As my statistics professor friend insists, it is quite impossible for 94% of any group to be above average for that same group.


As for the folks with high-level skills, it appears that some erroneously (and most probably subconsciously) assume everyone else has those same skills, thus they underrate their own on a relative basis. If you find that cutting dovetails is easy, something in your brain may say "This must be easy for everyone," thus you may underrate your own capabilities on a comparative basis.


In a later paper, Kruger and Dunning suggest that the root cause of the effect is that poor performers do not learn from feedback suggesting a need to improve.
It is interesting to note that there was some additional work done that may indicate that this is an American phenomenon. The effect is muted in Europeans and with East Asians, it appears that the opposite of the Dunning-Kruger Effect is true. East Asian people tend to underestimate their abilities, with an aim toward improving the self and getting along with others.


This is all quite interesting, but my friend (thankfully, I am quite confident, one of the 6% of college professors that has his feet firmly planted in the world of reality), suggested that external influences would have an impact on the Dunning-Kruger Effect. For example, my unhandyman may well have been receiving positive feedback from friends, family, and neighbors. Certainly it would be plausible that a beloved spouse who tells you how handy, creative, and resourceful you are could lead you to believe that your skills are, indeed, above average. That positive reinforcement might lead you to undertake more and more projects.


That input sounds logical, but highly skilled people often also receive positive feedback, and often the feedback is from others, outside the individual's circle of family and friends. This feedback might carry even more weight, yet many high-performers still consider themselves less adequate.


It is a mystery to me, but one worth exploring further. An instructor once told me that the more I learned, the more I would realize that I did not know. It may be that simple. In their first paper, Dunning and Kruger quoted Bertrand Russell: "One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision."


This should probably be tied to a poll, but I'm lazy :p

[Brian Kent]

The funny poll would be how many of us believe we are above average woodworkers and how many still believe that after reading about the Dunning -Kruger effect. Thanks for the good article and insights.

[John Pratt]

That is an interesting concept/observation. There are times when I know that I have skills above the average layman, but it only takes a look towards others in my field or other woodworkers to make me feel inadequate in my abilities. The creek is a perfect example of this. Just when you think you are gaining ground and producing some nice projects, someone will post a project, skill, or technique that puts you back on solid footing and you can reflect on how much more you need or want to learn. I think there is a direct correlation between those that seek knowledge/skills and varied opinions of those skills, and those that are content with the knowledge they have and only seek gratification from a select few around them. American Idol is a good example of people who think they are above average and are held in high regard by family and friends only to be cast aside as inadequate, while those that practice and seek further skill from outside their circle are more nervous about their abilities seem to excel.

[Stephen Tashiro]

Any person who doesn't think himself is above average in some important aspect of life is obviously suffering from low self esteem and needs our sympathy and guidance. The key to normal self esteem is to define the important aspects of life to be those areas where you excel. So , can't do math? -then take the outlook "Sure, I can't do math. But I have practical skills and I know how to write an understandable sentence". Having a little trouble with writing and fixing the car too? -then take the outlook "Sure, I can't do math or fix things or write well, but I am honest and I care for people. I'm not like the scum who run those big corporations." Having some problems in the scum department ? -you'll have to ask someone in that situation what to do, not me, of course. I'm sure there there is something else you do well.

The Kruger-Dunning effect is apparently defined for people's attitudes toward their professions. If we consider Life to be the profession, it is a nearly universal.

[Dan Hintz]

Stephen,

The effect is defined towards whatever skillset is being asked about, not just professions.

[Stephen Tashiro]

Stephen,

The effect is defined towards whatever skillset is being asked about, not just professions.

However, it is defined for a population of people who "employ" the skill set, isn't it? For example, if we asked the population of sawmillcreek about their skill in speaking Hindi, would we expect most people to rate themselves above average? - even if that only meant "above average in comparison to the other members".

[Jim Rimmer]

I have to agree with a post above that when I start thinking I'm a pretty good woodworker, someone here posts a project and I snap back to reality. As someone else said, it also has to do with knowing what you don't know. In each profession (or hobby) you have that but you also have the folks (or yourself) that don't know what they don't know. Most of the folks here at the Creek that encounter that latter category come here and ask questions.

All of my adult like (and even when I was a kid) I have worked in an industry that involves a lot of mechanical know-how. Many of those folks take that job knowledge home and fix their own cars or electronics or apply the same learning techniques to learn how to do home repairs. It makes it hard for me to realize that there a lot of people in our society that have no idea how to change a light switch, regrout tile, patch drywall, or a whole lot of other tasks that most folks at SMC take for granted. I occasionally finish a home repair or home improvement job and realize that fact. It is then that my ego gets the best of me and I remind my wife how many thousands of dollars we have saved over the years because I can do a lot of things myself. A lot of folks can't because of a lack of skills and as a result, a lack of the proper tools. So it is cheaper for them to hire it done. Just for grins, watch DIY Disaster on HGTV sometime.

[Jim Koepke]

This is interesting. There are other studies to parallel Dunning-Kruger:


What’s going on? How can we have things so wrong, and be so sure that we’re right? Part of the answer lies in the way our brains are wired. Generally, people tend to seek consistency. There is a substantial body of psychological research showing that people tend to interpret information with an eye toward reinforcing their preexisting views. If we believe something about the world, we are more likely to passively accept as truth any information that confirms our beliefs, and actively dismiss information that doesn’t. This is known as “motivated reasoning.” Whether or not the consistent information is accurate, we might accept it as fact, as confirmation of our beliefs. This makes us more confident in said beliefs, and even less likely to entertain facts that contradict them.

This clip is from an article that ran last year in the Boston Globe:

http://www.boston.com/bostonglobe/ideas/articles/2010/07/11/how_facts_backfire/?page=1

The subject matter is different, but I have seen the same thing in other fields.

I have also observed some of what goes into creating the false impressions some of my co-workers had about themselves and their work. The way some of them adjusted electronic equipment with a hammer makes me now think that maybe in a previous life they were woodworkers and were trying to set blades on old wooden planes.

Trying to get some people to change their ways is next to impossible.

Some of us are willing to try new ideas or to see if there is a better way.

Could it be that somehow "heritage and tradition" have become ingrained in our genetic make up?

jtk

[Dan Hintz]

However, it is defined for a population of people who "employ" the skill set, isn't it?
Some conditions should be obvious. What you initially suggested, however, was that the effect pertained solely to someone's profession. I'm saying the effect pertains to anyone who considers themselves to have a certain level of understanding on a subject or skill. (and in and of itself, this is only a portion of what D&K studied and concluded)

[Brian Tymchak]

It's an interesting theory that's made me stop and think a bit. While I don't reject the theory outright, without more research, I am skeptical about it's merit because, IMHO, there's a lot of subjectivity in play here. Going back to the beginning of the article that Dan posted, the terms "unskilled", "poor decisions", "incompetence", "highly skilled", etc are relative to the author's perception, and possibly not defined in either a quantitative or qualitative method. Am I as a software developer supposed to know what the univeral definition is of the "average" skill level in software development?? That definition doesn't exist. So, my thought is that maybe (likely) the measuring sticks are different between the authors and those they studied. With a group of people each defining their own measuring stick, yea, I can see how a 94% above-average response is possible.

Think about what makes a high quality box?

Is it that it has finely fitting dovetail joints, or matching grain wrapping around the sides? Or is that it holds 100 lbs of tools? All could be true or false. Depends on what your definition of "high quality" is. There could be a hundred factors that each in a group of woodworkers would use to make their own measuring stick about what a "high quality" box is.

Referring to the "the unhandyman that butchered dozens of projects in the house", I would hope that the author used some sort of filtering/interview process to get some comfort level that the handyman could and would do the work to the quality expectations of the author. but, sounds like not...

[Pat Germain]

This is interesting. I was aware of this phenomenon, but really didn't know it had a name.

Example: back when I was in school and first started dating a young lady (who is now my wife), I accompanied her to dinner at her grandparents' house. A woodworker had just delivered Grandma a custom-built hutch for her kitchen. Grandma, grandpa and other relatives were all heaping praise onto the builder of said hutch.

Now, I didn't know much of anything about woodworking back then. (And I'm certainly no expert today.) But I looked at the hutch and it sure didn't look good to me. It was built from construction grade plywood. On some areas, the manufacturer stamps were still visible. I could see the builder had cut some designs into the plywood with a router. Then he globbed on what appeared to be several, very heavy coats of Minwax gloss varnish.

I'm sure this builder believed his skills were far above average. He certainly was being validated by Grandma and everyone else around.

[Jim Koepke]

My position is that this phenomenon crosses many lines.

It is apparent in craftsmanship, or the lack of it in every trade.

It is even ingrained in people's ideas. Don't we all know someone who is often wrong on the facts but refuses to accept otherwise?

How about those who are convinced they got a great deal on some cheap tool, or other product, that is a piece of junk?

It would be hard to convince me that the unhandyman effect is limited only to hadypeople. It is the same thing with different names.

jtk

[Dan Hintz]

Pat,

He may or may not have been suffering form the effect, but without asking him to rate his skill level it's impossible to say. If you asked him and he said "I'm a skilled practitioner with all woodworking projects I sell", then yes, he would be a serious sufferer. If he instead replied "It was a quick and dirty job", then probably not.

[Lee Koepke]

Interesting thread Dan.

I find my self looking at what I do and realize a few things.
1. I am better today at what I make than I was yesterday
2. I will be better tomorrow than I am today
3. There are alot of people miles miles and miles ahead of me in both creativity and ability
4. There are alot of people not even capable of doing what I do.

Not sure where that puts me on the Dunning Scale ....

[Jim Rimmer]

This is interesting. There are other studies to parallel Dunning-Kruger:



This clip is from an article that ran last year in the Boston Globe:

http://www.boston.com/bostonglobe/ideas/articles/2010/07/11/how_facts_backfire/?page=1

The subject matter is different, but I have seen the same thing in other fields.

I have also observed some of what goes into creating the false impressions some of my co-workers had about themselves and their work. The way some of them adjusted electronic equipment with a hammer makes me now think that maybe in a previous life they were woodworkers and were trying to set blades on old wooden planes.

Trying to get some people to change their ways is next to impossible.

Some of us are willing to try new ideas or to see if there is a better way.

Could it be that somehow "heritage and tradition" have become ingrained in our genetic make up?

jtk

I think this is why some people see something on the internet or get an email that reinforces their belief and are quick to pass it on as truth without really examining its plausibility. I'm talking about condemning emails of activities of "politician that is hated today" or criminal activity warnings that when evaluated are not true.

[Mike Davis NC]

Most of the time I am not happy with the things I make, but other people tell me I do good work. I keep trying to improve and sometimes I can look back and see things that were much worse in the past. Not sure that means I'm improving though. The things in my head are so much better than the things that appear out in my shop. I guess I am about average for my skill level.

[Bill Cunningham]

Well, My wife STILL says I'm a great Lover...:D
Wait a minute!! How does 'she' know! :mad:

[Scott T Smith]

Dan, this is a very interesting topic.

Regarding the highly skilled folks that tend to under-rate their abilities, I wonder what impact that a person's ego has on how they rate themselves? An individual with a big ego may tend to think that their skills are superior, and someone without ego problems may presume that everybody else is at their same level.

I also wonder what influence that their personality type would have (standard Myers-Briggs scoring), in particular introverted versus extroverted personalities, and if this is an influence.

[David G Baker]

I try to be aware of my limitations and work within those limitations. At times I am jealous of other peoples abilities but find that instead of being jealous I can learn from them. I don't have a lot of original ideas but feel that I can do a fair job of copying others work and on occasion improve my abilities.

Dan H,
I really enjoy reading things that make me think. Thank you.

[Larry Edgerton]

Interesting article. I will be following it up a bit

I would say I am an average woodworker, and every time I have a new challange I have to build up to it. It drives me nuts that I am this way as I have a body of work that many would like to claim as their own, but still I am always fighting this demon within. I drive myself nuts with it sometimes and will get stuck on decisions because I always want things to be done the best possible way.

In an ironic twist, I get easily bored, and am always seeking something new to accomplish. I thrive on taking projects that others shy away from, only to torture myself with self doubt until they are done to my likeing. It would be easy to find a niche product and just make that, but then I would be bored. A Catch 22.....

I will read more.......

[John Christian]

I would take the view that constantly underrating your abilities is a large part of what makes a highly skilled practioner. Like one of the great taglines here.. If a fellow drilled a lot of really fine 90 foot wells and thought very highly of himself as a ninety foot well driller.. He might never drill to the hundred foot water. Regarding this theory and wood work and myself personally I have built many things large and small that people loved/liked/appreciated...... and that was a pleasant salve to all the flaws that I saw in the same work.

[Scott Shepherd]

I'd like to inject a question into the theory. I do believe the theory exists, but do you think that it's because someone actually "thinks" their work is good from day one, with no input, or do you think they present their feedback to uneducated individuals that give them false praise, causing the cycle to start, and then grow? For years I heard about some retired guy down the street from my parents. He did "beautiful" woodworking and he just made things day in and day out. All the neighborhood women said how great a craftsman he was. Then, they sold the house to a family member of mine. I began to do work on the house from time to time. What a butcher that guy was. I'm not sure I've ever seen anything as poorly engineered or executed as the work he did.

Did he think he was great because he just thought he was great or did he think he was talented because no one would tell him his work was poor, they all just praised him day and night? I think people are fearful to be honest. I've seen some poor work posted on various forums, not all woodworking related, only to be followed with "great job, that turned out excellent", rather than "Is that a gap at the end, what caused that, or here's how you avoid that in the future".

Just curious.

[Dan Hintz]

I'd like to inject a question into the theory. I do believe the theory exists, but do you think that it's because someone actually "thinks" their work is good from day one, with no input, or do you think they present their feedback to uneducated individuals that give them false praise, causing the cycle to start, and then grow?
There's certainly a lot more to the theory than what is presented at face value just by reading an abstract of the paper. Personally, I think feedback has some pull to it, but most I imagine comes from the individual's ability to judge his own work. I don't have to know much about turning a bowl on a lathe to know a bunch of tearout in the wood is a bad thing... I may not know how to correct it properly, but I definitely know it's not right. Same with putting up some crown molding... if there is continually a large gap at the end of every piece I cut, despite constant measurements, I know it's not perfect, and therefore I'm not going to consider myself a skilled craftsman. The inability to see these mistakes (i.e., someone's ability to recognize their own shortcomings) is what causes those people to have a higher sense of ability than what they are rightly entitled to.

[Larry Edgerton]

To go along with Scotts comments:

We have a contractor here that has absolutely no talent himself, hires his help from the bar, his work sucks, but........


He has a line of BS a mile long, and he usually has more work than anyone else. He drives those of us around here that are actually good at our jobs crazy. None of us have been able to figure out how he cons his way into some of these things, and then turns around and cons most of his customers into thinking they have a good house, far from the truth.

I have always wondered if he is lieing to himself, and therefore not lieing to his customers technically, or if he knows exactly what he is doing. He's one of those instantly your best buddy kind of guys, and I can't believe it flys, but people buy into it.

If I had to be like him to survive, I would be a greeter at WalMart........

[Dan Hintz]

Larry,

Those kinds of guys are almost always con artists at heart.

[Stephen Tashiro]

Maybe the whole "Why can't people write?" thread is an example of this effect.

[Larry Edgerton]

I wonder where people that use large type faces for their posts fit into this.........

One of my small annoyances, in case you haven't guessed......

[Eddie Watkins]

God makes us all just stupid enough that we think we are smart. ;) Somebody told me that a long time ago.

[Dan Hintz]

God makes us all just stupid enough that we think we are smart. ;) Somebody told me that a long time ago.
Are you sure that wasn't my wife? :p

[Mike Cruz]

There is a Creeker that references the D-K effect. Forgot who it is...

BTW, I am the best at everything I do. I mean that. Of course, what I mean, is that I am the best that I can be at that moment with those circumstances at everything I do. In other words, I alway try my best. Sometimes, my best is insufficient, though...

[Gene Howe]

Because I'm lazy and the LOML wanted it done NOW, we purchased a vanity from Lowes. I am absolutely positive that my woodworking skills FAR surpass the manufacturer of this piece of crap!
Does that observation translate to a superior evaluation of my cabinet making skills? Without question, the answer is a resounding NO!
One thing is certain however, I am an expert in avoiding work I don't want to do.

[Larry Edgerton]

One thing is certain however, I am an expert in avoiding work I don't want to do.

Me too! I do this crap for a living and I talked my wife into a pedestal sink.

Well, not actually, actually she talked me into a pedestal sink. { And with a wink and a nod, he avoids another home project]

[Ron Conlon]

There's certainly a lot more to the theory than what is presented at face value just by reading an abstract of the paper. Personally, I think feedback has some pull to it, but most I imagine comes from the individual's ability to judge his own work. I don't have to know much about turning a bowl on a lathe to know a bunch of tearout in the wood is a bad thing... I may not know how to correct it properly, but I definitely know it's not right. Same with putting up some crown molding... if there is continually a large gap at the end of every piece I cut, despite constant measurements, I know it's not perfect, and therefore I'm not going to consider myself a skilled craftsman. The inability to see these mistakes (i.e., someone's ability to recognize their own shortcomings) is what causes those people to have a higher sense of ability than what they are rightly entitled to.

I agree, Dan. I believe a necessary condition for improvement in any area is to be able to see the gap between the results of your current efforts and what you would consider "good." It may take countless hours or years of practice to attain a level that you are satisfied with, but if you are unable to distinguish the quality of the result, the prognosis for improvement is not good. There is a certain amount of bootstrapping that goes on during the improvement process, also, as the ability to discern quality improves as well.