Psychologists Dispute Dunning-Kruger Rule as Raw Data Shows Pure Noise

Dunning Kruger Effect Wrong, Scientists Claim — Sabine Hossenfelder

Sabine Hossenfelder

Both the classic Dunning-Kruger effect and its latest rebuttal fail because researchers fit arbitrary linear regressions to uncorrelated statistical noise from tests that measure learned knowledge rather than innate intelligence.

AI-assisted summary of Dunning Kruger Effect Wrong, Scientists Claim — original source. Check the original for full context.

Source published:

Key ideas

  • A new reanalysis of the largest available Dunning-Kruger dataset challenges the classic finding by flipping the regression axis, concluding that higher confidence reliably correlates with higher actual performance. (2:00)
  • The original 1999 study by Kruger and Dunning found that bottom-quartile performers ranked around the 12th percentile but estimated their performance at the 62nd percentile. (0:00)
  • Sabine Hossenfelder criticizes both the original research and recent rebuttals, rating them a '10 out of 10 on the bullshit meter' for imposing arbitrary linear fits onto fundamentally uncorrelated scatter plots. (4:39)
  • Training low performers in logic and reasoning improved both their actual scores and error recognition, indicating the underlying tests evaluate acquired knowledge rather than innate intelligence. (4:39)

Detailed notes

## Background This video was published on YouTube by Sabine Hossenfelder on August 11, 2026. The description notes that while the Dunning-Kruger effect has long been used to explain incompetent politicians and social media behavior, a reanalysis of the largest available dataset reached the opposite conclusion, prompting an examination into how both sides can view the same data and see conflicting results.

## TL;DR Sabine examines the foundational 1999 Kruger and Dunning study, which suggested that low performers severely overestimate their competence because they lack the metacognitive skills to evaluate their own abilities. She contrasts this traditional view with a recent paper that flips the regression analysis on a large 2021 dataset, demonstrating that confidence actually correlates positively with real performance. Ultimately, Sabine dismisses both interpretations as poor data analysis, arguing that researchers are merely fitting arbitrary lines through an uncorrelated cloud of noise. She concludes that these tests measure acquired knowledge rather than innate intelligence, remaining culturally popular primarily because people love using the effect as an insult against others.

## Atmosphere & style The presentation is sharp, wry, and punchy, driven by Sabine's signature brand of blunt scientific skepticism. The mood shifts from an entertaining recap of pop-psychology lore—complete with lemon juice bank robbers and John Cleese quips—into a rigorous, no-nonsense critique of statistical methodology. It feels engagingly irreverent and intellectually cathartic as she calls out flawed research practices, maxing out her 'bullshit meter' to cut through cultural confirmation bias.

## Bulletin - [02:00](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=120s) A new reanalysis of the largest available Dunning-Kruger dataset challenges the classic finding by flipping the regression axis, concluding that higher confidence reliably correlates with higher actual performance. - [00:00](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=0s) The original 1999 study by Kruger and Dunning found that bottom-quartile performers ranked around the 12th percentile but estimated their performance at the 62nd percentile. - [04:39](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=279s) Sabine Hossenfelder criticizes both the original research and recent rebuttals, rating them a '10 out of 10 on the bullshit meter' for imposing arbitrary linear fits onto fundamentally uncorrelated scatter plots. - [04:39](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=279s) Training low performers in logic and reasoning improved both their actual scores and error recognition, indicating the underlying tests evaluate acquired knowledge rather than innate intelligence.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=0s) The classic Dunning-Kruger effect posits that incompetent individuals lack the metacognitive capacity to recognize their own deficiencies, leading to inflated self-assessments. - [00:00](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=0s) David Dunning and Justin Kruger published their foundational paper in 1999, inspired by bank robber MacArthur Wheeler's belief that lemon juice would conceal his identity from security cameras. - [00:00](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=0s) In the original experiments across humor, grammar, and logic, participants in the bottom quartile severely overestimated their ability, scoring at the 12th percentile while predicting the 62nd. - [02:00](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=120s) The traditional Dunning-Kruger regression line minimizes prediction variance relative to actual test scores, generating the classic pattern of low-performer overestimation. - [02:00](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=120s) The authors of a new reanalysis argue that self-concept is stable while test scores fluctuate with daily noise, justifying a regression that minimizes score variance relative to predicted ability. - [02:00](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=120s) Flipping the regression analysis leads to the opposite conclusion: confidence positively predicts actual scores, and the gap where individuals underperform their predictions widens as ability increases. - [02:00](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=120s) Averaging both regression lines yields a third conflicting outcome, where low scorers underrate themselves and high scorers overrate themselves. - [04:39](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=279s) From a data analysis perspective, the underlying raw scatter plot is effectively an unstructured cloud of noise where fitting any straight line is scientifically questionable. - [04:39](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=279s) Because targeted training rapidly closes the performance and self-evaluation gap, the test batteries reflect acquired knowledge rather than raw cognitive ability. - [04:39](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=279s) The enduring cultural popularity of the Dunning-Kruger effect is driven by its utility as a social weapon and outrage bait, as observers readily apply it to adversaries while exempting themselves.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=0s) The Origins and Cultural Impact of the Dunning-Kruger Effect

The popular understanding of the Dunning-Kruger effect can be summed up simply: the dumber someone is, the more certain they are of their own brilliance. Over the years, this psychological phenomenon has become a universal catch-all to diagnose incompetent politicians, explain bizarre internet behavior, and lament the broader decline of social media. The effect was originally introduced in 1999 by psychologists Justin Kruger and David Dunning. Their seminal paper famously kicked off with the bizarre true story of MacArthur Wheeler, an inept bank robber who rubbed lemon juice across his face under the sincere conviction that chemical properties similar to invisible ink would render him completely invisible to security cameras.

To test whether incompetence truly breeds blind overconfidence, Kruger and Dunning evaluated university students across tests in humor, grammar, and logical reasoning, asking each participant to estimate how well they believed they had performed. The results appeared stark: participants who ranked in the bottom quartile scored around the 12th percentile on average, yet confidently estimated their performance to be around the 62nd percentile. Kruger and Dunning concluded that the very metacognitive skills required to generate a correct answer are identical to the skills needed to recognize poor performance. The duo was awarded an Ig Nobel Prize in psychology in 2000, and their finding quickly penetrated the broader cultural zeitgeist. As comedian John Cleese famously summarized when discussing his friendship with David Dunning: “Which means, and this is terribly funny, that if you're absolutely no good at something at all, then you lack exactly the skills that you need to know that you're absolutely no good at it.” Cleese wryly added that this insight neatly accounted for “not just Hollywood, but almost the entirety of Fox News.”

- **Original Authors & Year:** David Dunning and Justin Kruger published their foundational paper in 1999. - **Inspiration:** MacArthur Wheeler's 1995 lemon juice bank robberies in Pittsburgh. - **Tested Skills:** University undergraduate performance across humor, grammar, and logical reasoning. - **Baseline Discrepancy:** Bottom-quartile performers placed around the 12th percentile in reality, but self-estimated their skill at the 62nd percentile. - **Recognition:** Awarded the Ig Nobel Prize in psychology in 2000.

[02:00](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=120s) Reanalyzing the Data: Conflicting Regression Lines

Despite decades of widespread acceptance, authors of a new paper reanalyzed the largest available dataset on the subject—drawn from a 2021 study involving logic and grammar questions alongside self-estimated scores—and reached the exact opposite conclusion. When the raw data is plotted as a scatter plot of pale blue dots, the horizontal axis represents the test taker's actual score, the vertical axis tracks their self-predicted score, and a diagonal dashed blue line marks perfect calibration. The conventional Dunning-Kruger line is derived from a linear regression that minimizes the deviation of the prediction (asking: for all individuals who earned an actual score of, say, 5, what did they predict on average?). This standard fit produces an orange line showing that low performers substantially overestimate their ability, while high performers slightly underestimate themselves.

```mermaid flowchart TD accTitle: Regression Methods on Ability Versus Prediction Data accDescr: Flowchart contrasting the traditional Dunning-Kruger regression, the new paper's regression, and the mean regression. subgraph Data["2021 Grammar & Logic Dataset"] Dots["Scatter Plot: Actual Score (X) vs Predicted Score (Y)"] end

subgraph Approaches["Regression Perspectives"] DK["Standard Dunning-Kruger<br/>Regress Y on X (Minimize Prediction Error)"] NewStudy["New Reanalysis<br/>Regress X on Y (Minimize Score Error)"] MeanReg["Mean Fit<br/>Average of Both Fits"] end

subgraph Conclusions["Resulting Interpretation"] DK_Out["Low performers overestimate;<br/>High performers underestimate."] New_Out["Confidence predicts score;<br/>Underperformance grows with ability."] Mean_Out["Low scorers underrate;<br/>High scorers overrate."] end

Dots --> DK Dots --> NewStudy Dots --> MeanReg

DK --> DK_Out NewStudy --> New_Out MeanReg --> Mean_Out ```

However, the authors of the new study argue that traditional research asks the wrong question entirely. Instead of conditioning on actual scores, they argue researchers should ask: among everyone who predicted a specific score, how did they actually perform? The justification is that a person's underlying self-concept is relatively stable, whereas their actual test score fluctuates based on incidental noise such as poor sleep, what they ate the night before, their clothing, or whether it happens to be raining. Flipping the regression to minimize the deviation of the actual score yields a green line with an entirely different meaning: people generally score lower than they anticipate, the gap widens the higher their performance climbs, and, crucially, higher confidence reliably tracks with higher actual ability. When Sabine evaluates this shift, she highlights how contradictory models emerge from identical dots: “And even if you leave that aside, it's still the case that the more confident people are, the better they actually score.”

- **The Dataset:** Derived from a 2021 replication examining grammar and logical reasoning scores alongside participants' expected performance. - **Traditional Line (Orange):** Minimizes prediction variance (vertical distance relative to the horizontal actual score), suggesting low performers wildly inflate their competence. - **New Paper's Line (Green):** Minimizes actual score variance (horizontal distance relative to the vertical prediction), suggesting self-confidence is an accurate, positive predictor of actual achievement. - **Mean Regression Line:** Averaging both regression orientations produces an alternative line indicating that low performers underrate themselves while high performers overrate themselves.

[04:39](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=279s) Crappy Data Analysis and What the Tests Actually Measure

Stepping back from the statistical debate, Sabine offers a far more skeptical assessment of the entire dispute. Looking at the raw scatter plot from a physicist's perspective reveals an unstructured cloud devoid of any meaningful correlation. When data resembles random noise, fitting straight lines to it is fundamentally flawed methodology. Expressing her frustration with the literature, Sabine states: “Why the hell are you fitting any straight line to this? This is why I give this paper as well as all previous papers on the topic a 10 out of 10 on the bullshit meter.” In her view, researchers are simply drawing arbitrary linear regressions through noise until one produces the satisfying narrative that everyone else is an idiot.

A far more revealing insight from the original Kruger and Dunning research is that when poor performers were trained in logic and reasoning, their test scores improved, and they simultaneously developed the ability to spot their previous errors. As Sabine argues: “To me, this means that these tests don't actually measure intelligence. They measure knowledge.” Furthermore, the social reality of the effect contaminates future study: as awareness of Dunning-Kruger spreads, self-aware participants inevitably adjust their predictions. Ultimately, the cultural staying power of the effect relies on its utility as outrage bait and social weapon, because the most appealing aspect of the Dunning-Kruger effect is that people instantly diagnose it in everyone other than themselves.

Closing the episode, Sabine transitions to the video's sponsor, Henson Shaving. The team behind Henson brings backgrounds in aerospace manufacturing, having previously produced precision components for the International Space Station and the ExoMars rover. They bring this exact engineering rigor to razor design, positioning the blade at the precise geometric angle required to slice facial hair flush without cutting the skin. Rather than using molded plastic parts, their handles and heads are machined from metal to tolerances of one-hundredth of a millimeter (0.01 mm) to prevent blade chatter. Sabine notes that she prefers her husband cleanly shaven without nicks, jokes that the razor works equally well for women, and shares an offer providing 100 free replacement blades with the purchase of a razor using her promotional link.

- **Data Critique:** Sabine rates both the new paper and previous Dunning-Kruger papers a 10 out of 10 on the "bullshit meter" for fitting linear trends to uncorrelated scatter clouds. - **Knowledge vs. Intelligence:** Training participants in specific analytical techniques repaired both performance and metacognition, demonstrating the task measures learned information rather than baseline cognitive capacity. - **Observer Bias:** The psychological durability of Dunning-Kruger stems from its utility as an interpersonal insult that people readily attribute to political opponents and internet strangers. - **Sponsor Specifications (Henson Shaving):** - Aerospace manufacturing pedigree (ISS and ExoMars rover components). - Metal construction machined to tolerances of 0.01 mm. - Promotional offer includes 100 free blades when purchasing via the creator's link.

## Notable quotes > [02:00](https://www.youtube.com/watch?v=0XcEsuFs5Y8&t=120s) the more confident people are, the better they actually score. — Sabine Hossenfelder

## Mentioned David Dunning, Justin Kruger, MacArthur Wheeler, John Cleese, Fox News, Henson Shaving, International Space Station, ExoMars rover, Ig Nobel Prize, Sabine Hossenfelder

## Video description Upgrade Your Shaving Experience With Henson (100 Free Blades): https://hensonshaving.com/sabine

The dumber you are, the more certain you are that you aren't. That's the Dunning-Kruger effect, and it's been used to explain everything from bad politicians to the state of social media. Now psychologists have gone back to the biggest dataset we have and come out with the opposite conclusion. But there's a reason both sides can look at the same dots and see the opposite thing. Let's take a look.

Paper: https://doi.org/10.1037/rev0000644

#science #sciencenews #psychology #research

👕T-shirts, mugs, posters and more: ➜ https://sabines-store.dashery.com/ 💌 Support me on Donorbox ➜ https://donorbox.org/swtg 👉 Transcript with links to references on Patreon ➜ https://www.patreon.com/Sabine 📝 Transcripts and written news on Substack ➜ https://sciencewtg.substack.com/ 📩 Free weekly science newsletter ➜ https://sabinehossenfelder.com/newsletter/ 👂 Audio only podcast ➜ https://open.spotify.com/show/0MkNfXlKnMPEUMEeKQYmYC 🔗 Join this channel to get access to perks ➜ https://www.youtube.com/channel/UC1yNl2E66ZzKApQdRuTQ4tw/join 📚 Buy my book ➜ https://amzn.to/3HSAWJW

[Source](https://www.youtube.com/watch?v=0XcEsuFs5Y8)

中文简报

心理学界质疑达克效应：原始数据实为纯粹噪声 经典的达克效应及其最新反驳均站不住脚，因为研究人员将任意的线性回归强加于毫无关联的统计噪声之上，而相关测试衡量的只是后天学到的知识，而非先天智力。 Sabine审视了Kruger和Dunning于1999年开展的基础研究，该研究认为低水平者因缺乏评估自身能力的元认知技能，从而严重高估自己的胜任力。她将这种传统观点与近期一篇论文进行了对比；该论文对2021年的一个大型数据集调换了回归分析的轴向，证明自信心实际上与实际表现呈正相关。然而Sabine最终认为这两种解读都属于拙劣的数据分析，指出研究人员不过是在毫无关联的噪声散点云中强行拟合任意线条。她总结道，这些测试衡量的是后天习得的知识而非先天智力，之所以在文化中经久不衰，主要是因为人们乐于用这种效应来贬低他人。 对目前最大规模达克效应数据集的一项新重新分析通过调换回归轴，对经典结论提出了挑战，并得出结论认为更高的自信心与更优的实际表现可靠正相关。 (2:00) Kruger和Dunning于1999年进行的最初研究发现，处于表现最差四分之一区间的人群实际排名约为第12百分位数，但他们自估的表现却在第62百分位数。 (0:00) Sabine Hossenfelder对最初的研究和近期的反驳均提出批评，认为二者在根本不存在关联的散点图上强行进行任意线性拟合，在“胡扯指数”上可打“满分10分”。 (4:39) 对低水平者进行逻辑与推理培训不仅提高了他们的实际得分，也提升了其辨识错误的能力，这表明底层测试评估的是后天习得的知识，而非先天智力。 (4:39)

Reading summary

While the widely cited Dunning-Kruger effect claims that incompetent individuals overestimate their ability, recent reanalyses demonstrate that the effect is an artifact of regression choices on noisy data rather than a proven cognitive deficit, according to Sabine Hossenfelder. The 1999 Hypothesis and Cultural Rise (0:00) The Dunning-Kruger effect originated in a 1999 study by psychologists Justin Kruger and David Dunning, who sought to test whether incompetence obscures self-awareness. Testing university students across grammar, logic, and humor, they found that participants in the bottom quartile placed around the 12th percentile in reality but estimated their performance around the 62nd percentile. The authors concluded that the metacognitive skills required to generate correct solutions are the exact skills required to evaluate one's own competence. The findings, which won an Ig Nobel Prize in 2000, quickly became a universal cultural shorthand. It was popularized by figures like comedian John Cleese to dismiss political opponents, online commentary, and media figures as people lacking the capacity to recognize their own incompetence. Statistical Sensitivity and Contradictory Regressions (2:00) A reanalysis of the largest available replication dataset from 2021 indicates that the traditional conclusion depends on arbitrary statistical framing. The classic Dunning-Kruger curve relies on regressing predicted scores onto actual scores, minimizing prediction variance to show low scorers dramatically overrating themselves. However, the authors of the new paper argued that because self-concept is relatively stable while actual test performance varies due to transient factors like sleep, diet, or weather, researchers should instead regress actual scores onto predicted scores. Minimizing actual score variance produces the reverse conclusion: self-confidence positively correlates with actual achievement, people generally score lower than anticipated, and underperformance relative to expectations widens among higher scorers. Averaging both regression orientations produces yet another conflicting model where low performers underestimate and high performers overestimate themselves, showing that contradictory narratives can be derived from the exact same scatter plot. Methodological Flaws and Learned Knowledge (4:39) From a physical sciences perspective, Hossenfelder argues that both the original literature and the new paper suffer from fundamental methodological flaws. The underlying scatter plots resemble unstructured clouds of random noise, making any attempt to fit straight lines questionable. Hossenfelder assigns the literature a 10 out of 10 on her 'bullshit meter,' arguing that researchers are merely imposing linear narratives onto uncorrelated data. Furthermore, Hossenfelder highlights that Kruger and Dunning's own experiments showed that training poor performers in logic not only raised their scores but immediately enabled them to recognize their past mistakes. This indicates that these evaluations measure acquired knowledge rather than fixed cognitive traits or general intelligence. Hossenfelder concludes that the enduring popularity of the Dunning-Kruger effect relies on its utility as an interpersonal weapon and outrage bait, as observers readily attribute it to others while exempting themselves. Additionally, growing public awareness of the effect contaminates future psychological research by altering how participants self-report their expectations.

精炼阅读摘要

物理学者Sabine Hossenfelder指出，公众熟知的“越无知越自信”达克效应（Dunning-Kruger effect）缺乏严谨统计支持；最新论文表明不同回归方法会得出截然相反的结论，而原始数据本质上只是缺乏关联的噪点。 达克效应的起源与大众文化传播 (0:00) 达克效应最早由心理学者Justin Kruger与David Dunning于1999年提出，其灵感源自1995年MacArthur Wheeler将柠檬汁涂在脸上试图隐形抢劫银行的奇特案例。两位研究人员对大学生的幽默感、语法和逻辑推理能力进行测试，并让受试者估算自己的表现。结果显示，得分处于后四分之一（平均位于第12百分位数）的学生，自我评估却平均达到第62百分位数。Kruger与Dunning据此得出结论：缺乏产生正确答案技能的人，同样缺乏识别自身表现拙劣的能力。这项研究于2000年获得搞笑诺贝尔心理学奖。此后，这一概念在公众文化中迅速普及，喜剧演员John Cleese等人将其概括为“一个人若能力极差，便恰恰缺乏认清自身无能所需的技能”，使该效应逐渐演变成大众解释政客无能、网络怪象以及各类愚蠢行为的常用标签。 统计回归视角的翻转与矛盾结论 (2:00) 尽管达克效应被广泛接纳，一项针对2021年语法与逻辑题大型数据集（包含实际得分与自评得分）的重新分析却得出了相反结论。在以实际得分为横轴、预测得分为纵轴的散点图上，传统达克效应采用最小化预测误差的线性回归（按实际表现分组看平均预测），得出低分者大幅高估自身、高分者轻微低估自身的经典橙色回归线。然而新研究作者认为，受试者的自我认知相对稳定，而实际得分易受睡眠、饮食、着装或天气等偶然噪音干扰，因此应当最小化实际得分误差（按自评预测分组看实际表现）。采用这种翻转后的回归方式时，数据拟合出的绿色回归线表明：受试者实际得分普遍低于预期，表现越高落差越大，且高自信度反而稳定对应着更高的实际能力。若取两种回归方式的均值，甚至会出现低分者低估自身、高分者高估自身的新结果，显示出同一批散点仅凭回归视角的切换就会导出完全矛盾的模型。 数据本质质疑与测量属性反思 (4:39) Sabine从物理学家的视角对这场统计争论提出了更为尖锐的质疑。她认为该数据集的原始散点图本质上是一团没有明确关联的随机噪点，在根本不存在线性相关的数据上强行拟合直线属于糟糕的方法论，并因此给新论文及以往所有相关论文在“胡扯指数”上评出10分满分。她指出，研究人员实际上只是在随机噪音中肆意划线，以迎合“别人都是笨蛋”这一迎合大众的叙事。此外，原初Kruger与Dunning研究中的一个关键细节表明，当低分受试者接受逻辑与推理训练后，其测试表现不仅提高，也同时掌握了识别自身错误的能力。Sabine认为这证明此类测试衡量的根本不是先天的智力，而是后天习得的知识。同时，随着达克效应概念本身的广泛传播，具有自我意识的受试者会主动调整预期，进一步污染了后续实证研究的客观性。 Sabine认为达克效应之所以历久不衰，并非源于其科学严谨性，而是因为它成了极其便利的社交攻击武器和引发公众情绪的诱饵；人们热衷于引用该效应，本质上是因为每个人都极易将其诊断在他人身上，却从不适用于自己。

Source quotes

[2:00] Sabine Hossenfelder: the more confident people are, the better they actually score.

Original transcript

[0:00] The dumber people are, the more confident they  are that they're not dumb. This, in a nutshell, [0:06] is the Dunning Krueger effect, which has been used  to explain everything from incompetent politicians [0:12] to the demise of social media. But is it actually  true? The authors of a new paper have reanalyzed [0:19] the data and they say we've got it all wrong.  Actually, confidence is a good predictor for [0:25] actual performance. Let's have a look. The Dunning  Krueger effect was named after psychologists [0:31] Justin Krueger and David Dunning who reported it  first in 1999. Their paper opens with the example [0:38] of MacArthur Wheeler, a bank robber who rubbed  lemon juice on his face because he believed it [0:44] would make him invisible to security cameras.  Dying and Krueger tested university students [0:50] on questions in humor, grammar, and logical  reasoning. The participants then also had to [0:55] estimate how well they did. Those in the bottom  quarter performed around the 12th percentile [1:01] on average, but believe they were near the 62nd  percentile. Krueger and Dunning therefore argued [1:08] that the skills needed to perform well were also  needed to recognize poor performance. In 2000, [1:14] they won the IG Nobel Prize in psychology.  The Dunning Krueger effect has since been [1:20] independently reproduced a few times and has  entered pop culture. The best summary probably [1:25] comes from John Cle. There's a a wonderful bit of  research by a guy called David Dunning at Cornell [1:31] who's a friend of mine, I'm proud to say, who's  pointed out that in order to know how good you [1:36] are at something requires exactly the same skills  as it does to be good at that thing in the first [1:41] place. Which means, and this is terribly funny,  that if you're absolutely no good at something [1:47] at all, then you lack exactly the skills that  you need to know that you're absolutely no good [1:54] at it. And this explains not just Hollywood, but  almost the entirety of Fox News. The authors of [2:01] the new paper now used the biggest data set we  have on the Dunning Krueger effect which comes [2:06] from a 2021 study that also used a set of grammar  and logical questions and then asked participants [2:12] to guess their own score. Let me just show you  the data. I'll use the grammar questions but [2:18] the data set for the logic questions is very very  similar. The data is this cloud of pale blue dots. [2:25] On the horizontal axis you see the actual score  and on the vertical axis you see the score that [2:31] people predicted. The diagonal blue dashed line  that would be a perfect correlation if everyone [2:37] correctly predicted their score. The usual Dunning  Krueger effect shows up in this data by making a [2:43] linear regression that minimizes the deviation  of the prediction. That's the orange line for [2:49] the standard Dunning Krueger effect. One basically  asks if I look at everyone who scored say five, [2:55] what was their average prediction? What you see  is that the people who had a low score on average [3:00] overestimated how well they did whereas the people  who had a high score on average underestimated [3:06] themselves. The authors of the new paper now say  look this is doing it wrong. We're asking the [3:13] wrong question. We should instead ask how did  the people who predicted a certain performance [3:19] say five points actually perform. This is a more  reasonable question because people's expectation [3:25] of themselves is less variable than their actual  performance which might depend on you know what [3:30] they ate last night, what shirt they wear, if  it's raining today and so on. In this case, [3:35] you make a linear regression that minimizes the  deviation of the score. And what you get then is [3:41] this green line. And this green line tells  a completely different story. It says that [3:46] people on average score less than they thought  they would. However, the discrepancy between [3:52] prediction and score gets larger, the better the  performance. And even if you leave that aside, [3:58] it's still the case that the more confident  people are, the better they actually score. Okay, [4:04] interesting you may say. But wait, how can it be  that there are two different best fits to the same [4:11] data cloud? It's because each of these curves  is best in a different way. Loosely speaking, [4:17] for the standard Dunning Krueger, you minimize  the deviation into the horizontal direction for [4:23] the new paper in the vertical direction.  You could instead take the mean of both, [4:28] which would give you this line. And that tells  you that the people who score least underrated [4:34] themselves whereas the people who score highest  overrated themselves. Still, how can it be that [4:40] there are so many different ways to look at  this data? Well, as a physicist, I'd say just [4:46] look at the data. It basically doesn't have  any correlation whatsoever. Why the hell are [4:52] you fitting any straight line to this? This is why  I give this paper as well as all previous papers [4:57] on the topic a 10 out of 10 on the [ __ ] meter.  These are all crappy data analysis. If you draw [5:04] enough lines through the data, one will eventually  confirm that everybody else is an idiot. Let me [5:09] instead tell you something much more interesting.  The original experiments also found that teaching [5:16] poor performers the relevant reasoning skills  improved both their performance and their ability [5:23] to recognize their earlier mistakes. To me, this  means that these tests don't actually measure [5:30] intelligence. They measure knowledge. There  is also the problem that the more people know [5:35] of the Dunning Krueger effect, the more they'll  revise their prediction of their own knowledge. [5:41] And if we've learned anything from this, it's that  being confidently wrong makes excellent outrage [5:47] bait. You see, the best thing about the Dunning  Krueger effect is that everyone recognizes it [5:53] immediately in someone else. I find it fascinating  how much research goes into everyday objects like [6:01] the razors from today's sponsor, Henson. This  might come as somewhat of a surprise to you, [6:07] but Henson's team have a background in aerospace  engineering. They've built parts for the ISS and [6:14] the Exomars rover. They use the same precision to  design their Razor. They figured out at exactly [6:21] which angle the blades need to be positioned so as  to cut off hairs perfectly and not cut your skin. [6:27] The pieces that hold the blades are not plastic,  but cut out from metal to a precision of a hundth [6:33] of a millimeter. I prefer my husband cleanly  shaven and without cuts. And Henson's is by far [6:40] the best razor I've seen at work. Also works for  women. And of course, I have a special offer. If [6:46] you buy a Henson razor through my link or use my  code, you'll get 100 blades for free along with [6:54] it. You've never shaved up scientifically.  Thanks for watching. See you tomorrow.