Thank you for the effort on the piece. I’m happy to debate some points in this, but before that, I just wanted to clear something up first: you definitely attribute things to me here I never said, and attribute to me mistakes I never made. I don’t mean in a vague wishy-washy way where we both kind of have a point in some esoteric statistical argument that's complicated. I mean in a very clear and trackable and simple way, your piece contains mistakes.
To see what I mean, let’s focus on just one part, the one involving Anne Roe. In fact, let’s focus on one sentence. You say “Contrary to Hoel, it is not difficult to figure out the sample sizes of the students or how they were recruited, Roe tells us:”
Then you immediately quote Roe herself, bolding two phrases (I've capitalized them here):
“I made an attempt to get some graduate students to take the same test, just as a matter of general interest, BUT SUCCEEDED IN GETTING ONLY 10, and under circumstances which made it impossible to judge how they had been selected. I then dropped the idea of getting any comparison group until it was announced that tests were to be devised in connection with draft deferment. It then seemed to me that I should make some effort to get a reasonably exact idea of just where these eminent scientists stood in the general distribution of intelligence in the population. Such information might be of use,—particularly in determining the level at which exemption might be considered on this basis,—and was not obtainable anywhere else. I had the great good fortune about then to meet an old acquaintance, Dr. Irving Lorge, who came to my rescue and ARRANGED TO GIVE THE TEST TO ALL STUDENTS MATRICULATING AT TEACHERS COLLEGE, COLUMBIA, FOR A PHD THAT FEBRUARY."
Yet, all the relevant information in that passage is mentioned, very explicitly, in what I wrote on the subject of Roe’s sample, both how they were recruited and their size. Here's the entirety of what I wrote on the matter of Roe's comparison group, so this is the only sentence you could possibly be referring to:
“And while Roe didn’t publish without any comparison group to her chosen geniuses whatsoever, the comparison that she did use was only a graduating class of PhD students (sample size unknown, as far as I can tell) who also took some other more standard IQ tests of the day, and she basically just converted from their scores on the other tests to scores on her make-shift test of SAT questions.”
If you just read that, it makes literally no sense to write that "Contrary to Hoel, it is not difficult to figure out the sample sizes of the students or how they were recruited, Roe tells us:”
In fact, it makes even less sense because you appear to not comprehend what Roe herself wrote, and in the very passage you're quoting! For you also bolded “succeeded in getting only 10" as if that was the information you figured out about sample size. Let’s have that on record. Because this number is wholly irrelevant. It's not the sample size. Go re-read the paragraph. Roe says she attempted to get graduate students, and got only 10, and she couldn’t tell how they were selected. So then she dropped the idea altogether, and didn't use those scores at all (it's just backstory) until she heard about testing and the draft, and redoubled her efforts on her work. Then, later, someone “arranged to give the test to all students matriculating at Teachers College, Columbia, for a PhD that February” and that’s the unknown-sized sample that she used in the book, the one to which she compared the scores on her make-shift test (exactly as I wrote).
So a correct reading comprehension of Roe's passage here should tell us that, unless we can figure out exactly how many students were matriculating at Teachers College, Columbia, for a PhD, in a February of an unknown year sometime around the 1940s or 1950s, then yeah, the sample size of the class is unknown (and even if this number could be tracked down today, or you look at averages of classes, the sample size is not in the uploaded book, so the sentence is true regardless). That's just what she wrote. And what I wrote. Correctly. About what she wrote. But that's not what you wrote.
Overall, happy you took the time, appreciate the effort, but unhappy about the failures of reading comprehension for both me and Roe. That sort of thing would need to get resolved before we move to things like:
(a) Does pulling out the Flynn effect to explain why the past Nobel Prize winners scored so average on questions we find easy today ring convincing, or does it just show off how bad and nonsensical the IQ-to-genius literature is ? (e.g., the Flynn effect is mostly driven from rising averages, not PhD students and intellectuals - go read an older popular novelist and marvel at the vocabulary compared to today!)
(b) Does one graph on IQ and income (the one you show) overrule the other graph (the one I show) and ends the debate, or can I just flash my other graph in return, and so on for eternity?
(c) Is quoting the abstract of a paper I criticized for overstating its case in the abstract, and then reporting essentially the opposite in the results, convincing?
His general advice to distrust claimed high scores for specific, DEAD humans is good, too. - Adding deviations seems legit - though not highly accurate (I did a test once, where I allegedly scored 2 sd above the avg. of young teachers --- well, I just assume teachers are not much above the mean)
The Flynn-logic seems less convincing: a) Saying (stupidly) Einstein had an "IQ of 160", is meant as: he would score that, if he did one now. Not: "Today his score would be merely 120, but with that kind of smart he was a brightest candle on the cake in 1905." (Btw; who would expect Einstein to score very high on verbal test-parts? ) . Hoel gave just one example of an item, you quoted it, His claim: not a great item for high-IQ-testing. I agree, even if most folks in 1940 sucked at antonyms. Where to find the whole list of items?
Then again: in the middle ages it was admired as a rare feat of the highest erudition if one could read: silently. (And I met an Arab Professor of German, who could not do so. In German, at least). Flynn all the way down.
To me, the real question about high IQ is the "dog that didn't bark" question. Why does IQ top out around 200, even in the most extreme cases? Why not 500, or 50,000? And don't tell me "It's brain size". Parrots, with brains the size of a hazelnut are as intelligent or even moreso than elephants. A fairy fly, an insect smaller than the diameter of a human hair can fly, process visual information, mate, and reproduce. It's brain is microscopic. There has to be an evolutionary disadvantage to having an IQ above a certain level.
> It won't be entirely possible to prevent people from cheating and taking the test many times, or secretly training beforehand, but if we really wanted to, we could make a company that tests people in person using modern computer adaptive testing and give people proper high-range results. Is there a market for this? Probably not, but maybe. Certainly, online tests of this kind should be made. In fact, I am coincidentally working on a project like this right now.
How bad are practice effects? I've done some stuff like free IQ tests, similar to Raven's Progressive Matrices. Never knew how trustworthy were the results. Maybe 2 years ago I learned real Raven's 2 test is available for a few dollars from Pearson (if you lie you're psychiatrist or something). I got 129 that time.
I want to re-take it, because I didn't measure how long I took to solve it (so maybe it's too high), and also fucked up by taking hallucinogens (metocin) mid-way. I wanted them to kick in after I'm done, but miscalculated. Tho I probably wouldn't solve these last items correctly even if stuff didn't drift and hue-shift. Also it was 4AM.
Nice article, would have liked to hear more about the psychometrics of genius vs “very high is person”. I think I remember you was saying it was something like g + openness/creativity. Supports Hoel’s arg that genius isn’t just IQ
I can recommend the 1926 book by Cox "301 Geniuses" where they give thorough estimates for IQ. John Stuart Mill was estimated to 190 as one of the highest
Honestly the most interesting take aways from his piece were 1. When he showed some individuals having up to a *20 point* swing in IQ score between different tests and attempts, and the majority of people having some significant degree of variation. And 2. The fact that Richard Feynman apparently had an IQ of 125.
The following study looked at a sample of 1238 people with a mean IQ of 170:
A genome-wide association study for extremely high intelligence: "We used a case–control genome-wide association (GWA) design with cases consisting of 1238 individuals from the top 0.0003 (~170 mean IQ) of the population distribution of intelligence and 8172 unselected population-based controls. The single-nucleotide polymorphism heritability for the extreme IQ trait was 0.33 (0.02), which is the highest so far for a cognitive phenotype, and significant genome-wide genetic correlations of 0.78 were observed with educational attainment and 0.86 with population IQ"
Also, his claim that after a point IQ doesn't matter can be easily falsified. Look at number of publications and books, for instance. There are some academics who have only published one or two books in their lives. Others, like Vaclav Smil, have published over 40. Even among Nobel laureates there are clear differences in achievement. Some laureates got 2 Nobels, and some like Einstein are clearly above the rest.
(I claim to have an IQ of at least 100 with a probability of 50%.)
As for 160 or above IQ, and using the M=100 SD=15 normal distribution on a 330 million person population, there should be about 10,452 persons in that range in the US. So that's 10,452 leprechauns… should be good for finding pots of gold.
(Trust but verify: this number can be computed by typing 330000000*(1-pnorm(4)) in RStats.)
It's probably nonsense to use a Normal distribution for IQ as we get farther away from the mean, for any logic for using it beyond 200 would by necessity apply equally to negative IQ numbers.
PERSONAL SPECULATION: As for what the number measures, I'd venture that well below 160, say 130-145, it takes on ordinal-only rather than cardinal characteristics (ex: 150 is larger than 140, but that doesn't map to the same differences or ratios of the numbers below; that 150-140 difference is not equivalent to 120-110 and that 150/140 is not equivalent to 107/100). Possibly a similar ordinal-ness for numbers below say 55-70 as well.
I was hoping you’d comment on his stupid remarks. I don’t trust people that brag about high IQs either but I also don’t trust people who bash on IQ testing.
Thank you for the effort on the piece. I’m happy to debate some points in this, but before that, I just wanted to clear something up first: you definitely attribute things to me here I never said, and attribute to me mistakes I never made. I don’t mean in a vague wishy-washy way where we both kind of have a point in some esoteric statistical argument that's complicated. I mean in a very clear and trackable and simple way, your piece contains mistakes.
To see what I mean, let’s focus on just one part, the one involving Anne Roe. In fact, let’s focus on one sentence. You say “Contrary to Hoel, it is not difficult to figure out the sample sizes of the students or how they were recruited, Roe tells us:”
Then you immediately quote Roe herself, bolding two phrases (I've capitalized them here):
“I made an attempt to get some graduate students to take the same test, just as a matter of general interest, BUT SUCCEEDED IN GETTING ONLY 10, and under circumstances which made it impossible to judge how they had been selected. I then dropped the idea of getting any comparison group until it was announced that tests were to be devised in connection with draft deferment. It then seemed to me that I should make some effort to get a reasonably exact idea of just where these eminent scientists stood in the general distribution of intelligence in the population. Such information might be of use,—particularly in determining the level at which exemption might be considered on this basis,—and was not obtainable anywhere else. I had the great good fortune about then to meet an old acquaintance, Dr. Irving Lorge, who came to my rescue and ARRANGED TO GIVE THE TEST TO ALL STUDENTS MATRICULATING AT TEACHERS COLLEGE, COLUMBIA, FOR A PHD THAT FEBRUARY."
Yet, all the relevant information in that passage is mentioned, very explicitly, in what I wrote on the subject of Roe’s sample, both how they were recruited and their size. Here's the entirety of what I wrote on the matter of Roe's comparison group, so this is the only sentence you could possibly be referring to:
“And while Roe didn’t publish without any comparison group to her chosen geniuses whatsoever, the comparison that she did use was only a graduating class of PhD students (sample size unknown, as far as I can tell) who also took some other more standard IQ tests of the day, and she basically just converted from their scores on the other tests to scores on her make-shift test of SAT questions.”
If you just read that, it makes literally no sense to write that "Contrary to Hoel, it is not difficult to figure out the sample sizes of the students or how they were recruited, Roe tells us:”
In fact, it makes even less sense because you appear to not comprehend what Roe herself wrote, and in the very passage you're quoting! For you also bolded “succeeded in getting only 10" as if that was the information you figured out about sample size. Let’s have that on record. Because this number is wholly irrelevant. It's not the sample size. Go re-read the paragraph. Roe says she attempted to get graduate students, and got only 10, and she couldn’t tell how they were selected. So then she dropped the idea altogether, and didn't use those scores at all (it's just backstory) until she heard about testing and the draft, and redoubled her efforts on her work. Then, later, someone “arranged to give the test to all students matriculating at Teachers College, Columbia, for a PhD that February” and that’s the unknown-sized sample that she used in the book, the one to which she compared the scores on her make-shift test (exactly as I wrote).
So a correct reading comprehension of Roe's passage here should tell us that, unless we can figure out exactly how many students were matriculating at Teachers College, Columbia, for a PhD, in a February of an unknown year sometime around the 1940s or 1950s, then yeah, the sample size of the class is unknown (and even if this number could be tracked down today, or you look at averages of classes, the sample size is not in the uploaded book, so the sentence is true regardless). That's just what she wrote. And what I wrote. Correctly. About what she wrote. But that's not what you wrote.
Overall, happy you took the time, appreciate the effort, but unhappy about the failures of reading comprehension for both me and Roe. That sort of thing would need to get resolved before we move to things like:
(a) Does pulling out the Flynn effect to explain why the past Nobel Prize winners scored so average on questions we find easy today ring convincing, or does it just show off how bad and nonsensical the IQ-to-genius literature is ? (e.g., the Flynn effect is mostly driven from rising averages, not PhD students and intellectuals - go read an older popular novelist and marvel at the vocabulary compared to today!)
(b) Does one graph on IQ and income (the one you show) overrule the other graph (the one I show) and ends the debate, or can I just flash my other graph in return, and so on for eternity?
(c) Is quoting the abstract of a paper I criticized for overstating its case in the abstract, and then reporting essentially the opposite in the results, convincing?
And so on.
His general advice to distrust claimed high scores for specific, DEAD humans is good, too. - Adding deviations seems legit - though not highly accurate (I did a test once, where I allegedly scored 2 sd above the avg. of young teachers --- well, I just assume teachers are not much above the mean)
The Flynn-logic seems less convincing: a) Saying (stupidly) Einstein had an "IQ of 160", is meant as: he would score that, if he did one now. Not: "Today his score would be merely 120, but with that kind of smart he was a brightest candle on the cake in 1905." (Btw; who would expect Einstein to score very high on verbal test-parts? ) . Hoel gave just one example of an item, you quoted it, His claim: not a great item for high-IQ-testing. I agree, even if most folks in 1940 sucked at antonyms. Where to find the whole list of items?
Then again: in the middle ages it was admired as a rare feat of the highest erudition if one could read: silently. (And I met an Arab Professor of German, who could not do so. In German, at least). Flynn all the way down.
You are working on devising IQ test for high end results? I am interested. When you expect to finish it?
To me, the real question about high IQ is the "dog that didn't bark" question. Why does IQ top out around 200, even in the most extreme cases? Why not 500, or 50,000? And don't tell me "It's brain size". Parrots, with brains the size of a hazelnut are as intelligent or even moreso than elephants. A fairy fly, an insect smaller than the diameter of a human hair can fly, process visual information, mate, and reproduce. It's brain is microscopic. There has to be an evolutionary disadvantage to having an IQ above a certain level.
> It won't be entirely possible to prevent people from cheating and taking the test many times, or secretly training beforehand, but if we really wanted to, we could make a company that tests people in person using modern computer adaptive testing and give people proper high-range results. Is there a market for this? Probably not, but maybe. Certainly, online tests of this kind should be made. In fact, I am coincidentally working on a project like this right now.
How bad are practice effects? I've done some stuff like free IQ tests, similar to Raven's Progressive Matrices. Never knew how trustworthy were the results. Maybe 2 years ago I learned real Raven's 2 test is available for a few dollars from Pearson (if you lie you're psychiatrist or something). I got 129 that time.
I want to re-take it, because I didn't measure how long I took to solve it (so maybe it's too high), and also fucked up by taking hallucinogens (metocin) mid-way. I wanted them to kick in after I'm done, but miscalculated. Tho I probably wouldn't solve these last items correctly even if stuff didn't drift and hue-shift. Also it was 4AM.
Nice article, would have liked to hear more about the psychometrics of genius vs “very high is person”. I think I remember you was saying it was something like g + openness/creativity. Supports Hoel’s arg that genius isn’t just IQ
Strange to use so much time on Einsteins IQ, a well known fraud.
I can recommend the 1926 book by Cox "301 Geniuses" where they give thorough estimates for IQ. John Stuart Mill was estimated to 190 as one of the highest
Honestly the most interesting take aways from his piece were 1. When he showed some individuals having up to a *20 point* swing in IQ score between different tests and attempts, and the majority of people having some significant degree of variation. And 2. The fact that Richard Feynman apparently had an IQ of 125.
The following study looked at a sample of 1238 people with a mean IQ of 170:
A genome-wide association study for extremely high intelligence: "We used a case–control genome-wide association (GWA) design with cases consisting of 1238 individuals from the top 0.0003 (~170 mean IQ) of the population distribution of intelligence and 8172 unselected population-based controls. The single-nucleotide polymorphism heritability for the extreme IQ trait was 0.33 (0.02), which is the highest so far for a cognitive phenotype, and significant genome-wide genetic correlations of 0.78 were observed with educational attainment and 0.86 with population IQ"
Also, his claim that after a point IQ doesn't matter can be easily falsified. Look at number of publications and books, for instance. There are some academics who have only published one or two books in their lives. Others, like Vaclav Smil, have published over 40. Even among Nobel laureates there are clear differences in achievement. Some laureates got 2 Nobels, and some like Einstein are clearly above the rest.
(I claim to have an IQ of at least 100 with a probability of 50%.)
As for 160 or above IQ, and using the M=100 SD=15 normal distribution on a 330 million person population, there should be about 10,452 persons in that range in the US. So that's 10,452 leprechauns… should be good for finding pots of gold.
(Trust but verify: this number can be computed by typing 330000000*(1-pnorm(4)) in RStats.)
It's probably nonsense to use a Normal distribution for IQ as we get farther away from the mean, for any logic for using it beyond 200 would by necessity apply equally to negative IQ numbers.
PERSONAL SPECULATION: As for what the number measures, I'd venture that well below 160, say 130-145, it takes on ordinal-only rather than cardinal characteristics (ex: 150 is larger than 140, but that doesn't map to the same differences or ratios of the numbers below; that 150-140 difference is not equivalent to 120-110 and that 150/140 is not equivalent to 107/100). Possibly a similar ordinal-ness for numbers below say 55-70 as well.
I was hoping you’d comment on his stupid remarks. I don’t trust people that brag about high IQs either but I also don’t trust people who bash on IQ testing.
That doesn't explain why the media IQ isn't 500, or 50,000.