On High-Range Test Construction 15: Chris Cole, Why I’m Interested in Intelligence Testing
Publisher: In-Sight Publishing
Publisher Founding: March 1, 2014
Web Domain: http://www.in-sightpublishing.com
Location: Fort Langley, Township of Langley, British Columbia, Canada
Journal: In-Sight: Independent Interview-Based Journal
Journal Founding: August 2, 2012
Frequency: Three (3) Times Per Year
Review Status: Non-Peer-Reviewed
Access: Electronic/Digital & Open Access
Fees: None (Free)
Volume Numbering: 12
Issue Numbering: 3
Section: E
Theme Type: Idea
Theme Premise: “Outliers and Outsiders”
Theme Part: 31
Formal Sub-Theme: High-Range Test Construction
Individual Publication Date: August 22, 2024
Issue Publication Date: September 1, 2024
Author(s): Chris Cole
Author(s) Bio: Chris Cole is a longstanding member of the Mega Society.
Word Count: 799
Image Credits: Annie Spratt on Unsplash.
International Standard Serial Number (ISSN): 2369-6885
*Original publication April, 1993 here.*
*Please see the footnotes, bibliography, and citations, after the publication.*
Keywords: Analogies, Anthropology, Aristocrat, Bacchus, Brahmin, Ceres, Computer Science, Concept Mastery, Danube, Euphrates, Feynman, Geography, Intelligence, Maoris, Mythology, New Zealand, Peer-reviewed, Proletarian, Psychometrics, Putnam Exam, Spelling, Terman, Ultra Test, Vocabulary.
On High-Range Test Construction 15: Chris Cole, Why I’m Interested in Intelligence Testing
I have in hand a copy of the Terman Concept Mastery Test (Form T), which I got from Ron Hoeflin. This is the test used by Terman to track 1004 gifted children into mid-life. This test shows what is wrong with high-level intelligence tests.
Terman’s sample questions are fine:
- Shoe: Foot:: Glove: (a. Arm b. Elbow c. Hand)
- Kitten: Cat:: Calf: (a. Horse b. Cow c. Lion)
But it is all downhill from there. Here are some typical questions from the test, and what is wrong with them:
- Proletarian: Worker:: Brahmin: (a. Bull b. Aristocrat c. India)
This is a vocabulary question. - Bacchus: Revelry:: Ceres: (a. Agriculture b. Love c. Hunting)
This is a mythology question. - Danube: Black Sea:: Euphrates: (a. Persian Gulf b. Red Sea c. Caspian Sea)
This is a geography question. - Maoris: New Zealand:: Ainus: (a. China b. India c. Japan)
This is an anthropology question.
Is a spelling bee an intelligence test? It may be the case that spelling is correlated with intelligence, but it is not the same as intelligence. I think Terman meant to produce a concept mastery test (which is a fine synonym for intelligence, as far as I’m concerned), but he could not think of enough hard analogies, so he got lazy and used hard questions from other disciplines.
The trouble is, we now can produce machines that can spell much better than we can (or do the simple information look-ups that are required by the above questions). But we cannot produce machines that can master concepts better than we can. So it is now interesting to quantify how good people (and ultimately machines) are at mastering concepts. This is why I am interested in true intelligence tests, and why I think the Ultra Test is worth working on.
I will give one example of a good “aha!” (or concept mastery) problem:
Start with a half cup of tea and a half cup of coffee. Take one tablespoon of the tea and mix it in with the coffee. Take one tablespoon of this mixture and mix it back in with the tea. Which of the two cups contains more of its original contents?
Answer on next page.
The two cups end up with the same volume of liquid they started with. The same amount of tea was moved to the coffee cup as coffee to the tea cup. Therefore, each cup contains the same amount of its original contents.
Every year, most of the top U.S. math majors in college take the Putnam Exam, which is a twelve-question, six-hour exam. This exam is intended to weed out the very best, most promising young mathematicians, and history shows that it is sufficient, if not necessary, to score in the top ten on the Putnam to have a productive career in math. For example, Feynman scored in the top three (in fact, he scored number one, although this was not published).
Over the years, the Putnam Exam has evolved in the direction that I am trying to take the Ultra Test. Although the Putnam requires too much specialist knowledge to be an intelligence test, I reproduce below some questions from the Putnam that do not require much specialist knowledge and that I think give the feel of the exam.
- 1966 A-6: What is √ 1+2√1+3 √1+4 √1+…?
- 1967 A-3: If f(x)=nx^2−bx+c has integer coefficients, what is the least value of a such that f(x) has two distinct zeros in 0<x<1?
- 1972 B-2: A particle moving on a straight line starts from rest and attains a velocity v0v0 after traversing a distance s0s0. If the motion is such that the acceleration was never increasing, find the maximum time for the traversal.
- 1947 11: aa, bb, cc, dd are distinct integers such that (x−a)(x−b)(x−c)(x−d)=0. If x is an integer, what is it?
- 1949 B-3: If any two points on a closed plane curve are no more than one unit apart, what is the radius of the smallest circle that completely contains the curve?
Earlier in this issue, Kevin Langdon argues that we should submit our proposed (and supposed) intelligence tests to the peer-reviewed publication process, and proposes that we start a journal that will include academic psychometricians on the mailing list. This is an interesting non sequitur, although I think we all understand why Kevin makes it. The truth is that no reputable psychometrics journal would publish an analysis of any of our tests. This is for a variety of reasons with which we are all familiar, and which I will not belabor here.
The point I want to make is that the road to scientifically accurate and generally accepted high-level intelligence tests will be a long and winding one. I suspect that the initial inroads, by the way, will not come from psychometricians, or even from psychologists of any stripe, but rather from computer scientists, who will be working from much the same motivation that I am.
However, a journey of a thousand miles starts with a single step. We are already a few steps into this journey, and I would like to make the Ultra Test the next step. So please be on the lookout for good problems and send them in.
Footnotes
None
Citations
American Medical Association (AMA 11th Edition): Cole C. On High-Range Test Construction 15: Chris Cole, Why I’m Interested in Intelligence Testing. August 2024; 12(3). http://www.in-sightpublishing.com/high-range-15
American Psychological Association (APA 7th Edition): Cole, C. (2024, August 22). On High-Range Test Construction 15: Chris Cole, Why I’m Interested in Intelligence Testing. In-Sight Publishing. 12(3).
Brazilian National Standards (ABNT): COLE, C. On High-Range Test Construction 15: Chris Cole, Why I’m Interested in Intelligence Testing. In-Sight: Independent Interview-Based Journal, Fort Langley, v. 12, n. 3, 2024.
Chicago/Turabian, Author-Date (17th Edition): Cole, Chris. 2024. “On High-Range Test Construction 15: Chris Cole, Why I’m Interested in Intelligence Testing.” In-Sight: Independent Interview-Based Journal 12, no. 3 (Summer). http://www.in-sightpublishing.com/high-range-15.
Chicago/Turabian, Notes & Bibliography (17th Edition): Cole, C “On High-Range Test Construction 15: Chris Cole, Why I’m Interested in Intelligence Testing.” In-Sight: Independent Interview-Based Journal 12, no. 3 (August 2024).http://www.in-sightpublishing.com/high-range-15.
Harvard: Cole, C. (2024) ‘On High-Range Test Construction 15: Chris Cole, Why I’m Interested in Intelligence Testing’, In-Sight: Independent Interview-Based Journal, 12(3). <http://www.in-sightpublishing.com/high-range-15>.
Harvard (Australian): Cole, C 2024, ‘On High-Range Test Construction 15: Chris Cole, Why I’m Interested in Intelligence Testing’, In-Sight: Independent Interview-Based Journal, vol. 12, no. 3, <http://www.in-sightpublishing.com/high-range-15>.
Modern Language Association (MLA, 9th Edition): Cole, Chris. “On High-Range Test Construction 15: Chris Cole, Why I’m Interested in Intelligence Testing.” In-Sight: Independent Interview-Based Journal, vo.12, no. 3, 2024, http://www.in-sightpublishing.com/high-range-15.
Vancouver/ICMJE: Chris C. On High-Range Test Construction 15: Chris Cole, Why I’m Interested in Intelligence Testing [Internet]. 2024 Aug; 12(3). Available from: http://www.in-sightpublishing.com/high-range-15.
License & Copyright
In-Sight Publishing by Scott Douglas Jacobsen is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. ©Scott Douglas Jacobsen and In-Sight Publishing 2012-Present. Unauthorized use or duplication of material without express permission from Scott Douglas Jacobsen strictly prohibited, excerpts and links must use full credit to Scott Douglas Jacobsen and In-Sight Publishing with direction to the original content.
In-Sight Publishing 