On High-Range Test Construction 14: Chris Cole, Merger of Ultra and Short Form Tests
Publisher: In-Sight Publishing
Publisher Founding: March 1, 2014
Web Domain: http://www.in-sightpublishing.com
Location: Fort Langley, Township of Langley, British Columbia, Canada
Journal: In-Sight: Independent Interview-Based Journal
Journal Founding: August 2, 2012
Frequency: Three (3) Times Per Year
Review Status: Non-Peer-Reviewed
Access: Electronic/Digital & Open Access
Fees: None (Free)
Volume Numbering: 12
Issue Numbering: 3
Section: E
Theme Type: Idea
Theme Premise: “Outliers and Outsiders”
Theme Part: 31
Formal Sub-Theme: High-Range Test Construction
Individual Publication Date: August 22, 2024
Issue Publication Date: September 1, 2024
Author(s): Chris Cole
Author(s) Bio: Chris Cole is a longstanding member of the Mega Society.
Word Count: 799
Image Credits: Ivan Gromov on Unsplash.
International Standard Serial Number (ISSN): 2369-6885
*Original publication March, 1993 here.*
*Please see the footnotes, bibliography, and citations, after the publication.*
Keywords: Chinese mythology, Cultural bias, Easy problems, Exhaustive reference work, High scorers, Low discrimination value, Master Crossword Puzzle Dictionary, Mega Society, National Puzzlers League, Potential test takers, Sample population, Short-form test, Statistical instability, Test taker, Verbal analogy problems, Western mythology.
On High-Range Test Construction 14: Chris Cole, Merger of Ultra and Short Form Tests
Ron Hoeflin has graciously consented to a merger of the Short Form Test and his work-in-progress, the Ultra Test. This means he has effectively donated the problems from his seven trial tests, which represents over a year of hard work. I propose that we call the merged test the Ultra Test.
Ron has convinced me to abandon the idea of a short-form test, in the sense of a small number of problems. There are two reasons for this: first, a small number of problems leads to statistical instability, and will make norming difficult, and second, by necessity, a short test would have all hard problems, which may be off-putting. In addition, a longer test will allow us to include several easy “aha!” problems, which will both entice and instruct the test taker. In other words, the easy problems indicate what kind of problems the hard ones are.
It is important for the test takers to understand that the problems are not amenable to exhaustive reference work or tedious calculation. Otherwise, they will abandon the test as too time-consuming. This explains, I think, the sharp drop off in takers between the Mega and Titan Tests. I think the audience of potential test takers was “burned out” by the Mega Test. With the Ultra Test, I hope to reinvigorate that audience as well as attract a whole new audience. There are many people who could qualify for the Mega Society if we could just get them to take the damn test!
In order to get a test published anywhere, it will have to be normed. This means it will have to be tried by a sample population. The only sample population readily available is the readership of Ron’s journals. Ron and I would like to publish the Ultra Test in the September issue of Ron’s journals. This will give us adequate time to collect and score answers by early next year. Therefore, this is the deadline: all candidate problems for the Ultra Test must be received by September 1. So, please start thinking of “Ultra type” verbal and math problems and submit them.
Ron picked the 41 most discriminating verbal analogy problems from his trial tests. Ron calculates the percentage of high scorers who correctly answer a question and subtracts from this the percentage of low scorers who answer correctly. Thus, easy problems and hard problems have a low discrimination value. I further culled this list of 41 problems down to the following 12. The criteria I used are these:
- Avoid reference exercises.
If the definition of the word is obvious from the analogy, but the word is obscure, the problem becomes a matter of searching reference material. This is not a test of intelligence; it is a test of who has the biggest thesaurus. I encourage all members to obtain a copy of Herbert M. Baus’ Master Crossword Puzzle Dictionary. This book is the standard reference book of the National Puzzlers League and was able to answer 80% of the Quest Test. Barnes and Noble recently stocked up on these and sells them for $15. You can also order one from their 800 number.
- Avoid idioms.
Idioms are not familiar to people for whom English is a second language. Native English speakers are a minority of the world’s population. We should strive for a test that has a wider audience.
- Avoid mythology and religion.
We should not expect Chinese speakers of English to know as much Western mythology as we know Chinese mythology. I know next to nothing about Chinese mythology. By the way, lest anyone think this is an overly harsh criterion, did you know that there are more students of English in China than there are speakers of English in the US?
- Avoid wordplay.
A play on words is biased toward native English speakers.
- Avoid quotations, titles, etc.
Again, these are culturally biased.
- Avoid “A: synonym of A:: B:?” or “A: B:: synonym of A:?”
This is a catch-all criterion, meant to include analogies that do not fall into any of the above categories exactly, but which still are not so much analogies as they are definitions. The relation of synonymy is not a good basis for an analogy.
So here are the 12 new problems:
- Space: Hyperspace:: Vector: ?
- Image: Idea :: Hallucination: ?
- Wind: Rain :: Typhoon: ?
- Inward: Outward :: Infection: ?
- Column: Row :: File: ?
- Humbug: Bach :: Seek: ?
- 38: Pyongyang :: 49: ?
- Of ten: Factor :: Of magnitude: ?
- Say: Hear :: Imply: ?
- 2.54: Inch :: 3.26: ?
- A, AB, B, BO, O: BO :: A, C, E, G, T: ?
- Eggs: Grading :: Wounded: ?
In the next issue, we will present the spatial questions selected from Ron’s tests, as well as all the other questions that will no doubt begin pouring in from the members who have been inspired by Ron’s generosity.
Footnotes
None
Citations
American Medical Association (AMA 11th Edition): Cole C. On High-Range Test Construction 14: Chris Cole, Merger of Ultra and Short Form Tests. August 2024; 12(3). http://www.in-sightpublishing.com/high-range-14
American Psychological Association (APA 7th Edition): Cole, C. (2024, August 22). On High-Range Test Construction 14: Chris Cole, Merger of Ultra and Short Form Tests. In-Sight Publishing. 12(3).
Brazilian National Standards (ABNT): COLE, C. On High-Range Test Construction 14: Chris Cole, Merger of Ultra and Short Form Tests. In-Sight: Independent Interview-Based Journal, Fort Langley, v. 12, n. 3, 2024.
Chicago/Turabian, Author-Date (17th Edition): Cole, Chris. 2024. “On High-Range Test Construction 14: Chris Cole, Merger of Ultra and Short Form Tests.” In-Sight: Independent Interview-Based Journal 12, no. 3 (Summer). http://www.in-sightpublishing.com/high-range-14.
Chicago/Turabian, Notes & Bibliography (17th Edition): Cole, C “On High-Range Test Construction 14: Chris Cole, Merger of Ultra and Short Form Tests.” In-Sight: Independent Interview-Based Journal 12, no. 3 (August 2024).http://www.in-sightpublishing.com/high-range-14.
Harvard: Cole, C. (2024) ‘On High-Range Test Construction 14: Chris Cole, Merger of Ultra and Short Form Tests’, In-Sight: Independent Interview-Based Journal, 12(3). <http://www.in-sightpublishing.com/high-range-14>.
Harvard (Australian): Cole, C 2024, ‘On High-Range Test Construction 14: Chris Cole, Merger of Ultra and Short Form Tests’, In-Sight: Independent Interview-Based Journal, vol. 12, no. 3, <http://www.in-sightpublishing.com/high-range-14>.
Modern Language Association (MLA, 9th Edition): Cole, Chris. “On High-Range Test Construction 14: Chris Cole, Merger of Ultra and Short Form Tests.” In-Sight: Independent Interview-Based Journal, vo.12, no. 3, 2024, http://www.in-sightpublishing.com/high-range-14.
Vancouver/ICMJE: Chris C. On High-Range Test Construction 14: Chris Cole, Merger of Ultra and Short Form Tests [Internet]. 2024 Aug; 12(3). Available from: http://www.in-sightpublishing.com/high-range-14.
License & Copyright
In-Sight Publishing by Scott Douglas Jacobsen is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. ©Scott Douglas Jacobsen and In-Sight Publishing 2012-Present. Unauthorized use or duplication of material without express permission from Scott Douglas Jacobsen strictly prohibited, excerpts and links must use full credit to Scott Douglas Jacobsen and In-Sight Publishing with direction to the original content.
In-Sight Publishing 