Wednesday, October 24, 2007

Has a statistician demolished the idea of a single general intelligence factor?

Cosma Shalizi is a rather egotistical-sounding young man of apparently Afghan ancestry. He is also an assistant professor in the Department of Statistics at Carnegie Mellon University in Pittsburgh. On his blog, Three Toed Sloth, he has a long post demonstrating that a popular form of mathematical analysis generically called factor analysis cannot be used to demonstrate the existence of IQ (or 'g' as we psychometricans call it).

His criticisms are perfectly correct. Factor analysis cannot prove ANYTHING in fact. It is just a convenient but rather arbitrary way of summarizing a set of correlations. Factor analysis is such a weak technique that I myself in my more than 200 published academic papers have used it only rarely -- even though I normally present my research results in correlational form. It is my rather severe view that most (though not all) theories in psychology can be substantiated or dismissed with just one Pearsonian correlation coefficient. Anything else tends to suggest poor research design to me. (Yes. I do know about non-linear relationships. I tested for one such in my very first dissertation, in fact).

The sloth is also correct in saying that much of the research into IQ has used factor analysis as a way of summarizing findings. That the findings are not DEPENDANT on that descriptive technique is the point he appears to overlook.

The underlying finding for 100 years or more is that ability to solve one sort of puzzle generalizes strongly to the ability to solve other quite different sorts of puzzle. Problem solving ability is general, no matter how you choose to summarize that. And problem-solving ability is what IQ or 'g' measures.

Perhaps the biggest surprise is that verbal ability (as indexed -- say -- by the number of rare words understood) correlates well with mathematical ability (as indexed -- say -- by ability to detect complex numerical sequences). This is despite the fact that many people report being good in only one of those areas. I am in fact one of those who are good with words but shaky with mathematics. Yet I did for some years teach statistics at a major university. I obviously had some mathematical ability despite my discomfort with the subject. And it is the genes too. My son now teaches mathematics at another major university. And at school he was always a couple of years ahead of his class in reading ability too. Pesky stuff, that IQ.

The sloth should rise above his fascination with mathematical processes and focus on the underlying reality.

There is another over-enthusiastic blog here that claims that the sloth "offers an exhaustive demolition of the idea of a single general intelligence factor". The blogger apparently believes that the sloth has magically made lots of large correlation coefficients disappear. One of the commenters on that blog made an excellent point in addressing the common allegation that IQ tests measure only the ability to pass IQ tests. He said:

"If IQ tests are worthless, then so are college degrees. What do they really measure other than the ability to pass college classes? Nothing. Maybe we should assign MDs by lottery as well. I'll let you go first on the triple bypass surgery under your new world order."

(For more postings from me, see TONGUE-TIED, EDUCATION WATCH INTERNATIONAL, GREENIE WATCH, POLITICAL CORRECTNESS WATCH, FOOD & HEALTH SKEPTIC, GUN WATCH, SOCIALIZED MEDICINE, AUSTRALIAN POLITICS, DISSECTING LEFTISM, IMMIGRATION WATCH INTERNATIONAL and EYE ON BRITAIN. My Home Pages are here or here or here. Email me (John Ray) here.)

No comments:

Post a Comment