Saturday, May 05, 2007

Exploratory factor analysis, 1

  • In EFA, orthogonal solutions almost always provide simple structure
  • Pearson product-moment bivariate correlation matrix is the matrix of associations most commonly used in EFA
  • CFA uses covariance matrix. EFA uses correlation matrix; the reason for this standardization is that scales for tests used in educational, sociological, and psychological research are usually arbitrary and thus those scales are commensurable. The components obtained from correlation matrix in EFA and covariance matrix in EFA are not the same.
  • In EFA, we almost always use only standardized factor pattern coefficients.
  • In CFA, both unstandardized pattern coefficient and standardized pattern coefficients are computed.
  • When factors are uncorrelated, the CFA standardized factor pattern coefficients and the structure coefficients for given variables on given factors exactly equal each other
  • If using oblique (correlated) rotation, pattern coefficient and struture coefficient are different. Both of them should be reported.
  • factor score, latent variable score=composite (variable) score for each person on each factor, use "factor score matrix output" in SPSS, factor scores can be used in subsequent analyses instead of the measured variable scores, different factor score methods (regression, Bartlett, Anderson-Rubin), when principal component analysis is conducted, regression, Bartlett, and Anderson-Rubin factor scores for a give person on a given factor will all be the same; using other extraction method, they will differ
Factor extraction method: principal components analysis vs. factor analysis
  • Principal components analysis is the default factor extration method. Generall, we prefer principal components analysis because 1) a psychometrically sound procedure, 2) mathematically simpler than factor analysis, 3) common factor analysis might have factor indeterminacy problem, which is troublesome. Principal components analysis partitions the total variance in the original set of variables by finding variables that accounts for the largest amount of variance. The most amount of variance means about 75% or more, and often this can be accomplished with 5 or less components.
  • principal axes factor analysis
Factor rotation, In some cases, simple structure can't be obtained using orghogonal rotation. The researcher then must turn to oblique rotation to pursue simple structure.
Orthogonal rotation(uncorrelated factors), when factors are rotated orthogonally, the patten coefficient and structure coefficient matrices will contain exactly the same numbers, in a orthogonal rotation, if one variable has a pattern/structure coefficient of o.7--this variable has 49% of its variance in common with the factor; it another variable has a pattern/structure coefficient of 0.5--this variable has only 25% common variance with the factor, the first variable have double influence on the factor, compared with the second variable.
  • varimax (most commonly used)--to clean up the factors, ie, each factor tends to load high on a smaller number variables and low or very low on the other variables, this will make interpretation of the resulting factors easiler. When we use varimx rotation, the first rotated factor will no longer necessarily account for the largest amoung of variance.
  • Quartimax---to clean up the variables in order that each variable loads mainly on one factor).
Oblique rotation (correlated factors), If simple structure can't be obtained with orthogonal rotation, reflected in many variables having pattern/structure coefficients that are large in magnitude on two or more factors, oblique rotataion may be needed to derive interpretable factors. However,more parameters must be estimated in oblique rotation. Thus, oblique rotation results not only tend to fit sample data better but also tend to yield factors that replicate less well. --- unless oblique results are substantially better than orthogonal results, orthogonal results may be preferred. Whenever oblique rotation is performed, higher-order factors are implied, ie, the higher order factor explain the correlation among first-order factors, and should be derived and interpreted. When factors are rotated obliquely, the pattern coefficients and structural coefficients are different. Many reseachers suggest that correlated factors are much more reasonable to assume in most cases. There is no such thing as a "best" oblique rotation.
  • Promax(most commonly used)
  • direct oblimin, oblimax, quartimin, maxplane, onlimin, orthoblique
For oblique rotation, we will have two resulting matrix. The two matrices are different. Some researchers suggest that we need to report both matrices. ps, for orthogonal rotation, the two matrices are the same
  • factor pattern matrix---the elements are analogous to standardized regression coefficients from a multiple regression analysis, indicating the importance of that variable to the factor with the influence of the other variables partialled out
  • factor structure matrix--- the elements are simple correlations of the variables with the factors, that is, they are factor loadings. ps, loading is simply the Pearson correlation between the variable and the factor (linear combination of variables)
How many factors to select out?
  • choose component whose eigenvalue greater than 1
  • scree test
  • retain factors that would account for at least 70% of the total variance in the original variables
Subject & variable ratio
  • five subjects per variable as the minimum
Bartlett's sphericity test
  • this tests the null hypothesis that the variables in the population correlation matrix are uncorrelated. If one fails to reject null hypothesis, using this test, then there is no reason to do the components analysis since the variables are already uncorrelated.
Reading list
  • James Stevens, 2002, EFA and CFA, chapter 11, in Applied Multivariate Statistics for the Social Science, 4th ed. Lawrence. provide LISREL & EQS example to run CFA


Michael said...

This article is very detailed. I am much impressed. But I have a question " If I use promax for EFA, and then get a result of 2 components. Then can I use LISREL for CFA with promax rotatoin??? or CFA is only suitable to use varimax?

Anonymous said...

Thank you very much for your kind posts. Your blog is amazing!
lately I have been reading a lot of articles which are against the use of PCA. I would love to know some thoughts in regards to this.
Thank you!!!

Marcela Moraes said...
This comment has been removed by the author.