Thursday, May 03, 2007

chi square difference test for nested SEM models

  • Smaller χ2 values indicate better fitting model, an insignificant χ2 (p>.05, p>.01) is desirable
  • If the difference between two nested SEM models is significant, this implies that the model with more paths explains the data better. If there is no significant difference between two nested models, this implies that the more parsimonious model explains the data equally well compared to the fuller model and is preferred.
  • If the fit of the more restricted model (model with fewer free parameters) is about as good as that of the more general model (model with more free parameters), the restrictions can probably be accetpted, i.e, the simpler model is chosen and the more complex model is rejected.
  • conducted a chi-square difference where the chi-square1 - chi-square 2 is the chi-square difference. Chi-square difference isexamined with the df of df1-df2. If the difference is significant,favor the model with the smaller chi-square. If not, favor the more parsimonious model.
  • (1) Subtract the chi-square for the less restricive model from the chi-square for the more restrictive model.(2) Subtract the degrees of freedom for the less restrictive model from the degrees of freedom for the more restrictive model.(3) Refer the chi-square difference obtained in (1) to the chi-square distribution using the df obtained in (2).
  • Yuan, K.-H., & Bentler, P.M. (2004). On chi-square difference and z-tests in mean and covariance structure analysis when the base model is misspecified. Educational and Psychological Measurement, 64, 737-757.
  • Model trimming

Model 1 has more free parameters, better fit, lower chi-squre value, lower degree of freedom, χ2=2.28 df=1

Model 2 had additional restrictions, fewer free parameters (model 2 is the more restricted model, more parsimonious), worse fit, higher chi-squre value, higher degree of freedom, χ2=15.96, df=3

Model 2 is nested within model 1 (model 2's free parameters is a subset of model 1's free parameters)

The difference between the two χ2 is also distributed as a χ2 with degree of freedom equal to
the difference between the degrees of freedom for the two models. χ2= 15.96-2.28= 13.68
Whether the additional constraints in model 2 have significantly reduced the model's ability to fit the data (whether the increase in χ2 value,reducing model fit, is significant )?
After checking the χ2 table, we find p< .01 (significant)--the fit of the model has been significantly hindered by introducing the additonal constraints (the imposed restrictions in the more constrained model result in a significant decrement in model fit; the increase in χ2 value is siginicant in reducing model fit) ---- so we choose model 1, although model 2 is more parsimonious. However, if after checking χ2 table, we find p> .01 (insignificant)--the fit of the model has not been significantly hindered by introducing the additional constraints (chi-square difference is not significant, one can retain the hypothesis of validity of the imposed constraints; the increase in χ2 value is not significant in reduing model fit)--so we choose model 2, because model 2's added constraints doesn't significantly reduce model fit, but model 2 is more parsiminous than model 1

Model builidng

  1. If one has an initially poor model with high χ2, the next step is to free one or more parameters, adding paths, reducing the number of model constraints, to improve the model's fit. If the reduction in χ2 (the χ2 difference) is larger relative to the difference if d.f. between the two models, we have achieved a significant improvement in fit.
  2. Models can have parameters sequentially freed,ie., adding extra paths, and the resulting successive models can be compared to assess whether freeing a particular parameter led to a significant improvement in model fit. (more parameters--better fit--lower chi-square value)

Reading list:

  • Kumar, A., & Sharma, S. (1999). A metric measure for direct comparison of competing models in covariance structure analysis. Structural Equation Modeling, 6, 169-197.
  • Rigdon, E. E. (1999). Using the Friedman method of ranks for model comparison in structural equation modeling. Structural Equation Modeling, 6, 219-232.
  • Steiger, J. H., Shapiro, A., & Browne, M. W. (1985). On the multivariate asymptotic distribution of sequential chi-square statistics. Psychometrika, 50, 253-264.
  • Cheung & Rensvold (2002), Evaluating goodness of fit indexes for testingmeasurement invariance. SEM 9(2) 233-255
  • Kelloway (1995)Structural equation modeling in perspective. Journal ofOrganizational Behavior Vol 16, 215-224.
  • Brannick (1995) Critical comments on applying covariance structuremodeling. Journal of Organizational Behavior Vol 16, 201-213.

1 comment:

Anonymous said...

Is there any other ways to show convergent validity by SEM (AMOS)than AVE?
May I use CU Model for showing convergent validity?

please help me !