Important Continuous Statistical Distributions

Much of the distributional theory that underlies SEM today is built upon a handful of continuous statistical distributions--some well known, and some less so. All of these distributions are, in turn, related to each other. Any good advanced statistical text can provide details on the distributions discussed here.

Topics:

Normal Distribution

The normal, or Gaussian, distribution is one of the most familiar in statistics, endeared to statisticians by its simplicity and by virtue of the Central Limit Theorem (which states that a sample mean will follow an approximately normal distribution, if sample size is large enough, even if the data themselves are not normally distributed). The normal distribution has two parameters, one representing the mean of the variable and one representing the variance. The distribution is unimodal and symmetric about the mean. The standard normal distribution, which constrains the mean to 0 and the variance to 1, is probably the most widely known statistical distribution. The normal distribution also lies at the root of many other continuous statistical distributions.

One peculiar feature of the normal distribution is that it is completely described by its first two moments--higher order moments provide no new information about the distribution. This feature has both good and bad consequences. On the good side, statistical derivations involving the normal distribution are very much simplified, since higher order moments can be ignored. On the bad side, normal distributions increase the likelihood that the parameters of statistical models will not be identified, because there will be relatively few pieces of distinct information--fewer "knowns"--available for this purpose (Bekker, Merckens and Wansbeek, 1994).

Multivariate Normal Distribution

The multivariate normal, or spherical, distribution, is a generalization of the normal. Its parameters include not only the means and variances of the individual variables but also the correlations between those variables. Note that the multivariate normal distribution is not a mere composite of univariate normal distributions. Even if every variable in a set is normally distributed, it is still possible that the combined distribution is NOT multivariate normal. Therefore, to test for multivariate normality, researchers must compute multivariate measures of skewness and kurtosis. For this purpose, a coefficient of multivariate kurtosis due to Mardia (1970, 1985) is widely recommended. Many SEM software packages, which process raw data, compute this statistic.

Central and Noncentral Distributions

A (univariate) normally distributed variable may have either a zero mean or a nonzero mean. If the mean is zero, then we may say that the variable follows a central normal distribution. If the mean is nonzero, then the variable follows a noncentral distribution. At this level, the difference may be unimportant to the SEM researcher. However, when normally distributed variables are transformed, giving rise to other distributions, then the difference become significant. Central normal variables give rise to central distributions, while noncentral normal variables give rise to noncentral distributions. The resulting noncentral distributions are characterized by one additional parameter--a noncentrality parameter.

Elliptical Distribution

The normal distribution is a special case within the family of elliptical distributions. Elliptical distributions are symmetric and unimodal, but are not constrained regarding kurtosis. Thus, elliptical distributions themselves can be thought of as an intermediate point between the normal distribution and the general continuous distribution described in Browne's (1982) work on asymptotically distribution-free (ADF) estimation. Bentler's EQS package includes estimators specifically designed to be appropriate for elliptical distributions.

Chi-Square Distribution

The chi-square distribution is a univariate distribution which results when univariate-normal variables are squared, and possibly summed. While the normal distribution is symmetric, the chi-square distribution is skewed to the right, and has a minimum of 0. The degrees of freedom of the resulting chi-square distribution are equal to the number of variables that are summed. The mean of a chi-square distribution is equal to its degrees of freedom, and the variance is equal to twice the degrees of freedom.

These chi-square distributions are themselves additive. That is, a chi-square-distributed variable with d1 degrees of freedom can be added to one with d2 degrees of freedom to yield a chi-square-distributed variable with d1 + d2 degrees of freedom, as long as the two added variables are independent. Analogous results can be obtained by subtraction, under the same condition. Steiger, Shapiro and Browne (1985) showed that the independence condition was achieved for a priori-specified comparisons of the chi-square statistics from nested models. This gives rise to the familiar chi-square difference test.

As noted above, if some or all of the original normal variables had nonzero mean, then the result will be a noncentral chi-square distribution, with noncentrality parameter (lambda). The mean of this distribution is equal to df + , and the variance is equal to 2df + 4. The effect of the noncentrality parameter is to move the distribution to the right and to make it appear flatter and more symmetrical. Adding two independent noncentral chi-square distributed variables with noncentrality parameters 1 and 2 yields a noncentral chi-square distributed variable with noncentrality parameter (1 + 2). In principle, nested models can also be evaluated using difference tests, but the noncentrality issue complicates interpretation.

Wishart Distribution

This section is largely drawn from Arnold (1988).
The Wishart distribution is the multivariate analog to the chi-square, and is related to the multivariate normal in the same way that the chi-square is related to the univariate normal. Both central and noncentral Wishart distributions have been defined. As Bollen (1989), for example, notes, to say that certain discrepancy functions in SEM rely on multivariate normality in the data is really too restrictive. The essential requirement is that the covariance matrix of the data follow the (central) Wishart distribution. That is why Browne (1982) referred to the maximum likelihood estimation method as "maximum Wishart likelihood" or MWL.

If the covariance matrix, , is nonsingular, and if sample size exceeds the number of variables, then W, a matrix related to , will also be nonsingular. This is critical to deriving confidence intervals for and for simple, partial and multiple correlations.

References

Arnold, S. F. (1988). Wishart distribution. In S. Kotz & N. L. Johnson (eds.-in-chief), Encyclopedia of statistical sciences (vol. 9), pp. 641-645. New York: Wiley. Mardia, K. V. (1985). Mardia's test of multinormality. In S. Kotz & N. L. Johnson (Eds. in chief), Encyclopedia of statistical sciences (vol. 5), pp. 217-221. New York: Wiley.

Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57, pp. 519-530.


http://www.gsu.edu/~mkteer/continuo.html
Return to the SEMNET FAQ home page.
Return to Ed Rigdon's home page.
Last updated: May 9, 1996