First, the researcher may get a message saying that the input covariance or correlation matrix being analyzed is "not positive definite." Generalized least squares (GLS) estimation requires that the covariance or correlation matrix analyzed must be positive definite, and maximum likelihood (ML) estimation will also perform poorly in such situations. If the matrix to be analyzed is found to be not positive definite, many programs will simply issue an error message and quit.

Second, the message may refer to the asymptotic covariance matrix. This is not the covariance matrix being analyzed, but rather a weight matrix to be used with asymptotically distribution-free / weighted least squares (ADF/WLS) estimation.

Third, the researcher may get a message saying that its estimate of Sigma (), the model-implied covariance matrix, is not positive definite. LISREL, for example, will simply quit if it issues this message.

Fourth, the program may indicate that some parameter matrix within the model is not positive definite. This attribute is only relevant to parameter matrices that are variance/covariance matrices. In the language of the LISREL program, these include the matrices Theta-delta, Theta-epsilon, Phi () and Psi. Here, however, this "error message" can result from correct specification of the model, so the only problem is convincing the program to stop worrying about it.

S = e'Me

To an extent, however, we can discuss positive definiteness in terms of the sign of the "determinant" of the matrix. The determinant is a scalar function of the matrix. In the case of symmetric matrices, such as covariance or correlation matrices, positive definiteness wil only hold if the matrix and every "principal submatrix" has a positive determinant. ("Principal submatrices" are formed by removing row-column pairs from the original symmetric matrix.) A matrix which fails this test is "not positive definite." If the determinant of the matrix is exactly zero, then the matrix is "singular." (Thanks to Mike Neale, Werner Wothke and Mike Miller for refining the details here.)

Why does this matter? Well, for one thing, using GLS estimation methods involves inverting the input matrix. Any text on matrix algebra will show that inverting a matrix involves dividing by the matrix determinant. So if the matrix is singular, then inverting the matrix involves dividing by zero, which is undefined. Using ML estimation involves inverting Sigma, but since the aim to maximize the similarity between the input matrix and Sigma, the prognosis is not good if the input matrix is not positive definite. Now, some programs include the option of proceeding with analysis even if the input matrix is not positive definite--with Amos, for example, this is done by invoking the $nonpositive command--but it is unwise to proceed without an understanding of the reason why the matrix is not positive definite. If the problem relates to the asymptotic weight matrix, the researcher may not be able to proceed with ADF/WLS estimation, unless the problem can be resolved.

In addition, one interpretation of the determinant of a covariance or correlation matrix is as a measure of "generalized variance." Since negative variances are undefined, and since zero variances apply only to constants, it is troubling when a covariance or correlation matrix fails to have a positive determinant.

Another reason to care comes from mathematical statistics. Sample covariance matrices are supposed to be positive definite. For that matter, so should Pearson and polychoric correlation matrices. That is because the population matrices they are supposedly approximating *are* positive definite, except under certain conditions. So the failure of a matrix to be positive definite may indicate a problem with the input matrix.

Further, there are other solutions which sidestep the problem without really addressing its cause. These options carry potentially steep cost. They are discussed separately, below.

Dealing with this kind of problem involves changing the set of variables included in the covariance matrix. If two variables are perfectly correlated with each other, then one may be deleted. Alternatively, principal components may be used to replace a set of collinear variables with one or more orthogonal components.

In regard to the asymptotic weight matrix, the linear dependency exists not between variables, but between elements of the moments (the means and variances and covariances or the correlations) which are being analyzed. This can occur in connection with modeling multiplicative interaction relationships between latent variables. Jöreskog and Yang (1996) show how moments of the interaction construct are linear functions of moments of the "main effect" constructs. Their article explores alternative approaches for estimating these models

Estimators of the asymptotic weight matrix converge much more slowly, so problems due to sampling variation can occur at much larger sample sizes (Muthén & Kaplan, 1985, 1992). Using an asymptotic weight matrix with polychoric correlations appears to compound the problem. Where sampling variation is the issue, Yung and Bentler (1994) have proposed a bootstrapping approach to estimating the asymptotic weight matrix, which may avoid the problem.

If the problem lies with the polychoric correlations, there may not be a good solution. One approach is to use a program, like EQS, that includes the option of deriving all polychoric correlations simultaneously, rather than one at a time (cf., Lee, Poon & Bentler, 1992). But be warned--Joop Hox reports that the computational burden is enormous, and it increases exponentially with the number of variables.

Ed Cook has experimented with an eigenvalue/eigenvector decomposition approach. If a covariance or correlation matrix is not positive definite, then one or more of its eigenvalues will be negative. After decomposing the correlation matrix into eigenvalues and eigenvectors, Ed Cook replaced the negative eigenvalues with small (.05) positive values, used the new values to compute a covariance matrix, then standardized the resulting matrix (diving by the square root of the diagonal values) so that the result was again was a correlation matrix. Ed reported that the bias resulting from this process appeared to be small.

Arbuckle, J. L. (1996). Full information estimation in the presence of incomplete data. In G. A. Marcoulides & R. E. Schumacker (Eds.), Advanced structural equation modeling: Issues and techniques (pp. 243-78). Mahwah, NJ: Lawrence Erlbaum.

Gerbing, D. W., & Anderson, J. C. (1987). Improper solutions in the analysis of covariance structures: Their interpretability and a comparison of alternate respecifications. Psychometrika, 52(1--March), 99-111.

Jöreskog, K. G., & Yang F. [now Fan Yang Jonsson] (1996). Nonlinear structural equation models: The Kenny-Judd model with interaction effects. In G. A. Marcoulides & R. E. Schumacker (Eds.), Advanced structural equation modeling: Issues and techniques (pp. 57-88). Mahwah, NJ: Lawrence Erlbaum.

Lee, S.-Y., Poon, W.-Y., & Bentler, P. M. (1992). Structural equation models with continuous and polytomous variables. Psychometrika, 57(1--March), 89-105.

Muthén, B. & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38, 171-89.

Muthén, B. & Kaplan, D. (1992). A comparison of some methodologies for the factor analysis of non-normal Likert variables: A note on the size of the model. British Journal of Mathematical and Statistical Psychology, 45, 19-30.

Wothke, W. (1993). Nonpositive definite matrices in structural modeling. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 256-93). Newbury Park, CA: Sage.

Yung, Y.-F., & Bentler, P. M. (1994). Bootstrap-corrected ADF test statistics in covariance structure analysis. British Journal of Mathematical and Statistical Psychology, 47, 63-84.

http://www.gsu.edu/~mkteer/npdmatri.html Return to the SEMNET FAQ home page.

Return to Ed Rigdon's home page.

Last updated: June 11, 1997