# Simultaneous equations model

**Simultaneous equations models** are a type of statistical model in which the dependent variables are functions of other dependent variables, rather than just independent variables.^{[1]} This means some of the explanatory variables are jointly determined with the dependent variable, which in economics usually is the consequence of some underlying equilibrium mechanism. For instance, in the simple model of supply and demand, price and quantity are jointly determined.^{[2]}

Simultaneity poses challenges for the estimation of the statistical parameters of interest, because the Gauss–Markov assumption of strict exogeneity is violated. And while it would be natural to estimate all simultaneous equations at once, this often leads to a computationally costly non-linear optimization problem even for the simplest system of linear equations.^{[3]} This situation prompted the development, spearheaded by the Cowles Commission in the 1940s and 1950s,^{[4]} of various techniques that estimate each equation in the model seriatim, most notably limited information maximum likelihood and two-stage least squares.^{[5]}

## Structural and reduced form[edit]

Suppose there are *m* regression equations of the form

where *i* is the equation number, and *t* = 1, ..., *T* is the observation index. In these equations *x _{it}* is the

*k*1 vector of exogenous variables,

_{i}×*y*is the dependent variable,

_{it}*y*is the

_{−i,t}*n*1 vector of all other endogenous variables which enter the

_{i}×*i*

^{th}equation on the right-hand side, and

*u*are the error terms. The “−

_{it}*i*” notation indicates that the vector

*y*may contain any of the

_{−i,t}*y*’s except for

*y*(since it is already present on the left-hand side). The regression coefficients

_{it}*β*and

_{i}*γ*are of dimensions

_{i}*k*1 and

_{i}×*n*1 correspondingly. Vertically stacking the

_{i}×*T*observations corresponding to the

*i*

^{th}equation, we can write each equation in vector form as

where *y _{i}* and

*u*are

_{i}*T×*1 vectors,

*X*is a

_{i}*T×k*matrix of exogenous regressors, and

_{i}*Y*is a

_{−i}*T×n*matrix of endogenous regressors on the right-hand side of the

_{i}*i*

^{th}equation. Finally, we can move all endogenous variables to the left-hand side and write the

*m*equations jointly in vector form as

This representation is known as the **structural form**. In this equation *Y* = [*y*_{1} *y*_{2} ... *y _{m}*] is the

*T×m*matrix of dependent variables. Each of the matrices

*Y*is in fact an

_{−i}*n*-columned submatrix of this

_{i}*Y*. The

*m×m*matrix Γ, which describes the relation between the dependent variables, has a complicated structure. It has ones on the diagonal, and all other elements of each column

*i*are either the components of the vector

*−γ*or zeros, depending on which columns of

_{i}*Y*were included in the matrix

*Y*. The

_{−i}*T×k*matrix

*X*contains all exogenous regressors from all equations, but without repetitions (that is, matrix

*X*should be of full rank). Thus, each

*X*is a

_{i}*k*-columned submatrix of

_{i}*X*. Matrix Β has size

*k×m*, and each of its columns consists of the components of vectors

*β*and zeros, depending on which of the regressors from

_{i}*X*were included or excluded from

*X*. Finally,

_{i}*U*= [

*u*

_{1}

*u*

_{2}...

*u*] is a

_{m}*T×m*matrix of the error terms.

Postmultiplying the structural equation by Γ^{ −1}, the system can be written in the **reduced form** as

This is already a simple general linear model, and it can be estimated for example by ordinary least squares. Unfortunately, the task of decomposing the estimated matrix into the individual factors Β and Γ^{ −1} is quite complicated, and therefore the reduced form is more suitable for prediction but not inference.

### Assumptions[edit]

Firstly, the rank of the matrix *X* of exogenous regressors must be equal to *k*, both in finite samples and in the limit as *T* → ∞ (this later requirement means that in the limit the expression should converge to a nondegenerate *k×k* matrix). Matrix Γ is also assumed to be non-degenerate.

Secondly, error terms are assumed to be serially independent and identically distributed. That is, if the *t*^{th} row of matrix *U* is denoted by *u*_{(t)}, then the sequence of vectors {*u*_{(t)}} should be iid, with zero mean and some covariance matrix Σ (which is unknown). In particular, this implies that E[*U*] = 0, and E[*U′U*] = *T* Σ.

Lastly, the identification conditions require that the number of unknowns in this system of equations should not exceed the number of equations. More specifically, the *order condition* requires that for each equation *k _{i} + n_{i} ≤ k*, which can be phrased as “the number of excluded exogenous variables is greater or equal to the number of included endogenous variables”. The

*rank condition*of identifiability is that rank(Π

_{i0}) =

*n*, where Π

_{i}_{i0}is a (

*k − k*)×

_{i}*n*matrix which is obtained from Π by crossing out those columns which correspond to the excluded endogenous variables, and those rows which correspond to the included exogenous variables.

_{i}## Estimation[edit]

### Two-stages least squares (2SLS)[edit]

The simplest and the most common estimation method for the simultaneous equations model is the so-called two-stage least squares method,^{[6]} developed independently by Theil (1953) and Basmann (1957).^{[7]}^{[8]} It is an equation-by-equation technique, where the endogenous regressors on the right-hand side of each equation are being instrumented with the regressors *X* from all other equations. The method is called “two-stage” because it conducts estimation in two steps:^{[6]}

*Step 1*: Regress*Y*on_{−i}*X*and obtain the predicted values ;*Step 2*: Estimate*γ*,_{i}*β*by the ordinary least squares regression of_{i}*y*on and_{i}*X*._{i}

If the *i*^{th} equation in the model is written as

where *Z _{i}* is a

*T×*(

*n*) matrix of both endogenous and exogenous regressors in the

_{i}+ k_{i}*i*

^{th}equation, and

*δ*is an (

_{i}*n*)-dimensional vector of regression coefficients, then the 2SLS estimator of

_{i}+ k_{i}*δ*will be given by

_{i}^{[6]}

where *P* = *X* (*X* ′*X*)^{−1}*X* ′ is the projection matrix onto the linear space spanned by the exogenous regressors *X*.

### Indirect least squares[edit]

Indirect least squares is an approach in econometrics where the coefficients in a simultaneous equations model are estimated from the reduced form model using ordinary least squares.^{[9]}^{[10]} For this, the structural system of equations is transformed into the reduced form first. Once the coefficients are estimated the model is put back into the structural form.

### Limited information maximum likelihood (LIML)[edit]

The “limited information” maximum likelihood method was suggested M. A. Girshick in 1947,^{[11]} and formalized by T. W. Anderson and H. Rubin in 1949.^{[12]} It is used when one is interested in estimating a single structural equation at a time (hence its name of limited information), say for observation i:

The structural equations for the remaining endogenous variables Y_{−i} are not specified, and they are given in their reduced form:

Notation in this context is different than for the simple IV case. One has:

- : The endogenous variable(s).
- : The exogenous variable(s)
- : The instrument(s) (often denoted )

The explicit formula for the LIML is:^{[13]}

where *M* = *I − X* (*X* ′*X*)^{−1}*X* ′, and *λ* is the smallest characteristic root of the matrix:

where, in a similar way, *M _{i}* =

*I − X*(

_{i}*X*′

_{i}*X*)

_{i}^{−1}

*X*′.

_{i}In other words, *λ* is the smallest solution of the generalized eigenvalue problem, see Theil (1971, p. 503):

#### K class estimators[edit]

The LIML is a special case of the K-class estimators:^{[14]}

with:

Several estimators belong to this class:

- κ=0: OLS
- κ=1: 2SLS. Note indeed that in this case, the usual projection matrix of the 2SLS
- κ=λ: LIML
- κ=λ - α (n-K): Fuller (1977) estimator.
^{[15]}Here K represents the number of instruments, n the sample size, and α a positive constant to specify. A value of α=1 will yield an estimator that is approximately unbiased.^{[14]}

### Three-stage least squares (3SLS)[edit]

The three-stage least squares estimator was introduced by Zellner & Theil (1962).^{[16]}^{[17]} It can be seen as a special case of multi-equation GMM where the set of instrumental variables is common to all equations.^{[18]} If all regressors are in fact predetermined, then 3SLS reduces to seemingly unrelated regressions (SUR). Thus it may also be seen as a combination of two-stage least squares (2SLS) with SUR.

## Using cross-equation restrictions to achieve identification[edit]

In simultaneous equations models, the most common method to achieve identification is by imposing within-equation parameter restrictions.^{[19]} Yet, identification is also possible using cross equation restrictions.

To illustrate how cross equation restrictions can be used for identification, consider the following example from Wooldridge ^{[19]}

y_{1} = γ_{12} y_{2} + δ_{11} z_{1} + δ_{12} z_{2} + δ_{13} z_{3} + u_{1}

y_{2} = γ_{21} y_{1} + δ_{21} z_{1} + δ_{22} z_{2} + u_{2}

where z's are uncorrelated with u's and y's are endogenous variables. Without further restrictions, the first equation is not identified because there is no excluded exogenous variable. The second equation is just identified if δ_{13}≠0, which is assumed to be true for the rest of discussion.

Now we impose the cross equation restriction of δ_{12}=δ_{22}. Since the second equation is identified, we can treat δ_{12} as known for the purpose of identification. Then, the first equation becomes:

y_{1} - δ_{12} z_{2} = γ_{12} y_{2} + δ_{11} z_{1} + δ_{13} z_{3} + u_{1}

Then, we can use (z_{1},z_{2},z_{3}) as instruments to estimate the coefficients in the above equation since there are one endogenous variable (y_{2}) and one excluded exogenous variable (z_{2}) on the right hand side. Therefore, cross equation restrictions in place of within-equation restrictions can achieve identification.

## Applications in social science[edit]

Across fields and disciplines simultaneous equation models are applied to various observational phenomena. These equations are applied when phenomena are assumed to be reciprocally causal. The classic example is supply and demand in economics. In other disciplines there are examples such as candidate evaluations and party identification^{[20]} or public opinion and social policy in political science;^{[21]}^{[22]} road investment and travel demand in geography;^{[23]} and educational attainment and parenthood entry in sociology or demography.^{[24]} The simultaneous equation model requires a theory of reciprocal causality that includes special features if the causal effects are to be estimated as simultaneous feedback as opposed to one-sided 'blocks' of an equation where a researcher is interested in the causal effect of X on Y while holding the causal effect of Y on X constant, or when the researcher knows the exact amount of time it takes for each causal effect to take place, i.e., the length of the causal lags. Instead of lagged effects, simultaneous feedback means estimating the simultaneous and perpetual impact of X and Y on each other. This requires a theory that causal effects are simultaneous in time, or so complex that they appear to behave simultaneously; a common example are the moods of roommates.^{[25]} To estimate simultaneous feedback models a theory of equilibrium is also necessary – that X and Y are in relatively steady states or are part of a system (society, market, classroom) that is in a relatively stable state.^{[26]}

## See also[edit]

## References[edit]

**^**Martin, Vance; Hurn, Stan; Harris, David (2013).*Econometric Modelling with Time Series*. Cambridge University Press. p. 159. ISBN 978-0-521-19660-4.**^**Maddala, G. S.; Lahiri, Kajal (2009).*Introduction to Econometrics*(Fourth ed.). Wiley. pp. 355–357. ISBN 978-0-470-01512-4.**^**Quandt, Richard E. (1983). "Computational Problems and Methods". In Griliches, Z.; Intriligator, M. D. (eds.).*Handbook of Econometrics*. Volume I. North-Holland. pp. 699–764. ISBN 0-444-86185-8.**^**Christ, Carl F. (1994). "The Cowles Commission's Contributions to Econometrics at Chicago, 1939–1955".*Journal of Economic Literature*.**32**(1): 30–59. JSTOR 2728422.**^**Johnston, J. (1971). "Simultaneous-equation Methods: Estimation".*Econometric Methods*(Second ed.). New York: McGraw-Hill. pp. 376–423. ISBN 0-07-032679-7.- ^
^{a}^{b}^{c}Greene, William H. (2002).*Econometric analysis*(5th ed.). Prentice Hall. pp. 398–99. ISBN 0-13-066189-9.CS1 maint: ref=harv (link) **^**Basmann, R. L. (1957). "A generalized classical method of linear estimation of coefficients in a structural equation".*Econometrica*.**25**(1): 77–83. doi:10.2307/1907743. JSTOR 1907743.CS1 maint: ref=harv (link)**^**Theil, Henri (1971).*Principles of Econometrics*. New York: John Wiley.CS1 maint: ref=harv (link)**^**Park, S-B. (1974) "On Indirect Least Squares Estimation of a Simultaneous Equation System",*The Canadian Journal of Statistics / La Revue Canadienne de Statistique*, 2 (1), 75–82 JSTOR 3314964**^**Vajda, S.; Valko, P.; Godfrey, K.R. (1987). "Direct and indirect least squares methods in continuous-time parameter estimation".*Automatica*.**23**(6): 707–718. doi:10.1016/0005-1098(87)90027-6.**^**First application by Girshick, M. A.; Haavelmo, Trygve (1947). "Statistical Analysis of the Demand for Food: Examples of Simultaneous Estimation of Structural Equations".*Econometrica*.**15**(2): 79–110. doi:10.2307/1907066. JSTOR 1907066.**^**Anderson, T.W.; Rubin, H. (1949). "Estimator of the parameters of a single equation in a complete system of stochastic equations".*Annals of Mathematical Statistics*.**20**(1): 46–63. doi:10.1214/aoms/1177730090. JSTOR 2236803.CS1 maint: ref=harv (link)**^**Amemiya, Takeshi (1985).*Advanced Econometrics*. Cambridge, Massachusetts: Harvard University Press. p. 235. ISBN 0-674-00560-0.CS1 maint: ref=harv (link)- ^
^{a}^{b}Davidson, Russell; MacKinnon, James G. (1993).*Estimation and inference in econometrics*. Oxford University Press. p. 649. ISBN 0-19-506011-3.CS1 maint: ref=harv (link) **^**Fuller, Wayne (1977). "Some Properties of a Modification of the Limited Information Estimator".*Econometrica*.**45**(4): 939–953. doi:10.2307/1912683. JSTOR 1912683.CS1 maint: ref=harv (link)**^**Zellner, Arnold; Theil, Henri (1962). "Three-stage least squares: simultaneous estimation of simultaneous equations".*Econometrica*.**30**(1): 54–78. doi:10.2307/1911287. JSTOR 1911287.CS1 maint: ref=harv (link)**^**Kmenta, Jan (1986). "System Methods of Estimation".*Elements of Econometrics*(Second ed.). New York: Macmillan. pp. 695–701.**^**Hayashi, Fumio (2000). "Multiple-Equation GMM".*Econometrics*. Princeton University Press. pp. 276–279.- ^
^{a}^{b}Wooldridge, J.M., Econometric Analysis of Cross Section and Panel Data, MIT Press, Cambridge, Mass. **^**Page, Benjamin I.; Jones, Calvin C. (1979-12-01). "Reciprocal Effects of Policy Preferences, Party Loyalties and the Vote".*American Political Science Review*.**73**(4): 1071–1089. doi:10.2307/1953990. ISSN 0003-0554. JSTOR 1953990.**^**Wlezien, Christopher (1995-01-01). "The Public as Thermostat: Dynamics of Preferences for Spending".*American Journal of Political Science*.**39**(4): 981–1000. doi:10.2307/2111666. JSTOR 2111666.**^**Breznau, Nate (2016-07-01). "Positive Returns and Equilibrium: Simultaneous Feedback Between Public Opinion and Social Policy".*Policy Studies Journal*.**45**(4): 583–612. doi:10.1111/psj.12171. ISSN 1541-0072.**^**Xie, F.; Levinson, D. (2010-05-01). "How streetcars shaped suburbanization: a Granger causality analysis of land use and transit in the Twin Cities".*Journal of Economic Geography*.**10**(3): 453–470. doi:10.1093/jeg/lbp031. hdl:11299/179996. ISSN 1468-2702.**^**Marini, Margaret Mooney (1984-01-01). "Women's Educational Attainment and the Timing of Entry into Parenthood".*American Sociological Review*.**49**(4): 491–511. doi:10.2307/2095464. JSTOR 2095464.**^**Wong, Chi-Sum; Law, Kenneth S. (1999-01-01). "Testing Reciprocal Relations by Nonrecursive Structuralequation Models Using Cross-Sectional Data".*Organizational Research Methods*.**2**(1): 69–87. doi:10.1177/109442819921005. ISSN 1094-4281.**^**2013. “Reverse Arrow Dynamics: Feedback Loops and Formative Measurement.” In*Structural Equation Modeling: A Second Course*, edited by Gregory R. Hancock and Ralph O. Mueller, 2nd ed., 41–79. Charlotte, NC: Information Age Publishing

## Further reading[edit]

- Fomby, Thomas B.; Hill, R. Carter; Johnson, Stanley R. (1984). "Simultaneous Equations Models".
*Advanced Econometric Methods*. New York: Springer. pp. 437–552. ISBN 0-387-90908-7. - Maddala, G. S.; Lahiri, Kajal (2009). "Simultaneous Equations Models".
*Introduction to Econometrics*(Fourth ed.). New York: Wiley. pp. 355–400. ISBN 978-0-470-01512-4. - Ruud, Paul A. (2000). "Simultaneous Equations".
*An Introduction to Classical Econometric Theory*. Oxford University Press. pp. 697–746. ISBN 0-19-511164-8. - Sargan, Denis (1988).
*Lectures on Advanced Econometric Theory*. Oxford: Basil Blackwell. pp. 68–89. ISBN 0-631-14956-2. - Wooldridge, Jeffrey M. (2013). "Simultaneous Equations Models".
*Introductory Econometrics*(Fifth ed.). South-Western. pp. 554–582. ISBN 978-1-111-53104-1.

## External links[edit]

- About.com:economics Online dictionary of economics, entry for ILS
- Lecture on the Identification Problem in 2SLS, and Estimation on YouTube by Mark Thoma