Modification Indices in SEM Model

Modification indices are part of the analysis that suggest model alterations to achieve a better fit to the data. Making changes via modification indices should be done very carefully and have justification. Blindly using the modification indices to achieve a better model fit can capitalize on chance and result in model adjustments that make no sense.With a large sample size, even a small discrepancy can trigger a modification indices flag.With modification indices, improve- ment in fit is measured by the reduction of chi-square. In AMOS, modification indices are concerned with adding additional covariances within a construct’s indicators or relationship paths between constructs. Note that the modification indices option in AMOS will not run if you have missing data.

In the output of the modification indices, AMOS will list potential changes by add- ing covariances between error terms and also presenting possible relationships between constructs (listed as regression weights). In a CFA, we are concerned only with covari- ances between error terms. All other modification indices for a CFA are inappropriate. The modification indices will have an initial column that simply says “MI”, which stands for modification indices threshold. This value presented under the MI heading is the reduc- tion of the chi-square value by adding an additional covariance. A modification indices threshold value needs to be at least be 3.84 to show a significant difference. If you look at a chi-square difference table (see the Appendix), a reduction of one degree of freedom by adding a covariance will require a change in a chi-square value of 3.84 to be significant on the .05 level. In AMOS, the default threshold is a value of 4 where any potential modifica- tion below this value is not presented. The next column in the modification indices output is “Par change”. This column is the estimated change in the new parameter added when the model is altered.

As stated, with a CFA, you are concerned only with modification indices related to the covariances, but to clarify this, covariances of indicators within a construct. It is inappropriate to covary indicators across constructs even though the modification indices will suggest it. Below are some suggestions of “dos and don’ts” with modification indices using our previous example of Customer Delight and Positive Word of Mouth.

Figure 4.3 Modifications With a CFA

In My Opinion: Correlating Error Terms Within a Construct

In instances where an unobserved variable has multiple indicators, you might want to correlate the error terms if it is theoretically justifiable and helps to explain the variance within the construct. This is often done when the error of one indicator helps in know- ing the error associated with another indicator. Frequently, indicators within a construct are very similar to one another and there is redundancy between the indicators.

How big does the modification indices value need to be to add another covariance? I rarely consider any valid modification suggestion if the chi-square value is not chang- ing by at least 10. A chi-square value of 10 at the expense of one degree of freedom is almost significant at a .001 level. Saying that, I do not always covary error terms even when a modification suggestion is over 10. I look for instances where the change is extreme. It is not uncommon to have a modification suggestion greater than 20 in large models.You want to explore what modifications will actually make an impact in regards to your model. A modification suggestion with a chi-square change of 7 will probably show relatively little influence or change in the model itself. Correlating error terms can strengthen factor loadings within a construct, and in rare cases I have seen it actually lower factor loadings. Ultimately, correlating an error term is about explaining com- mon variance across indicators instead of treating them as independents which could hurt your overall model fit.

If you have multiple error terms within a construct that are showing high modifi- cation indices levels, then you have cause for concern. This is a sign that you are not capturing different parts of the unobservable construct. The unexplained error in the construct is very similar across the indicators. This could be a sign that your indicators are extremely redundant and you are not really asking different questions.You may find that model fit is substantially increased when you correlate multiple error terms within a construct, but you are also raising other concerns that your indicators are lacking in regards to capturing the unobservable. An equal concern for multiple high modification indices within a construct is the possibility that this shared unexplained error is the result of a method bias in how the data was collected.

Some researchers from a traditional perspective think that you should not corre- late error terms even within the construct (Gerbing and Anderson 1984).The rationale is that the need for a covariance between error terms suggests there is an unknown common source or construct that is the result of this common unexplained variance. Potential remedies instead of correlating error terms were to include an “unknown” factor into the model, which ultimately was equally problematic. In my opinion, the idea that you should never correlate error terms is too restrictive. Complex models and constructs with numerous indicators are inevitably going to have error terms that are eligible to be correlated. Before correlating an error term, you need to assess if this high modification index is a result of redundancy in your indicators or if you have a poten- tially larger problem of a missing influence or a method bias.

Source: Thakkar, J.J. (2020). “Procedural Steps in Structural Equation Modelling”. In: Structural Equation Modelling. Studies in Systems, Decision and Control, vol 285. Springer, Singapore.