A post-hoc or observed power analysis is a test of power after the sample has been collected. Using the existing sample size and effect size, you can determine the power of a specific rela- tionship in your model. Using a post-hoc test for power assessment can often be misleading. These post-hoc power analyses can be (inappropriately) used as the reason or justification for non-significance. In essence, it was non-significant because the researcher did not achieve enough power from the estimated covariance matrix in predicting the relationship. More con- cerning is the relatively small sample size necessary to achieve acceptable power ratings in SEM. Taking our full structural model example used previously, if we wanted to calculate the post-hoc power for the Positive Word of Mouth estimates, we would need the number of predictors to Positive Word of Mouth (1), the squared multiple correlation of the construct (.254), the probability level (.05%), and the sample size (which was 500) in our example.
Figure 10.41 Full Structural Model to Test Post-Hoc Power
The post-hoc power analysis, based on this information, states that the power to assess the rela- tionship of Positive Word of Mouth is π = 1, or a 100% chance of finding this result again if rep- licated. If I repeat this post-hoc power test and reduce the sample to 100, the power is π = .99. Again, this is almost a 100% chance of finding the same result, with a fifth of the sample. Let’s take the sample down to 25 and run the calculation again. At a sample of 25, the power is π = .81, which is still considered an acceptable level of power to find an effect. To put it bluntly, post-hoc power analyses are not helpful and do not help to justify relationships found after the analysis. Power calculations are best suited in the design phase of research, not in the post-hoc analysis phase.
Source: Thakkar, J.J. (2020). “Procedural Steps in Structural Equation Modelling”. In: Structural Equation Modelling. Studies in Systems, Decision and Control, vol 285. Springer, Singapore.