Assessing the Performance of Analytical Procedures: A Best Case Scenario

• Published in: The Accounting review 1990-07, Vol.65 (3), p.557-577
• Main Authors: Wheeler, Stephen, Pany, Kurt
• Format: Article
• Language: English
• Subjects: Accounting methods; Accounting theory; Accounts receivable; Analysis; Auditing; Auditing procedures; Cost of sales; Effectiveness; Efficiency; Error rates; False negative errors; Financial audits; Modeling; Time series; Time series models
• ISSN: 0001-4826; 1558-7967

This study extends the existing research on the audit effectiveness of analytical procedures in a setting that used actual accounting data seeded with "material" simulated accounting errors. Five sample companies, whose revenues represented a wide range of time-series behavior, were selected to analyze the effects of eight commonly encountered accounting errors on 15 often-used analytical procedures (eight ratios and seven accounts). A "best case" scenario was induced by using, among other factors, single-industry companies, quarterly data, and more sophisticated expectation models than had been used in prior studies. The best predicting of six candidate models (four naive, a regression, and the Census X-11 time-series model) was used to generate quarterly predictions for comparison with actual data seeded with the largest of four empirically based materiality measures. Five investigation rules, including two simple percentage change rules and a statistical rule using three different alpha levels, were applied to prediction errors to determine whether error investigations were correctly signaled. The results of prediction model selection were dominated by X-11, followed by regression. Also, X-11 emerged as the "best" model more often for ratios than for accounts, while the reverse was true for the regression models. Overall, the analytical procedures examined did not signal (Type I and Type II error rates) very well when applied in isolation to quarterly data. However, when the quarterly signaling resulting were "annualized," and when an annual material error was seeded into an individual quarter's data, the results were much more encouraging. The lowest error rates were observed for instances where the primary substantive test would have been direct recomputation (i.e., interest and depreciation errors). The assertion of SAS No. 56 that income statement accounts should be more predictable than balance sheet accounts was contradicted, but the evidence is limited. The seeded quarterly material errors were generally swamped by prediction errors of the best-predicting expectation models. A significant correlation was observed between the ratio (prediction error/materiality) and the incidence of Type II signaling errors, indicating that this relationship might be used as a filter to determine when analytical procedures are likely to be effective audit tests.