Principles for Examining Predictive Validity: The Case of Information Systems Spending Forecasts.

Author List: Collopy, Fred; Adya, Monica; Armstrong, J. Scott;
Information Systems Research, 1994, Volume 5, Issue 2, Page 170-179.
Research over two decades has advanced the knowledge of how to assess predictive validity. We believe this has value to information systems (IS) researchers. To demonstrate, we used a widely cited study of IS spending. In that study, price-adjusted diffusion models were proposed to explain and to forecast aggregate U.S. information systems spending. That study concluded that such models would produce more accurate forecasts than would simple linear trend extrapolation. However, one can argue that the validation procedure provided an advantage to the diffusion models. We reexamined the results using an alternative validation procedure based on three principles extracted from forecasting research: (1) use ex ante (out-of-sample) performance rather than the fit to the historical data, (2) use well-accepted models as a basis for comparison, and (3) use an adequate sample of forecasts. Validation using this alternative procedure did confirm the importance of the price- adjustment, but simple trend extrapolations were found to be more accurate than the price-adjusted diffusion models.
Keywords: adjusted diffusion models; Brown's linear exponential smoothing; Combined forecasts; Damped trend exponential smoothing; Diffusion; Extrapolation; Forecast accuracy; Information systems spending; Price
Algorithm:

List of Topics

#226 0.353 models linear heterogeneity path nonlinear forecasting unobserved alternative modeling methods different dependence paths efficient distribution probabilities demonstrate observed heterogeneous probability
#124 0.212 validity reliability measure constructs construct study research measures used scale development nomological scales instrument measurement researchers developed validation discriminant results
#215 0.131 data classification statistical regression mining models neural methods using analysis techniques performance predictive networks accuracy method variables prediction problem measure
#77 0.123 information systems paper use design case important used context provide presented authors concepts order number various underlying implementation framework nature
#21 0.082 research information systems science field discipline researchers principles practice core methods area reference relevance conclude set focus propose perspective inquiry
#49 0.055 adoption diffusion technology adopters innovation adopt process information potential innovations influence new characteristics early adopting set compatibility time initial current