Formal statistics and informal data analysis, or why laziness should be discouraged

In: Statistica Neerlandica, Volume 42, Issue 2, p. 83-90

Abstract

The paper gives a personal view on how empirical research and subsequent data analysis and reporting should ideally proceed. Although the introduction concedes that practical limitations and imperfect knowledge usually impose restrictions on our possibilities to approximate this ideal situation, it is argued that investigators and statistical consultants should make a strong effort to come close to it. At the stages of design, analysis, and reporting, this requires careful consideration of the selection of the units of analysis, the choice of variables to be included, their measurement process, and the extent to which relevant assumptions are fulfilled. In so far as exploratory data analysis techniques offer the suggestion that they can produce results without such careful consideration, their use should be discouraged unless they are followed by a confirmatory analysis in which the claims of generalizability are better substantiated. Some protest is raised against the idea that the data themselves have a clear structure lying ready to be unveiled by the investigator by means of some numerical or spatial representation, which would permit to draw general conclusions.