Questioning our fundamental assumptions: Scientific measures of reliability

Loading...
Thumbnail Image

Date

2008

Journal Title

Journal ISSN

Volume Title

Publisher

School of Education, UWI, St. Augustine

Abstract

Have you every wondered why many phenomena that thousands of people believe in and claim to have experienced cannot be 'proven' by science; phenomena such as effects of the moon on human behaviour, and supernatural and paranormal events such as ESP, remote viewing, and out-of-body experiences. Typically, scientific results of well-designed experiments report correlations that show any such unusual human-contextual interactions are no better than chance, for example "Ivan Kelly, James Rotton and Roger Culver (1996) examined over 100 studies on lunar effects and concluded that the studies have failed to show a reliable and significant correlation (i.e., one not likely due to chance).... If so many studies have failed to prove a significant correlation between the full moon and anything, why do so many people believe in these lunar myths?" (Carroll, 2006). One simple reason that correlation studies show no significant results might not be because there is nothing there to measure but because the assumptions on which correlations are based attribute such unusual human-contextual interactions to randomness. Many users of correlation are unaware of these assumptions and many statisticians rarely question them. It might be said of these experiments that if we repeatedly do something the wrong way we will continue to reliably get the wrong answer. This paper explains very simply, for a non-statistical audience, the random-error assumption of classical test theory that is used to scientifically measure reliability of such phenomena. A study of 1,331 Grade 9 children in 43 Jamaican schools is presented to demonstrate that classic correlational measures of reliability do not recognize unusual but reliable human-contextual interactions recorded by these children. A simple alternative constructivist correlational measure is demonstrated, which is sensitive to such unusual but reliable human-contextual interactions. The significance of this paper is that it is fundamental to correlation studies in education, particularly in non-standard Caribbean populations

Description

Table of Contents

Keywords

Educational research, Measurement instruments, Reliability, Jamaica

Citation

Bastick, T. (2008). Questioning our fundamental assumptions: Scientific measures of reliability. In L. Quamina-Aiyejina (Ed.), Reconceptualising the agenda for education in the Caribbean: Proceedings of the 2007 Biennial Cross-Campus Conference in Education, April 23-26, 2007, School of Education, UWI, St. Augustine, Trinidad and Tobago (pp. 531-545). St. Augustine, Trinidad: School of Education, UWI.