4 Interpreting Test Results
Of course, just running the appropriate test is only part of the process for testing for concomitant variation. And, even if we find a relationship in the sample data, to what extent can we claim through statistical inference, that the relationship persists in the general population?
The second part of the process is interpreting the results of the test. The interpretation takes two assessments:
1) How likely is it that any relationship uncovered is a random statistical artifact of the sample? And 2) How strong is the relationship? The first refers to the likelihood of making what is known as a Type I Error. The second refers to the extent to which the Independent variable actually influences the Dependent variable.
How likely is it that any relationship uncovered was solely the result of a random aspect of the data (and, therefore, likely not true for the entire population)?
Since statistical tests of concomitant variation are based on random samples of the population, they must rely on probability theory to determine how likely the results from that particular sample reflect the population. Probability Theory indicates that it is possible, with varying degrees of likelihood, to randomly select samples that do not accurately represent the population.[1] So, for example, the data might suggest that a relationship between the variables exists, when the results are due only to what is known as random sampling error.
Researchers generally test to see if the results obtained were likely to result from this random sampling error.[2] The logic behind the process is:
a) Start with the idea that a relationship DOES NOT exist and, if the relationship does not exist, determine the probability of getting the calculated results of the test. For example, if a relationship does not exist, would the calculated results (or those even more extreme) randomly occur 50% of the time? 20%? 10%? 5%? 1%? Less?
b) If the results are ‘likely’ to occur randomly, then it would be unwise to assume that they occurred for any reason other than random chance and the idea that there is a relationship among the variables is not supported. In statistical lingo, this is called the ‘failure to reject the null hypothesis.’
However, if the results suggesting a relationship – when none was expected – are ‘unlikely’ to have occurred by random chance, then they probably happened for some other reason; namely, that a relationship DOES, in fact, exist between the two variables. [In statistical lingo, this is called the ‘rejection of the null hypothesis.’]
c) Yes, but, how unlikely do the results have to be to claim that any differences were NOT the result of random chance? Generally, researchers leave it up to their audience (readers) to make this determination for themselves. Standard values in the social sciences are: 10% (when the results are expected to be suggestive, rather than definitive), 5% (the typical and most common default value), and 1% (when the r