Confidence

Explanations > Social Research > Statistical principles > Confidence

Description | Example | Discussion | See also

Description

Due to variation in the world, analyses and forecasts can seldom be done with complete certainty. They can often, however, be made with a degree of confidence that can be described numerically.

For example if a random man in a known population comes into a room, you might be able to say that there is a 95% chance that he will be 150 cm tall, plus or minus 9 cm. In other words, if you selected 100 random people, 95 are likely to be within this range.

Confidence thus has two aspects:

A range of possible measures (eg. 141 cm to 159 cm).

These are the confidence limits.
The difference is called the confidence interval (CI).
A probability that the forecast value lies within these limits.

The confidence interval is usually measured as a plus-or-minus deviation from the mean (assuming a symmetrical distribution).

The probability of a conclusion or forecast being correct may be expressed as a percentage is often expressed as decimal number, which is usually reversed, for example a 95% confidence is written as .05.

You can thus say that a person is six feet high, plus or minus half an inch, with a confidence in this assessment of 50%. This means that if you assessed 100 people of the same height (or the same person on different days), you would be right only 50% of the time.

Discussion

A critical question is 'what is statistically significant?' For a researcher, this means being able to say that their experiment has proved something true or false.

In sampling, the higher the value of the standard error, the greater the sampling variability and hence the wider the confidence interval.

Statistician R. A. Fisher proposed that a confidence 95% (p=0.05) could be claimed as 'statistically significant' and this is commonly used. For strong confidence, a 99% (0.01) figure may be sought.

On graphs, confidence limits may be shown as whisker lines, extending above and below each point or bar top, although these whiskers may also show standard deviation, standard error or some other measure of spread. It is therefore important to indicate what whiskers mean.

Confidence and statistical significance is all well and good, but significance does not always equate to usefulness and the effect is often also calculated to determine this. Thus you can have a high confidence that a change is statistically significant, but that change can be so tiny as be not very significant in real terms.

As many experiments work with samples and sample means, this principle works here also.