Generally in statistics, bias results when there is a discrepancy between the expected value and the computed value due to a systematic error. It is a deviation which is not the result of randomness. In comparison to random errors, systematic errors can be potentially more serious because they cannot necessarily be avoided by implementing minor changes.
An estimator is only considered unbiased when the mean of a sampling distribution is equal to the population value of interest.
For example, suppose there is an upcoming federal election you want to survey individuals in Ontario to make future predictions about the election outcome. You survey 6 communities in Ontario and randomly sampled 500 individuals in each community. After analyzing the data, you realized that preference for a particular political party is completely dependent upon the community individuals resided in.
This sample design exemplifies a sampling error because not all individuals in Ontario had an equal probability of being sampled. Thus, the results are biased in that they only represent a small proportion of the province. Furthermore, preference accompanies communities and this cannot be changed by sampling larger sample sizes in each area. This is a systematic error which cannot be avoided and thus, the sampling criterion needs to be modified.
In statistics, there are various types of biases which can arise. Some examples in addition to sampling bias include spectrum bias, which results from evaluations of already biased patient samples in diagnostic tests, and omitted-variable bias. Furthermore, some biases apply specifically to hypothesis testing.
In statistics, bias can be inevitable in some cases. However, recognizing potential flaws in a system or accounting for bias in experiments, can assist in minimizing their impact. Furthermore, multiple measures are often taken in an attempt to try and eliminate as much bias as possible. For example, when sampling there are a multitude of different random sampling methods which can be implemented for different experimental designs.
Image credit: Wikimedia Commons© BrainMass Inc. brainmass.com December 10, 2018, 2:14 am ad1c9bdddf