Bias within systematic and non-systematic literature reviews: the case of the Balanced Scorecard

“What is the impact of bias when selecting studies for a literature review as well as interpreting the results from these selected studies for non-systematic literature reviews as compared to systematic literature reviews?” To measure and compare the impact of bias on both systematic and non-systematic literature reviews a non-systematic case study was selected. This case study was repeated using the same aim and boundaries, but with a systematic approach towards the reviewing process. The case study and the repeat review were then compared to each other. Sources of bias and their impact were then identified for both the case study and the repeat review. The results showed that both the non-systematic case study and the systematic repeat review were impacted by bias when selecting and interpreting studies (place of publication bias, citation bias, etc.) potentially affecting a significant number of the included studies in the repeat review and the case study. This evidence refutes claims made by authors like Tranfield, David & Palminder (2003) which deemed the systematic literature review to be superior. However, improving the non-systematic case study is hard, because it lacks a clear search strategy while the systematic repeat review does provide this.