In medical and public health research, many studies are observational.In these studies, the treatment is not randomly assigned to participants.Therefore, the differences in outcomes between treatment groups could be due to imbalances of characteristics that are related to the outcome of interest prior to the treatment.Herein we investigate how we can use propensity scores, the conditional probability of receiving treatment given the observed information, to make valid causal inference in observational studies.Theoretical results for the bias are given for linear response models that use the propensity score as a linear covariate.The bias depends on the relationship between the propensity score, the treatment indicator and the functional form of the covariate.Various methods for estimation of the treatment effect are explored.We show that the bias is influenced by the overlap in the distributions