Assessing the demand level of survey questionnaires

In this paper meta-analyses of experiments in question form and wording are presented. The demand level of survey questionnaires as perceived by respondents (Slovenian Housing Survey, 2005) was measured by a single question, which was varied in its question form and wording. Each respondent answered one version of the question in a split ballot experimental design. Multiple Classification Analysis was used to evaluate the effects of the question wording experiments and of respondent's characteristics (age and education) on the demand level of the survey as perceived by respondents. Results show that the terms "survey" and "survey questions on average" are equivalent to all respondents, and that the term "demanding" is understood and used differently by older respondents than the term "difficult". Formal balance has no effect on the estimated level of difficulty of the survey for respondents; however, labeling of extreme values has a strong effect on the estimated level of difficulty in the Housing survey.