Research and Statistics

A 10-year-old boy recently received a diagnosis of diabetes. His mother feels that he would benefit from learning about coping styles and behavior management techniques from other children who have diabetes. However, you are unaware of the effectiveness of this type of group intervention.A parent is concerned about her child developing asthma because he has had shortness of breath and wheezing at times. The child's older sister recently received an asthma diagnosis, based on similar symptoms. You are curious about the prevalence of asthma in siblings.Generalizability refers to the extent to which the results of a study apply to individuals and circumstances beyond those studied. (1) Commonly referred to as external validity, generalizability is the degree to which a given study's findings can be extrapolated to another population. Results of a study are considered generalizable if they have relevant characteristics of and implications for more individuals than those in the sample studied.Generalizability is a crucial aspect of both research design and interpretation of research in practice. In research design, the researcher must balance achieving strong internal validity with generalizability. Generalizability is one important criterion in interpreting study results and deciding whether study findings would apply to those in an individual's medical practice.Internal validity (the ability to make conclusions about relationships between variables, based on study design and methods) is closely tied to generalizability, but efforts to improve internal validity can pose a threat to generalizability. Both internal validity and generalizability are critical elements in research design but can be difficult to achieve simultaneously.Randomized, controlled trials generally have strong internal validity due to the nature of the treatment versus control group design. However, in an effort to make the two groups as similar as possible at baseline, this study design often reduces generalizability. In the case of the patient who has diabetes, a randomized, controlled trial may show that an intervention to improve coping and behavior related to diabetes management is effective. Generalizability may be reduced if the intervention was effective for only a specific group whose characteristics were very similar to those of a comparison group.Sample selection is an important component of both internal validity and generalizability. To achieve balance between them, the goal in sample selection should be to increase the heterogeneity of the sample without minimizing the internal validity of the study. One method of improving generalizability is to use methods such as random sampling. A random sample reduces any bias in the selection of the study sample, which is one of the major threats to external validity. Selection bias refers to an error or problem with the way in which participants for a study are selected.Because such sampling techniques often are not possible, it is imperative to consider a study's eligibility (or inclusion) criteria. Achieving generalizability to a wider population requires broad eligibility criteria. Examples of broad eligibility criteria include children of various ages, of diverse backgrounds, and from different patient care settings. However, this mix might not be feasible to achieve and reduces the precision of the study. Again, it is important to consider the balance of generalizability and internal validity. Broad eligibility criteria tend to improve generalizability; narrow eligibility criteria may improve the precision and feasibility of a study. (1)Another important threat to generalizability is the setting in which the research is conducted. When designing a study based solely in a hospital setting, there are implied limits in the generalizability of the study's findings because of the uniqueness of the type of setting in terms of treatment and practices. Such a limitation could be addressed by conducting research in multiple or variable settings, including clinics and private practices. (2)When reading the results of studies published in the medical literature, it is important to consider the study's generalizability. In interpreting study results, the clinician should ask questions such as: How is the sample drawn? What are the eligibility criteria? How are the setting and sample of the study related to my own setting and population?In your investigation of whether coping and behavior modification group interventions are effective in patients who have diabetes, you discover one study that indicates that a 10-week group session improved patients’ diabetes management skills significantly. However, on closer examination, you determine that the study eligibility criteria limited the sample to teenagers 16 years of age and older, who had family members who had diabetes and who lived in a predominantly rural community with limited access to health-care resources. This could raise concerns about the generalizability of this study's results to your own patient, who is 10 years old, has no family member who has diabetes, and lives in a relatively wealthy suburban neighborhood.In looking at the research available to determine the prevalence of asthma in siblings, you find a study indicating that 40% of children who received the diagnosis had a sibling who had the same diagnosis. You feel that rate to be considerably higher than what you would expect based on your years of experience as a practitioner. In looking at the sampling techniques, you determine that the authors used a convenience sample based on 300 children who had at least one sibling presenting at a clinic in one single month. You might have concerns about the generalizability of this study because the sample used in the study does not reflect the larger population, including the population of your own patients.