The systematic review/meta‐analysis epidemic: a tale of glucocorticoid therapy in total knee arthroplasty

Optimal peri-operative pain management is one of the key factors to enhance recovery in orthopaedic procedures aiming at early mobilisation. It has demonstrated benefits including decreasing pulmonary and thromboembolic complications as well as facilitating early physiotherapy and rehabilitation 1. Consequently, we have seen numerous randomised controlled trials (RCT) on different analgesic approaches, but often with limited power to allow definite recommendations. Therefore, as generally accepted, pooling of data from available RCTs into systematic reviews and meta-analyses has become a gold standard 2. However, at the same time, it has been realised that interpretation of systematic reviews and meta-analyses may be hindered by poor design and inappropriate inclusion of individual RCTs, which may lead to flawed conclusions 3. Often, for some research topics, there appear to be a greater number of systematic reviews/meta-analyses than original RCTs, probably due to the fact that it is easier to perform a systematic review/meta-analysis than perform a high-quality RCT 3. Unfortunately, even a low-quality systematic review may be published because they are frequently cited which may increase the impact factor of a journal. Consequently, the medical profession, researchers and journal editors are faced with an important dilemma on how to assess the clinical validity of these analyses. This article aims to discuss this problem in detail, using a specific example from the effect of pre-operative glucocorticoids in pain management for total knee arthroplasty (TKA). We used TKA because it has inherent challenges to optimise analgesia and facilitate rehabilitation, and where steroids may have interesting positive effects due to reduction of the inflammatory response to surgery thereby reducing pain and fatigue, and apparently without significant adverse effects on wound healing and infections 4. A PubMed search was performed on 2 September 2019 using the following search strategy: glucocorticoid AND hip OR knee OR total joint AND arthroplasty AND RCT or systematic review or meta-analysis. Since 2015, a total of 12 systematic reviews/meta-analyses have been published on the analgesic effects of pre-operative glucocorticoid administration after joint arthroplasty 5-16. Of these, only five 5, 6, 9, 10, 16 were prospectively registered and seven studies focused only on TKA 5-10, 16, while five reviews combined trials of TKA and total hip arthroplasty (THA). The TKA-specific systematic reviews/meta-analyses 5-10, 16 included all the TKA-related trials in the analyses from the combined THA/TKA reviews. Interestingly, several reviews stated that they were the first in the field. Even more interesting, the number of included trials varied between analyses. Thus, the included RCTs varied: n = 4 in 5; n = 14 in 6; n = 11 in 7; n = 6 in 8; n = 8 in 9; n = 7 in 10; n = 4 in 16; and n = 9 in 17, which could not be directly related to year of publication. Overall, 10 of the 12 reviews were from Chinese institutions and 5 of these have were published in the open access journal Medicine (Baltimore) (IF 2018 by Clarivate Analytics 1.870). A detailed assessment of the 28 RCTs included in the TKA-specific systematic reviews 5-10, 16, 17 is presented in Table 1. The trials included in the systematic reviews/meta-analyses have considerable scientific flaws which make interpretation of these analyses challenging (Table 1). None of the systematic reviews/meta-analyses provided detailed information regarding the inclusion and exclusion criteria. The potential effect of the pre-operative glucocorticoid dose in TKA vs. placebo to be determined from a RCT was only discussed in three systematic reviews 18-20, whereas three other systematic reviews provided additional information with regards to repeat glucocorticoid dosing 20-23. However, although the overall conclusions from all the systematic reviews are in favour of pre-operative glucocorticoids, the dose used in the included RCTs varied significantly – from 8 mg to about 25 mg dexamethasone and a total dose of about 40 mg dexamethasone. Several systematic reviews included two RCTs performed in patients undergoing bilateral TKA 24, 25 without any specific comments. Obviously, it is inappropriate to include RCTs of bilateral TKA compared with unilateral TKA, as the postoperative pain characteristics between these two surgical procedures vary significantly. One review included a RCT 26 that was performed in patients undergoing unicompartmental knee arthroplasty (UKA), and here again it is inappropriate to combine this smaller surgical procedure with unilateral TKA. Most of the systematic reviews/meta-analyses included some of the 10 RCTs that assessed peri-articular glucocorticoid administration 27-36. However, none of these 10 RCTs included a corresponding intravenous (i.v.) glucocorticoid dose, thereby preventing sufficient interpretation regarding an analgesic effect of systemic vs. peri-articular glucocorticoid in TKA. Of note, these RCTs used different doses of the glucocorticoid as well as had variable drug mixtures in the injected solution. One RCT 36 was undertaken in staged bilateral TKA with appropriate random allocation of patients to glucocorticoid infiltration vs. placebo in relation to the first vs. second procedure. Of note, peri-articular glucocorticoid administration from nine RCTs has been assessed in a recent meta-analysis, but with the same methodological limitations for sufficient interpretation 17. This review also included one RCT 30 in patients undergoing UKA along with others including patients undergoing unilateral TKA, and one review included two non-RCTs 37, 38. Of interest, a RCT 39 was included in some of the systematic reviews, although it was retracted due to fraud. Another RCT 40 was included in some systematic reviews despite the absence of specific pain data between THA and TKA. Two included RCTs 41, 42 in some meta-analyses presented no specific pain data and one included RCT 43 was available only as an abstract. Another included RCT 44 evaluated perineural glucocorticoid administration, which again, similar to reasons discussed regarding peri-articular glucocorticoid administration cannot be lumped when interpreting analgesia after i.v. glucocorticoid. One included RCT 45 assessed the effects of intra-articular glucocorticoid given 1 week pre-operatively in pain-sensitised patients which is a totally different scientific question and patient population, thereby precluding comparison with other glucocorticoid studies in a pooled systematic review/meta-analysis. Finally, in some of the systematic reviews, different names (Chinese) are shown in the tables compared with the reference list making detailed interpretation difficult for the reader and emphasising the low quality of the article writing and presentation. In summary, this detailed methodology analysis of individual trials included in the published systematic reviews/meta-analyses on the potential analgesic effects of peri-operative glucocorticoid in TKA and documents flaws in methodology preventing sufficient interpretation. Although the authors of these systematic reviews have used an accepted approach to the analyses (i.e. the Cochrane approach) of the included data, our analysis shows a significant problem with the applied methodology, since most of these systematic reviews did not consider the type of surgery and the different routes of administration, different timing of administration and different doses of glucocorticoids. In addition, variations in the type of anaesthetic technique (i.e. spinal anaesthesia vs. general anaesthesia) and the analgesic techniques (e.g. use of non-opioid analgesics such as paracetamol and non-steroidal anti-inflammatory drugs, and use of regional analgesia such as peripheral nerve blocks and peri-articular infiltration) were either not considered or variable between studies. None of them discussed why previous reviews had included a different number of studies or the significant variability in inclusion of trials. Our analysis emphasises the challenges faced by busy clinicians who lack the time and knowledge to go into details of individual RCTs, and often use systematic reviews/meta-analyses to remain updated. Nevertheless, with regard to the specific question about analgesic effects of peri-operative glucocorticoid administration, it remains of major interest due to the documented positive effects from the few well-designed studies 18-23. However, questions about the optimal dose and indications for repeat dosing remain unanswered 4. Importantly, the main message of our analysis is to call for a critical analysis of individual trials before pooling the data into a systematic review 3. This must be a combined approach between researchers, reviewers, and editors of medical journals. Our analysis also demonstrates a significant problem regarding the editorial handling and review of such systematic reviews/meta-analyses. We suggest that for systematic review/meta-analyses to be clinically valid, they should discuss why the review is performed, what is new compared with previous reviews and detailed information on the specific inclusion/exclusion criteria. GJ has received honoraria from Baxter Pharmaceuticals and Pacira Pharmaceuticals. No other conflict of interest declared.