Article Text
PDF
Abstract
Since its introduction by the Nuremberg Code and the Declaration of Helsinki, the place held by ethics in biomedical research has been continuously increasing in importance. The past 30 years have also seen exponential growth in the number of biomedical articles published. A systematic review of the literature is the scientific way of synthesising a plethora of information, by exhaustively searching out and objectively analysing the studies dealing with a given issue. However, the question of ethics in systematic reviews is rarely touched upon. This could lead to some drawbacks, as systematic reviews may contain studies with ethical insufficiencies, may be a possible way to publish unethical research and may also be prone to conflict of interest. Finally, informed consent given for an original study is not necessarily still valid at the systematic review level. There is no doubt that routine ethical assessment in systematic reviews would help to improve the ethical and methodological quality of studies in general. However, ethical issues change so much with time and location, and are so broad in scope and in context that it appears illusory to search for a universal, internationally accepted standard for ethical assessment in systematic reviews. Some simple suggestions could nevertheless be drawn from the present reflection and are discussed in the paper.
- Ethics
- research
- systematic review
- review
- meta-analysis
- scientific research
- codes of/position statements on professional ethics
- ethics committees/consultation
- position statements (of organizations/groups)
- policy guidelines/inst
- review boards/review cttes
Statistics from Altmetric.com
- Ethics
- research
- systematic review
- review
- meta-analysis
- scientific research
- codes of/position statements on professional ethics
- ethics committees/consultation
- position statements (of organizations/groups)
- policy guidelines/inst
- review boards/review cttes
Introduction
Since its introduction by the Nuremberg Code (1947)1 and the Declaration of Helsinki (1964),2 the place held by ethics in biomedical research has been continuously increasing in importance. It is now accepted that the methodological quality of clinical trials goes hand in hand with proper consideration of the ethical dimension of clinical research.3 Today, more than at any time in the past, it would be unacceptable to circulate and use results of clinical trials carried out in conditions incompatible with the respect due to the person participating as the subject of the research. To avoid such studies taking place, ethics committees play the fundamental role of assessing the ethical aspects of research protocols before subjects are included in a study. The editors of biomedical periodicals also ensure that the ethical nature of studies accepted for publication is taken into account.4 Biomedical journals' guidelines for authors point out the ethical criteria that must be respected. Thus, it has theoretically become impossible to publish original research that does not respect fundamental ethical principles.
In parallel, the past 30 years has seen exponential growth in the number of biomedical articles published. It is not rare for several studies concerning the same clinical question to be published within a few months of each other, with similar methods but conflicting results. A systematic review of the literature is the scientific way of synthesising a plethora of information, by exhaustively searching out and objectively analysing the studies dealing with a given issue. When appropriate, a meta-analysis (statistical grouping of the results) is made and it is then possible to perform a heterogeneity analysis to try to find explanations for the discordance of the results. Meta-analyses of clinical trials thus constitute the highest level of evidence in the graduated levels frequently used when the concept of evidence-based medicine is applied.
Paradoxical as it may seem, the question of ethics in systematic reviews is rarely touched upon. The lack of ethical assessment in systematic reviews can probably be explained by the secondary nature of systematic reviews: it is commonly accepted, albeit completely implicitly and erroneously, that the original studies included in systematic reviews automatically respect the most fundamental criteria of ethics. In consequence, the question of ethics is nowadays incompletely posed when considering systematic reviews. This could lead to some problems, as will be discussed in the first section of this article. Further, we shall point out the arguments and counter-arguments regarding a methodical assessment of ethics in systematic reviews and, finally, suggestions will be proposed to enhance the consideration given to ethics in systematic reviews.
What are the problems?
Systematic reviews may contain studies with ethical insufficiencies
Due to the intrinsic nature of systematic reviews and meta-analyses (ie, technique of aggregating data from different studies), ethical heterogeneity between studies exists as well as heterogeneity of methodological quality. For some studies included in systematic reviews, ethical considerations could be of insufficient quality. Weingarten et al wrote in 2004 that assessing ethics in systematic reviews would ‘encourage reviewers to identify those occasional studies that were so unethical that there may be doubts about the morality of using the results’.5
Systematic review as a possible way to publish unethical research
The methodology used for conducting systematic reviews does nothing to prevent the inclusion of unethical studies. On the contrary, the inclusion of unethical studies is made possible and even encouraged by the will to make an exhaustive search for information. In an attempt to avoid publication bias, studies that have never been published are actively sought by the authors of systematic reviews even though the risk of including studies that do not respect ethical principles is certainly higher among unpublished articles. The relative weight of such studies in a meta-analysis may well be sufficient to greatly influence the main result. In the extreme, it is even possible to imagine that systematic reviews can be an indirect way of publishing the results of research carried out in ethically unacceptable conditions. Without an evaluation of the ethical aspect of studies included in meta-analyses, it is materially impossible to prevent the publication of unethical research under the protection of the label ‘systematic review’. Although the risk seems small today, it is nevertheless real and the history of clinical research is full of examples that have become sadly notorious6 7 and should serve as a warning against a drift towards research outside an ethical framework.
Informed consent given for an original study is not necessarily still valid at the systematic review level
Traditionally, systematic reviews and meta-analyses are performed using data from original authors' reports or publications. The data used for statistical pooling are usually treatment effect estimates (eg, relative risks), with the associated degree of uncertainty (eg, CIs). Consequently, these also-called ‘aggregate data meta-analyses’ are likely to have the same objectives as the original studies, and there is no particular ethical problem regarding the original informed consent of patients. In recent years, an increasingly popular method has been developed: the ‘meta-analysis of individual participant data’. In this type of meta-analysis, raw individual-level data for each original study are obtained and used for synthesis.8 This approach offers clinical and statistical advantages over the traditional approach. It also requires a much greater commitment in time and resources, and the collaboration of the investigators who conducted the original studies, who must provide the necessary data. Methods to ‘de-identify’ clinical trials data have been developed to ensure confidentiality in the published reports9 but another ethical concern arises with the use of such data. Although we found no examples in the current literature, it is theoretically possible that a meta-analysis of individual participant data has a different objective from some of the original studies included. This could be in conflict with the Nuremberg statement according to which it ‘should be made known to [the experimental subject] the nature, duration and purpose of the experiment; the method and means by which it is to be conducted’.1 It could also be in conflict with the Declaration of Helsinki: ‘in medical research involving competent human subjects, each potential subject must be adequately informed of the aims, methods … and any other relevant aspects of the study’.2
A systematic review is biomedical research that could also be prone to conflict of interests
Making a systematic review is scientific research work in its own right, leading to an original production revealing new results. In spite of well-described methods to enhance the quality of meta-analyses,10 there remains an element of subjectivity in the overall process of performing them (eg, during selection of studies or interpretation of results). This subjectivity is inherent in all biomedical research but, even more than in any other biomedical research, authors of systematic reviews have to clearly mention potential conflicts of interest and how their work was funded. Indeed the guidelines for good clinical practice mostly take their inspiration from the conclusions of systematic reviews. The Cochrane Collaboration and the majority of editors have already integrated these parameters in their instructions to authors.4 10
Should we encourage methodical assessment of ethical issues in systematic reviews? Arguments and counter-arguments
Arguments in favour of a methodical assessment of ethical issues in systematic reviews
‘To increase awareness in the scientific community about the need for high ethical standards in research on humans’.5 This concerns the enhancement of ethical research conduct, and the need for good quality reporting of ethical considerations.
To identify studies with manifest ethical insufficiency, and so to limit the risk of basing a clinical practice on clinical trials that did not respect ethical principles.
To discourage the publication of non-ethical research under the protection of the label ‘systematic review’.
To regulate and control the ethical considerations in meta-analyses concerning individual participant data (confidentiality and informed consent).
To encourage the full respect of the conflict of interest statement and the financial disclosure.
Arguments against a methodical assessment of ethical issues in systematic reviews
In 2010, there is no broad consensus on how to proceed with the assessment of ethical issues in systematic reviews. Weingarten et al proposed a standardised protocol in 2004 for evaluating the ethical aspects of clinical trials included in systematic reviews.5 This protocol, inspired by Foster's work,11 was based on four ethical considerations: a first concerning the aims of the research, a second concerning the moral duty of the investigators, a third relative to the rights of the subjects participating in the research and a final consideration relative to the global ethical assessment of the research by an ethics committee. However, we are obliged to observe that this protocol has not yet come into use in the biomedical community.
Supposing that a recognised protocol exists, it could hardly be used to estimate the ‘ethical quality’ (neither quantitatively nor qualitatively) for each original study, mainly because of the possible lack of reporting of such information in study reports. Insufficient reporting of ethical issues in original papers does not necessarily mean that the studies were not conducted in the respect of ethical principles, even though failing to report ethical requirements in original articles may already imply a lower concern for ethical issues in studies.12
As stated by Weingarten et al, ‘ethical standards depend not only on the time the study was performed but also on place and social mores’.5 It could seem vain to compare the ethical quality of different studies, conducted in different places, at different times and on different populations, with a standardised approach. On the contrary, even supposing that a ‘contextual approach’ to ethical considerations would be feasible in a systematic review, it would be useless, since inference of results of the systematic review is likely to deal with a contemporary population.
Assessing ethical issues in original studies does not protect from false or fraudulent reports regarding the respect of ethical requirements in those studies. Thus a possible adverse side-effect could be the false security many readers of systematic reviews feel when no ethical problem is reported.
Several items used in the checklists/scales for the assessment of ethics in research studies also have methodological implications. For example, the protocol by Weingarten et al considered the necessity for an adequate sample size calculation, the publication status of the study and the suitability of comparators.5 As stated by Handollin in a BMJ rapid response to Weingarten et al, ‘judgement of ethical standards, almost certainly based on inadequate information, might detract from an appraisal of the validity of the evidence’. Most current systematic reviews already comment on these items without a specific ethical assessment.
Proposed solutions
There can be no doubt that routine ethical assessment in systematic reviews would help to improve the ethical and methodological quality of studies in general. However, ethical issues change so much with time and location and are so broad in scope and context that it appears illusory to search for a universal, internationally accepted standard for ethical assessment in systematic reviews. Some simple suggestions could nevertheless be drawn from the present reflection.
As we can consider the declaration of Helsinki as the foundation of modern ethics, the ethical assessment could be simply based on the most fundamental ethical principles: research based on the results of experimental studies, consent of an independent committee for the protection of persons, informed consent of each participant, qualification of investigators and positive benefit/risk assessment.
We should thereafter distinguish between two aspects of ethical insufficiency in original papers: insufficient reporting of ethical considerations and/or manifest disregard of one or more fundamental ethical principles. In the first case, information on ethical considerations may be not well reported or may be incomplete. As for the analysis of the methodology, it may be advisable to contact the authors for further information. Contacting the authors is a healthy approach from the methodological point of view and from the standpoint of deontology among research workers. It does indeed enable complementary information to be obtained but it also allows the author of the original study to give his informal ‘consent’ to inclusion of his data in a systematic review and validation of the data extracted by the author of the review. In the second case, readers of the systematic review should be informed of the ethical insufficiencies of some studies revealed by the systematic review approach.
Then it is our opinion that authors of systematic reviews should guarantee a minimum of ethical assessment and at least provide a brief discursive report of the ethical assessment of original studies. We suggest that, once the ethical characteristics have been collected, they can be summed up in a short discussion paragraph or in the form of a descriptive table, depending on the nature or the number of ethical irregularities. It is up to the author of the systematic review, depending on his field of research, to exclude certain studies as unacceptable from an ethical point of view and/or to analyse sub-groups according to different ethical parameters. To the best of our knowledge, such an approach is hardly ever applied.
This approach could be encouraged by some minor addendum in relevant reporting guidelines. Standards exist for reporting research, and dozens of biomedical reporting guidelines are now available.13 These useful guidelines are specific to each type of study, and the most frequently used ones are the CONSORT checklist for clinical trials,14 the STROBE checklist for observational studies15 and the STARD checklist for diagnostic accuracy studies.16 Today, none of these reporting guidelines contain directions for ethical considerations, as has already been pointed out several times.5 17 We fully understand the point of view of some authors of these reporting guidelines, who point out that the lack of ethical considerations in their checklists by no means indicates that they are not important, but simply that they fall within the competence of national and/or international institutions.18 That said, we believe that this point of view has become unsatisfactory at the systematic review level as neither ethics committees nor editors of biomedical journals usually assess the ethical quality of studies included in systematic reviews. Two main reporting guidelines for meta-analyses of aggregate data exist: the PRISMA checklist for systematic reviews and meta-analyses19 (formerly known as QUOROM) and the MOOSE checklist for meta-analyses of observational studies.20 Neither of them contains items for the ethical assessment of the studies included. The last section of the PRISMA checklist is the ‘Funding’ section, with only one item, also called ‘Funding’. A possible addendum could be the addition of a second item ‘Ethics’ in a newly-titled ‘Funding and Ethics’ section. This new checklist item could remind the authors to ‘Indicate if all studies have respected the fundamental ethical principles consistent with the Declaration of Helsinki; if applicable, discuss ethical insufficiencies’. Such an addendum could also be proposed in the Discussion section of the MOOSE guidelines.
We also suggest that editors of biomedical journals who publish systematic reviews of individual participant data should encourage the authors of such studies to accompany their manuscript by a statement regarding the fact that prospective analyses have been reviewed and approved by an independent ethical committee.
Conclusion
In our opinion, meta-research should not ignore ethical considerations, at least to avoid the risks mentioned in this paper and at best to enhance the ethical quality of biomedical research. We suggest that ethical assessment should be an integral part of the methodology intrinsic to systematic reviews. Some simple measures could be applied to reduce the current lack of ethical assessment in systematic reviews. It will demand greater vigilance of authors, ethics committees, editors and peer-reviewers. The failure to take ethical considerations into account in systematic reviews is a surprising, and virtually problematic, deficiency at present. Let us promise ourselves that it will be made good in the future.
Acknowledgments
The authors thank all the members of the Diplôme d'Université d'Odontologie Légale et d'Éthique, 2008–2009, Toulouse Dental Faculty, for helpful comments.
References
Footnotes
Competing interests None.
Provenance and peer review Not commissioned; not externally peer reviewed.