Article Text
PDF
Abstract
Introduction The production of clinical practice guidelines (CPGs) has grown in the past years. Notwithstanding, the quality of these documents and their recommendations for the treatment of schizophrenia in children and adolescents is still unknown.
Objective To assess the quality of the guidelines and recommendations for the treatment of schizophrenia in this population.
Methods CPGs from 2004 to December 2020 were identified through a systematic search on EMBASE, MEDLINE, PsycINFO, PubMed, Epistemonikos, VHL, Global Index Medicus and specific CPG databases. The CPGs’ quality was independently assessed by three reviewers using AGREE II and they were considered of high quality if they scored ≥60% in domains 3 and 6. The evidence classification systems were described, the quality of recommendations was assessed in pairs using AGREE-REX and the recommendations were compared.
Results The database search retrieved 3182 results; 2030 were screened and 29 were selected for full-text reading. Four guidelines were selected for extraction. Two CPGs were considered of high quality in the AGREE II assessment. We described the commonly agreed recommendations for each treatment phase. The pharmacological recommendations were described in all treatment phases. Scores of AGREE-REX were lower for psychosocial recommendations.
Conclusion There are still few clinical studies and CPGs regarding schizophrenia in children and adolescents. The quality of the documents was overall low, and the quality of the recommendations report has much to improve. There is also a lack of transparency about the quality of the evidence and the strength of the recommendations.
Protocol registration number CRD42020164899.
- child & adolescent psychiatry
- schizophrenia & psychotic disorders
- mental health
Data availability statement
No data are available. No additional data available.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
To our best knowledge, the critical appraisal of the clinical practice guidelines for the treatment of schizophrenia in children and adolescents was never performed using the Appraisal of Guidelines Research and Evaluation tools.
Even though differences between raters were not assessed, the critical appraisal conducted in pairs or in groups of three or four assessors in our study improved reliability in the overall rating of guidelines.
We chose to focus on specific guidelines for schizophrenia in children and adolescents, which may have resulted in failure to identify pertinent recommendations that might have been included in broader guidelines.
In our study, we have used a structured search strategy that has been revised by two experienced librarians to minimise publication bias.
We used validated and well-known instruments which assure the internal and external validity of the study.
Introduction
Schizophrenia is a chronic mental disorder with a low prevalence, and its precocious form is rare and debilitating.1 2 Onsets in childhood are extremely rare, occurring in less than 1% of the cases, despite the high prevalence of psychotic symptoms in healthy children.3 4 Epidemiological studies about early and very early-onset schizophrenia (EOS and VEOS) are also rare due to the late identification of the disorder and historic events.5 In the 1970s, neurodevelopmental disorders were grouped with EOS and VEOS in the childhood psychosis category which endured throughout the decade, making epidemiological states imprecise.5 6
Because of the rarity and severity of the disorder, VEOS and EOS require a combination of antipsychotic medication, a close follow-up of the patient, and psychological and psychosocial interventions.4 7 8 The diagnostic criteria are the same throughout all life stages of a person, but there are some known differences in the evidence about the use of antipsychotics by young patients, and psychosocial follow-up after the first episode is especially important for them in their future outcomes.9 10
Documents that compile recommendations for the treatment of disorders like VEOS and EOS, such as clinical practice guidelines (CPGs), can help decision-making and lead the practitioners to more evidence-based decisions in their practice.11 12 However, there is still a deficiency in the use of such documents by health professionals, due to the overwhelming number of documents, conflicting recommendations, the lack of knowledge on how they can be implemented, resistance to changing their practices and also a perception of the guidelines’ use as a ‘too rigid and simplified’ way of doing medicine.11 13 14
To overcome these challenges and implement best practices in health services, especially in the mental health area, practitioners should have access to high-quality CPGs and trustworthy recommendations. Systematic assessments of these documents can help summarise the knowledge gaps and inconsistencies, and also indicate the best documents that can be used and/or adapted to clinical practice by using quality appraisal instruments, such as the Appraisal of Guidelines Research and Evaluation (AGREE) tools.15
The AGREE II tool was launched in 2009 and is the current version of the tool developed in 2003 by a group of international guideline developers and researchers to address the high variability in the quality of the CPGs.16–18 Later on, in 2019, after many researchers realise that high-quality CPGs could not guarantee the quality and trustworthiness of their recommendations, the AGREE Consortium launched the AGREE Recommendation Excellence (AGREE-REX) to help them assess the quality of the recommendations.18–20 This tool complements the AGREE II assessment and can be used in the whole document, in groups of recommendations and/or in specific recommendations.21
This kind of assessment of CPGs was lacking for schizophrenia in children and adolescents and had the potential of improving the treatment of these young patients. With this study, we wanted to summarise the existing CPGs for schizophrenia in children and adolescents, determine their quality using the AGREE II tool, assess the quality of their recommendations using the AGREE-REX tool and compare the CPGs to check if the recommendations provided by high-quality ones were more complete than those provided by others.
Materials and methods
The protocol for this study was previously published in an open-access journal22 before the beginning of the study (online supplemental material 1). The methodological survey was registered in the International Prospective Register of Systematic Reviews (PROSPERO) database (online supplemental material 1).
Supplemental material
Study design
The present study has been reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (online supplemental material 2).
Supplemental material
Eligibility criteria
Inclusion criteria
Following the recommendations of Johnston et al23 in their methodological paper for systematic reviews of guidelines, we decided to display our eligibility criteria in the PICAR (Population and clinical indication(s), Intervention(s), Comparator(s), Attributes of the CPGs, Recommendation characteristics) format instead of the usual PICO (Population, Intervention(s), Comparator(s), Outcome(s)) format (online supplemental material 3).
Supplemental material
Exclusion criteria
Guidelines for schizophrenia caused by misuse of substances and guidelines for schizophrenia associated with other mental disorders were excluded. If there was a more up-to-date version of the guideline; the available version was incomplete or contained only a summary of the information; the document was the translation of a guideline published in another language; and if there was a consensus document, evidence summary or algorithm, it was excluded since they were not equivalent to guidelines.
Selection of studies
Data sources
The following electronic databases from 2004 to December 2020 were searched: EMBASE (Excerpta Medical Database, via Ovid); MEDLINE (via Ovid); PsycINFO (via Ovid); PubMed; Epistemonikos; Virtual Health Library (VHL); Global Index Medicus. Specific databases for clinical guidelines were also searched: ECRI Institute (www.guidelines.ecri.org), National Institute for Health and Care Excellence (www.nice.org.uk), Canadian Agency for Drugs and Technologies in Health (www.cadth.ca), Canadian Medical Association (www.cma.ca), Canadian CPG Infobase: CPG Database (www.cma.ca/En/Pages/clinical-practice-guidelines.aspx), Scottish Intercollegiate Guidelines Network (www.sign.ac.uk), Australian CPG (http://www.clinicalguidelines.gov.au/) and the Guidelines International Network (http://www.g-i-n.net/) database. The databases list was defined with the help of two experienced librarians.
Other data source features
We checked the reference list of eligible studies, review studies and secondary studies to identify other possible guidelines. Authors were contacted in case the guideline had been published only in summary or where important information was missing.
Search strategies
The keywords were used according to the terms of the Medical Subject Headings to identify relevant studies. The search terms that were used for the databases are provided in online supplemental material 4. The search strategy was adapted for each database consulted.
Supplemental material
Determination of eligibility
References were managed in EndNote (V.X8.2; New York City: Thomson Reuters, 2018), and duplicates were removed. Titles and abstracts were assessed by groups of three reviewers, independently, using a consensus approach, to check if they met the eligibility criteria. A full read of the CPGs was conducted by the same reviewers, also independently, to confirm the eligibility of the guidelines. Discrepancies were resolved by consensus and a fourth reviewer assisted in the final decision if necessary. The most up-to-date guideline was used if there was a case of duplicate publications. All documents related to the guidelines (such as supplemental documents and summaries of recommendations) were searched manually by one or two reviewers.
Data extraction
The information was organised in a Microsoft Excel worksheet; the same groups of three reviewers, independently, extracted the data. Discrepancies were resolved through discussion and consensus. If this process was not effective, a fourth reviewer was responsible for the tiebreaker. Previously, reviewers were calibrated by extracting at least three guidelines of different quality levels and reaching a consensus. The results were discussed with a previously trained fourth reviewer. This procedure was repeated until the reviewers could extract the data.
The following data were extracted: the number of authors, year of publication, update time, organisations (government, medical society, university or other), type of guideline (formulated, adapted, updated or revised), country of development, type (diagnosis, prevention, pharmacological and non-pharmacological treatment, and/or other), treatments described, target population, design of studies included (systematic review, consensus, overview of systematic reviews and/or other), methods of recommendation formulation (consensus, not mentioned, others) and methods of classifying the quality of evidence (Grades of Recommendation, Assessment, Development, and Evaluation (GRADE),24 Oxford,25 not mentioned or other).
Quality assessment of clinical practice guidelines
The AGREE II was used to evaluate the quality of the guidelines. The tool has been translated and validated for the Portuguese language (Brazil), and this version was used in this study. It includes six domains: (1) scope and purpose; (2) stakeholder involvement; (3) rigour of development; (4) clarity of presentation; (5) applicability; and (6) editorial independence, containing 23 items in total. Scores are on a Likert scale of 1 (totally disagree) to 7 (totally agree) for each item.18 26
A group of three reviewers conducted the quality assessment of the guidelines. Differences between two or more scores for each item were considered a discrepancy. The reviewers were previously trained by assessing a guideline provided by the ‘My AGREE PLUS’ platform and one of the selected papers. This first assessment was discussed, and after that, we conducted the rest of the assessments.
The final score was decided by consensus. In case of no consensus, a fourth reviewer helped in the final decision. The quality of the CPGs was calculated for each domain as instructed by the AGREE II user manual.27 Since the six domains are independent, the scores were calculated as the sum of the individual items in each domain. The total obtained was presented as a relation percentage of the maximum possible score for each domain. The evaluation was conducted using the ‘My AGREE PLUS’ platform.18
We considered a high-quality CPG those that got ≥60% on domains associated with the reliability (domains 3 and 6) since those apply to the methodology and editorial independence, fundamental items for our evaluation.
Description, comparison and quality assessment of the recommendations
The level of evidence of the studies that originated the recommendations was not used as a criterion for comparing, for each of the selected guidelines used a different method and, consequently, different criteria and quality classification. This variability would make the comparison less trustworthy since a study classified in one system as high quality could receive a different classification when evaluated by another.
The assessment described and compared the psychological, psychosocial and pharmacological recommendations of intervention. We anticipated the important influence of culture/country on the recommendation of psychosocial and psychological interventions. If appropriate, we analysed such differences.
In this study, we compared the recommendations found in the CPGs with each other. The treatment recommendations and the classification system used to assess the level of evidence for the studies that compose the recommendations were independently extracted by two researchers. Disagreements among researchers were resolved by consensus; in the absence of consensus, a third investigator helped in the decision.
The recommendations were grouped into the following categories: pharmacological, psychosocial and psychological, according to their similarities through an interactive process between researchers. CPGs that shared similar recommendations were noted. We evaluated if recommendations from different CPGs addressed the same topics and compared them to identify differences. When two or more CPGs showed conflicting recommendations, this was defined as a disagreement. We opted to describe the interventions present in all the CPGs selected, to verify if the high-quality CPGs presented similarities in their recommendations with the ones of lower quality.
We assessed the quality of the recommendations using the AGREE-REX instrument.21 This tool is divided into three domains: (1) clinical applicability; (2) values and preferences; (3) implementability. It has nine items in total and scoring is made on a 7-point Likert scale. It can be applied either in each recommendation if the user believes that there is variability in the quality of recommendations or wants to investigate selected recommendations, or in the whole guideline, if the user perceives that there is a consistency in the recommendations, is interested in all recommendations or wants to save time for any reason. It also has an optional item for suitability for use, scored on a 7-point Likert scale as well. We opted to assess groups of recommendations (psychological, psychosocial and pharmacological) in pairs, using a consensus-based approach, allowed by the instrument.
The assessment of recommendations was conducted in pairs, independently, using a consensus whenever there was a discrepancy. If the discrepancy could not be solved, we reached a third reviewer to help in the final decision in a similar process as the one conducted on the guidelines assessment. The assessors were previously trained; they assessed one of the selected documents and discussed the results and possible doubts before conducting the rest of the assessments. The scoring was conducted in a similar way to the AGREE II scoring, following the indications of the AGREE-REX manual: we performed a tertile split of the domain scores of the candidate CPG and classified the document as being high quality, moderate quality or low quality based on each tertile.21
Data synthesis
Descriptive tables were made to show the results. Statistical analyses were performed using Microsoft Excel and STATA software (V.14.2). For all AGREE II domains, descriptive statistics were calculated as mean (SD) only.
Changes after protocol publication
Since we have adopted a consensus approach for the discrepant scores of AGREE II, which was also used with the AGREE-REX scoring, the intraclass correlation coefficient (ICC) analysis of agreement between reviewers, previewed in the protocol,22 was not conducted. Also, because of the low number of selected CPGs for the final evaluation and extraction, the assessment of changes and improvements in the quality of guidelines over time, after the latest version of the AGREE instrument, using the Wilcoxon rank-sum test (Mann-Whitney test), was not conducted.
Patient and public involvement
Patients did not participate on the study design. After publication, we aim to contact health policy-makers to inform about the results and to invite them to collaborate with us in a dissemination plan.
Results
From 3182 titles retrieved in the database search, 2030 records were screened and 29 were selected for full-text reading. After this phase, four were included (figure 1).
Flowchart of guideline identification.
Of the selected CPGs, two were newly elaborated, one was elaborated using the ADAPTE methodology and one did not mention the methodology for elaboration. The evidence appraisal system was different in all of them; only the National Institute for Health and Care Excellence (NICE) CPG28 presented a similar appraisal system to the Canadian CPG29–34 because it was used in the ADAPTE process of this last one (table 1).
Characteristics of the selected documents
For the AGREE II appraisal, the Canadian and the NICE CPGs had higher scores, but only the NICE CPG scored more than 60% in domains 3 and 6. The Orygen CPG35 had higher scores only in the domains 1 and 4 and the Grover and Avasthi36 CPG had all scores under 60% (table 2).
AGREE II* scores for the quality of the selected clinical practice guidelines for schizophrenia in children and adolescents
In table 3, we described the commonly agreed recommendations for each treatment phase. The full tables containing all the recommendations extracted are divided into pharmacological, psychosocial and psychological categories, and in first episode of psychosis, acute phase, relapses and chronic treatment recommendations, and can be found in online supplemental material 5. Pharmacological recommendations were described in all treatment phases in all CPGs, but the psychological and psychosocial ones were mostly focused on the chronic treatment with only NICE and Orygen CPGs addressing all the phases.
Supplemental material
Commonly agreed recommendations divided by treatment phase
Figure 2 shows the distribution of the selected guidelines by country. Three out of four guidelines were from high-income countries. It also shows the distribution of recommendations by category (pharmacological, psychological and psychological) and by treatment phase (first episode, acute episode, relapses and chronic treatment). All four CPGs had recommendations for at least one of the treatment phases in each category.
Distribution of recommendations by country, category and phase of the treatment. The icons represent each of the recommendations categories (pharmacological, psychosocial and psychological). Each icon represents the existence of recommendations for each of the four treatment phases assessed (first episode, acute episode, relapses and chronic treatment). Whenever the icon is faded, it means the recommendations were incomplete for the treatment phase.
Table 4 describes the quality assessment scores obtained in the application of AGREE-REX. Psychosocial recommendations had lower scores compared with pharmacological and psychological ones, which had more similar scores. Domain 1 of clinical applicability had overall higher scores.
AGREE-REX* scores for the quality of the recommendations presented in the selected clinical practice guidelines for schizophrenia in children and adolescents, divided by treatment category
Discussion
Main findings
Of the four CPGs assessed, two of them28 29 33 34 had scores higher than 60% in domains 3 and 6. Two were newly developed,28 35 one used the ADAPTE methodology29 33 34 and one did not present completely the methodology used in the development.36 About the evidence appraisal system, one of the CPGs did not inform if used any type of evidence appraisal36 and two presented the same system28 29 33 34 due to one of them having used the other in the ADAPTE process. We used the fourth reviewer for consensus just once.
All the CPGs presented the three types of recommendations (pharmacological, psychosocial and psychological). Only one of them presented specific recommendations on medication; the other three had just indications about the choice of antipsychotics. Regarding the psychosocial and psychological recommendations, two of the guidelines focused more on the first episode and chronic treatment, presenting little or no recommendations for the other treatment phases. The highest scores in the AGREE-REX assessment were in domain 1 of clinical applicability. The psychosocial interventions had lower scores when compared with the psychological and pharmacological scores.
Comparison with previous studies
In our results, the first noticeable aspect was that most of the selected CPGs were from high-income countries. This lack of guidance from middle-low-income and low-income countries was present in other critical appraisals for schizophrenia.37 38 The ADAPTE process, used in one of the high-income countries’ CPGs, could help fasten the publication of guidelines in less resourceful contexts due to its flexible nature and possibility of being used by groups with different amounts of resources.39
There was also the variability of evidence appraisal systems used. The same problem has been found in other critical appraisals of CPGs for mental health disorders.38 40–43 This might indicate that, even though the GRADE approach is recommended in the development of this type of document,44 it seems that it is still not well established in the CPGs’ development processes. The use of GRADE in future developments could help mitigate this inconsistency and the standardisation of evidence appraisal could benefit decision-makers, helping them compare and use the best available evidence.40 41 44
At the AGREE II assessment, the NICE and the Canadian CPGs28–34 completed the criteria to be considered of high quality. The NICE CPG for children had many recommendations adapted from the adult version,28 which, in the assessments of CPGs for the treatment of schizophrenia in adults conducted by Bradford et al38 and Keating et al,37 was also the highest score. The Canadian CPG adapted most recommendations from the NICE CPG, which can be one of the reasons for it also having high scores.
Overall, the domain with the worst scores in the AGREE II assessment was domain 5 of applicability. This domain usually presents lower scores in critical appraisals.19 37 38 41–43 45 This is a controversial domain because information about implementation can be found in other documents outside the guideline scope and this can interfere with the scoring of the domain.46 High scores in domain 5 also cannot guarantee that the CPG is implementable in a specific context.47
There was variability between the scores of the three domains of AGREE-REX in the separate assessments. Psychosocial scores were often lower than pharmacological and psychological scores in all domains. This difference between categories was also evident in the extraction of recommendations, where we noticed that psychological and psychosocial recommendations were often left aside when addressing acute episodes and relapses and had lower evidence basing them. Psychological and psychosocial interventions, although having scored very close to the ones obtained for pharmacological interventions, sometimes even surpassing them, in the domain of clinical applicability of AGREE-REX, still showed low evidence supporting these recommendations. Most of what has been produced in the past years, regarding this type of intervention, addresses the cognitive functioning of young patients with schizophrenia and lacks in showing follow-up results, as has been found in the systematic review conducted by Anagnostopoulou et al.10
The lack of evidence also impacts pharmacological recommendations. Many regulatory agencies around the world recommend that patients with schizophrenia younger than 13 years of age do not take any antipsychotic medication.48 While there has been some indication of age in some recommendations through the selected CPGs, most of them did not address this impossibility or even indicated that it was a recommendation for off-label use. Also, we still find a barrier in the antipsychotic prescription for this age group, where most of the existing medication presented adverse drug effects, mostly weight gain, which can lead to several health problems in children and teenagers.49 50
Research in this age group is generally complex and present many barriers. As seen in the CPGs, part of the difficulty in creating a reliable document is the fact that there are few clinical trials being carried out with children and adolescents. This is due to ethical implications and the challenging balance between risk and benefits that must be reached before any clinical research.51 Some of the efforts that would be important for future research are looking at children and young people as an active part of the investigation, as well as thinking about strategies to listen to parents and give them transparency in the development of the intervention and study design.52
Strengths and limitations
Four documents addressed our eligibility criteria and most of them were from high-income countries. The fact that the disorder studied is rare and the lack of resources and/or interest in the development of CPGs in middle-low-income and low-income countries is a barrier to the publication of more CPGs about the topic can indicate a possible publication bias in our results.
The studies selected also lacked recent updates. The two CPGs with high quality did not present an update in almost 5 years. The lack of updating limits the assessment since although we have a high-quality CPG, evidence is always changing and we cannot guarantee that the CPG remains trustworthy.
In the present study, we have used a structured search strategy that has been revised by two experienced librarians, and the critical appraisal was conducted in pairs or in groups of three or four assessors to avoid selection bias and minimise publication bias. We used validated and well-known instruments which assure the internal and external validity of the study.
We also used consensus scoring in our AGREE assessments, where all discrepant results were discussed between the assessors. This approach helps raise agreement between assessors, reliability and precision of the results.
Implications for clinical practice in health systems
The low number of CPGs for the treatment of schizophrenia in childhood and adolescence combined with the uncertainty of the evidence and the low quality of such documents can contribute to no advances in the field and the heterogeneity in the treatment of this type of patient. Clinicians aiming for an evidence-based practice should have access to better documents, preferably addressing their contexts. Implementation practices should also be better described in these documents to help decision-makers in their health systems, giving clearer instructions, information about costs, equity and context adaptation, something lacking, in different proportions, in those documents.
Implications for researchers
There is still low evidence subsiding the CPGs for schizophrenia in this age range. More clinical research is needed, mainly for psychological and psychosocial treatments in acute and relapse phases, but also for treatments that are still off-label for patients with schizophrenia under 13 years of age.53 The conduction and use of network meta-analysis in the recommendations creation process also could help improve the quality and trustworthiness of the recommendations.
Countries also should subside panels of creation or adaptation of guidelines for their contexts, mostly the low-income and middle-low-income countries, where these types of documents are still not a reality. The use of the ADAPTE process can be a great alternative for this purpose due to its flexible nature and possibility of being used by groups with different amounts of resources.39
Conclusion
CPGs for schizophrenia regarding the treatment of children and adolescents are still incipient. There are few publications on the subject, lacking both clinical studies and new CPGs, mostly in countries of middle-low or low income. The quality of the documents is overall low, and the report of recommendations has still much to improve, mostly in psychological and psychosocial areas. There is also a lack of uniformity in care conduct present in the recommendations that contribute, in some sense, to the variability of the treatment.
The quality of the evidence and the strength of the recommendations also lack transparency. These aspects could benefit from a standardisation of the evidence appraisal systems in future publications, such as the use of the GRADE approach.
Data availability statement
No data are available. No additional data available.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
Acknowledgments
The authors would like to thank the contribution of Genevieve Gore, from McGill University, who helped with the selection of databases and search strategy.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @nigars2003@yahoo.com, @lulopesbr
Contributors Study concept and design: MRA and LCL. Methodology: LCL. Drafting of the manuscript: MRA and LCL. Review and editing of the manuscript: MRA, CdCB, SB-F, DOdM, RCFM, JCdO, FCG, NS, CVMA and LCL. Guarantor: LCL.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.