Nutrition Users’ Guides: an introduction to structured guides to evaluate the nutrition literature

Worldwide, chronic non-communicable illnesses represent the largest burden of disease. There is good evidence that healthy eating habits represent one of the cornerstones to primary prevention of chronic disease.1–3 Nutrition-related deficiencies and disease may have a substantial impact on disability-adjusted life years and death. For instance, one 2017 estimate suggested that dietary habits accounted for 10.9 million deaths globally.4

Despite the evidence that nutrition can play an important role in curbing the burden of chronic disease, nutrition plays only a minor role in most clinicians’ practice. For those trained in medicine, providing nutritional counselling in clinical practice comes with a unique set of challenges, including a lack of nutrition training in medical school, lack of time and lack of financial compensation for offering such counselling.5 6 For those trained in nutrition, there may be considerable variability in evidence-based practice training.7

Providing the highest-quality nutrition care depends on a clinicians’ knowledge and skills in evidence-based practice, including skills to ask a structured question and acquire, appraise, interpret and apply the best available evidence within the context of clinical expertise and patient values and preferences.7–9 However, dietitians, physicians and other healthcare workers may lack these skills,7 10 11 with a recent literature review finding no studies among dietitians that have reported on their competencies in interpreting the absolute magnitude (size) of effect, or the certainty of evidence in effect, competencies essential for optimising clinical nutrition decision-making.7

For those who lack evidence-based practice skills, or the time to use their skills, pre-appraised and pre-interpreted evidence by methodologists allows clinicians to more accurately inform and guide their patients with clinical decision-making. While many comprehensive up-to-date resources with pre-appraised/interpreted evidence in medicine exist (eg, UpToDate, Dynamed, McMaster Plus; BMJ Best Practice), pre-appraised/interpreted evidence specific to clinical and public health nutrition (ie, Evidence Analysis Library, Practice-based Evidence in Nutrition) are fewer and not always up to date.

In healthcare practice and policy, both clinicians and authorities have accepted the importance of evidence-based practice, including shared decision-making informed by the best available evidence as well as patient or population values and preferences. This is no less true for nutrition care, for which clinicians and policy-makers must consider the best available evidence and its bearing on unique nutrition-related decisions based on the values and preferences of the target patient or population.9 12 Value-sensitive and preference-sensitive decisions must ultimately follow from weighing the trade-offs between potential benefits, harms and costs of alternative treatment or prevention strategies, the certainty of evidence informing estimates of these key considerations, and the ability of people to implement and adhere to a nutrition or diet recommendation, particularly if burdensome.

We anticipate that this series of articles will provide a welcome and thought-provoking set of structured users’ guides for clinicians and healthcare professionals (eg, health service workers, policy-makers) using nutrition research articles to guide practice and policy, a series of articles that can also be used to support informed, shared decision-making with patients and clients. The series will include guidance on how to appraise, interpret and apply randomised trials, cohort studies, systematic reviews and practice guidelines, while also laying the foundation for future articles considering emerging approaches to nutrition and medical research (eg, Mendelian randomisation; platform and adaptive trials; planetary health). As compared with a variety of other “appraisal” checklists used in nutrition that focus almost exclusively on methodological quality and/or risk of bias (ie, appraisal), particularly unique to our framework is the guidance on interpreting and applying study results including interpreting the absolute magnitude of treatment effect (eg, from trivial to small, moderate or large) for people-important health outcomes7 13 14 and applying the evidence together with patient or client values and preferences. A third component of evidence-based practice is health service expertise. For example, expert, experienced clinicians play an important role in interpreting the evidence, including laboratory results, and guiding patients to make informed decisions. Ideally, clinicians are highly skilled in the principles of evidence-based practice9 and are particularly skilled in determining the best available evidence (eg, systematic reviews, umbrella reviews, practice guidelines) and engaging with patients based on their unique circumstances, including the patients’ health-related values and preferences. A clinical practice based on expertise alone however may, in many instances, be less trustworthy given that clinicians (ie, humans) are prone to many cognitive biases and to clinical time constraints.5 6

With respect to nutrition users’ guide articles on summary evidence (systematic reviews, guidelines), this series will adhere to guidance from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group that, in a four-category classification of certainty of evidence (high, moderate, low and very low), considers systematic reviews of randomised controlled trials (RCTs) to start at high certainty evidence, with guidance on five factors (ie, risk of bias, indirectness, inconsistency, imprecision, publication bias) that can justify rating the certainty of evidence down.15 16 Observational studies typically start at low certainty evidence,9 15 or if they start at high (eg, Risk Of Bias In Nonrandomized Studies of Interventions risk of bias tool), they are typically downgraded for serious issues of bias.17 The reason for this guidance is that observational studies inevitably suffer from a risk of substantial residual confounding18 and risk of publication bias,19 and only in the presence of large associations (eg, relative risk >2 or <0.5) or valid dose-response gradients can they provide higher certainty evidence, provided there are no other serious 20limitations (table 1).

Nutrition evidence is unique in that RCTs may be absent for many clinical or public health nutrition questions, many nutrition RCTs lack genuine placebos and are challenging to blind, RCTs of dietary interventions may be challenging to adhere to and may not be long enough to capture impacts on hard endpoints (eg, mortality), and, unlike pharmaceuticals, many interventions aim to modify existing variables that participants are exposed to.9 Thus, some have argued that evidentiary standards should differ for nutrition, and that prospective cohort studies may sometimes be as helpful as RCTs in judging estimates of effect of therapy or exposure.21–25 Indeed, the GRADE method includes provisions that authors of systematic reviews based on observational studies may rate up the certainty of evidence in certain situations (table 1), resulting in moderate or high certainty evidence, when justified typically based on large associations or a credible dose-response gradient.18 20 The GRADE method also includes provisions for downgrading systematic reviews of RCTs26 to account for some of the inherent challenges in nutrition interventions that compromise the certainty of evidence (eg, risk of bias related to poor or infrequent dietary intake assessments, and indirectness when the target intervention is a nutrient evaluated in pragmatic trials that are not isocaloric).

GRADE also provides guidance on moving from the best available evidence (ie, systematic reviews) to decision-making (ie, guideline recommendations) using an evidence-to-decision framework.27 When, for example, it is clear that the desirable outcomes (benefits) outweigh undesirable outcomes (harms) and when the certainty of evidence is high, a strong recommendation is typically most appropriate, particularly if all or almost all patients or members of the public would be willing to take the intervention based on a systematic summary of their values and preferences. If there is a close balance between the benefits and the harms and the certainty of evidence is low to moderate, a conditional (or weak) recommendation is most appropriate. With conditional recommendations, shared decision-making is particularly encouraged, and decisions should be driven by the patients or target populations’ values and preferences (box 1).

Those who support applying the GRADE approach to nutrition research recognise that biases, such as unaccounted confounders in observational studies, often impact the validity of effect estimates.28–30 For instance, a meta-analysis of nine cohort studies including over 290 000 patients claimed a 25% (95% CI 7% to 40%) relative risk reduction in coronary heart disease among men consuming supplemental vitamin C.31 In contrast, the Physicians’ Health Study II,32 a factorial RCT that enrolled over 14 500 male physicians and followed them for upwards of 10 years, demonstrated no difference in coronary heart disease when comparing vitamin C versus placebo (HR 0.99; 95% CI 0.89 to 1.11). These same studies showed discrepant results between cohort and randomised trials for vitamin E and cardiovascular disease. These examples demonstrate there were confounding variables that accounted for the 25% relative risk reduction in heart disease associated with vitamin C reported in observational studies assessed in the meta-analysis.

Nutrition content expertise is needed to apply the GRADE approach to nutrition research, as interpretation and application require knowledge of topics such as nutritional biochemistry and/or physiology, the strengths and limitations of nutrition assessment methods (ie, dietary intake data, biomarkers), and knowledge of behavioural science. For instance, in well-designed and conducted nutrition studies, known prognostic factors should include both baseline dietary intakes and indicators of baseline nutrient status if potentially valid biomarkers exist (eg, red blood cell omega-3 fatty acids status, or 25-hydroxyvitamin D status in an omega-3 or vitamin D intervention study).33 Few observational studies or RCTs in nutrition quantify the exposure of interest present at baseline in the diet and/or endogenously in tissues. Failure to consider baseline intake and/or status as a component of the inclusion criteria or as pre-specified subgroup analysis may put the results of a study at risk of bias, particularly when attempting to assess the effectiveness of a nutrient relative to baseline nutrient status, such as the utility of vitamin D supplementation in those with lower baseline vitamin D status. Indeed, large randomised controlled trials aiming to answer such questions have typically recruited participants irrespective of their baseline vitamin D status, ultimately recruiting individuals with average baseline 25-hydroxyvitamin D levels already in the observed ‘protective’ range.34

Another nuance of nutrition interventions is that many nutrition interventions lack an inert placebo to compare against. Rather, modifying intakes of specific foods typically occurs via their replacement for other foods (ie, the replacement of saturated fat-rich foods with polyunsaturated fat-rich foods), leading to an inherent substitution effect. Specific substitutions may not be explicit in the intervention’s design, complicating interpretation of observed effects without rigorous dietary assessment methods to identify modifications. Ultimately, multiple studies are typically needed with consistent results to provide high confidence in the relationship between specific foods/nutrients and health outcomes, accounting for their effects relative to various potential comparator foods/nutrients. These unique elements of nutrition interventions are critical to consider in systematic reviews and meta-analyses of the literature, as many similar interventions captured by inclusion criteria may have resulted in variable substitutions and may have occurred in populations with varying baseline intakes/status, resulting in varying results.

As the field of nutrition continues to tackle diet-chronic disease relationships, and embraces ‘precision nutrition’ in an effort to explain inter-individual variation in the response to nutrition interventions, it is critical to ensure evidentiary standards are methodologically robust and transparent (including the need for study protocols for observational studies) to ensure nutrition recommendations provided to the patient or members of the public are optimally trustworthy. Overall, we anticipate that the guidance that this series generates will ultimately further the understanding and discussion of how the nutritional literature can best guide clinical and population health practice considering issues of validity (risk of bias), the magnitude and precision of exposure or treatment effects on health outcomes, and the applicability of study results to the target patient or population.35–37 This series can also be used by those designing courses in critical appraisal and evidence-based nutrition practice, while also serving to empower health service workers, guideline panellists and policy-makers to better understand the strengths and limitations of the nutrition literature.