Article Text

Download PDFPDF

Measurement properties of instruments assessing permanent functional impairment of the spine: a systematic review protocol
  1. Suelen Meira Goes1,
  2. Catherine M Trask2,
  3. Catherine Boden3,
  4. Brenna Bath1,
  5. Daniel Cury Ribeiro4,
  6. Paul Hendrick5,
  7. Lynne Clay4,
  8. Xiaoke Zeng2,
  9. Stephan Milosavljevic1
  1. 1 School of Physical Therapy, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
  2. 2 Canadian Centre for Health and Safety in Agriculture (CCHSA), University of Saskatchewan, Saskatoon, Saskatchewan, Canada
  3. 3 Leslie and Irene Dubé Health Sciences Library, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
  4. 4 School of Physiotherapy, University of Otago, Dunedin, New Zealand
  5. 5 School of Health Sciences, University of Nottingham, Nottingham, UK
  1. Correspondence to Dr Suelen Meira Goes; suelen.goes{at}usask.ca

Abstract

Introduction Permanent functional impairment (PFI) of the spine is a rating system used by compensation authorities, such as workers compensation boards, to establish an appropriate level of financial compensation for persistent loss of function. Determination of PFI of the spine is commonly based on the assessment of spinal movement combined with other measures of physical and functional impairments; however, the reliability and validity of the measurement instruments used for these evaluations have yet to be established. The aim of this study is to systematically review and synthesise the literature concerning measurement properties of the various and different instruments used for assessing PFI of the spine.

Methods Three conceptual groups of terms (1) PFI, (2) spinal disorder and (3) measurement properties will be combined to search Medline, EMBASE, CINAHL, Web of Science, Scopus, PEDro, OTSeeker and Health and Safety Science Abstracts. We will examine peer-reviewed, full-text articles over the full available date range. Two reviewers will independently screen citations (title, abstract and full text) and perform data extraction. Included studies will be appraised as to their methodological quality using the COnsensus-based Standards for the selection of health Measurement INstruments criteria. Findings will be summarised and presented descriptively, with meta-analysis pursued as appropriate.

Ethics and dissemination This review will summarise the current level of evidence of measurement properties of instruments used for assessing PFI of the spine. Findings of this review may be applicable to clinicians, policy-makers, workers’ compensation boards, other insurers and health and safety organisations. The findings will likely provide a foundation and direction for future research priorities for assessing spinal PFI.

PROSPERO registration number CRD42017060390.

  • disability
  • back
  • range of movement
  • psychometrics
  • validity
  • reliability
  • responsiveness

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • This systematic review will include a broad range of instruments used to assess permanent functional impairment in individuals with spinal conditions in peer-reviewed articles.

  • This systematic review protocol is presented in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses Protocols.

  • A strength of this review is the use of the internationally recognised, validated COnsensus-based Standards for the selection of health Measurement INstruments guidelines to assess the methodological quality of the included studies.

  • A limitation of this review is language bias, since only studies in English will be included. However, studies published in English that describe cross-cultural validation of instruments from English into other languages will be included.

Introduction

Spinal disorders are the leading cause of work disability, representing around 65% of all musculoskeletal injury/illness claims associated with lost work time, posing a considerable economic, personal and societal burden.1 Internationally, occupational low back pain is the single most prevalent health problem related to absenteeism among workers, responsible for approximately 25% of costs from work-related conditions.2

Spinal disorders such as low back pain are often complex and impact a wide range of multidimensional aspects of health and physical function, such as disability.1 Disability is an umbrella term for impairments, activity limitations and participation restrictions, referring to difficulties encountered in any or all of these three areas of functioning.3 Such complex and interactive processes can be challenging when trying to assess patients’ disability using a dichotomous approach. For example, insurance providers might require thresholds to be set for impairment severity, activity limitations or participation restriction in order to determine eligibility for financial compensation or other benefits.3 4

Permanent functional impairment (PFI) or permanent impairment, is a rating systems used by compensation authorities, to establish an appropriate level of financial compensation.5–9 Evaluation of PFI requires selecting appropriate outcome measures.10 However, the metrics of PFI ratings are not uniformly specified nor universally adopted by workers’ compensation boards, varying in terms of specific PFI rating guides as well as the adoption of function-based criteria.6 9 11 Feasibility, reliability, validity, utility and cost are important aspects that should be taken into consideration when selecting clinical measurement instruments for determination of PFI.12 Although impairment can contribute to limited function and ultimately may have consequences for physical functioning, a clinical examination finding of impairment does not always correspond to a functional loss.13

Instruments used to assess spinal PFI have been critically evaluated, since their measurements are usually based only on impairment.14–18 Poor validity and reliability of non-invasive clinical measurement instruments and the presence of significant measurement error14–17 are criticisms that have led to a number of alternative impairment-based spinal assessment systems such as: American Medical Association Guides’ Diagnosis-Related Estimate,11 Diagnosis-Based Impairment,5 McBride’s method of spinal impairment evaluation19 or Physical Impairment Index.18 20 21 However, these alternative systems for rating PFI present poor descriptions of standardisation and normative values.10 22

The outcome of spinal PFI assessment by compensation bodies can have considerable social, economic and health impacts, yet the reliability and validity of instruments currently available have not been systematically evaluated. To achieve a fair and accurate outcome, clinicians, researchers and government bodies must have access to the most accurate level of evidence regarding methods assessing spinal PFI. To our knowledge, no previous review has assessed the reliability and validity of available instruments for determining PFI of the spine. This manuscript presents the protocol of an ongoing systematic review with the objective to review and synthesise the literature concerning measurement properties of the instruments used for assessing PFI of the spine. Due to the diversity of instruments that might be used to assess PFI of the spine, we anticipate the identification of distinct metrics for measuring PFI will allow specific subgroup analyses for review and ultimate discussion on the strength of support for each instrument.

Methods

Design

This systematic review protocol has been registered with International Prospective Register of Systematic Reviews (registration number CRD42017060390). This protocol will be reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols statement and checklist.23

Eligibility criteria

Inclusion criteria

We will include peer-reviewed articles, with no date limitation, investigating measurement properties of instruments that assess PFI of the spine in individuals with spinal disorders aged from 18 to 65 years. When evaluating PFI, acceptable studies will include measures of impairment (ie, RoM, muscle strength, coordination, endurance and sensation), functional limitation (ie, self-report instruments of physical function and functional performance measures) and permanency of the impairment (ie, duration of the impairment and the likelihood of improvement).3 24

Exclusion criteria

We will exclude letters to the editor, book reviews and short communications. We will also exclude clinical protocols, case reports and series, systematic reviews, meta-analysis, articles not published in English-language, studies intended for screening, diagnosis and prognosis of spinal pathologies as well as studies with specimen-based, cadaver-based, cellular-based, artificial-based and computer-based models. We will also exclude studies of spinal conditions caused by congenital and developmental abnormalities, neoplasm, infection and systemic inflammatory disorders.25 Exclusion will take place at the screening stage, following the literature search, in order to avoid the risk of excluding relevant articles.

Information sources

Studies will be identified by searching electronic databases, scanning reference lists of articles and consultation with experts in the area. A preliminary search plan was developed and pretested by a professional health sciences librarian (CB). The following databases were searched: Medline—OVID (1946 to 20 April 2017); EMBASE—OVID (1947 to 21 April 2017); CINAHL—Cumulative Index to Nursing and Allied Health Literature (1937 to 25 April 2017); Web of Science (1900 to 21 April 2017); Scopus (1995 to 25 April 2017); PEDro (Inception to 22 April 2017); OTSeeker (Inception to 22 April 2017) and Health and Safety Science Abstracts—ProQuest (1988 to 22 April 2017).

Search strategy

To capture all measurement instruments used to evaluate PFI of the spine, our search strategy will focus on combining three conceptual groups of terms: (1) PFI, (2) spinal disorder and (3) measurement properties. The Medline search strategy is presented in table 1.

Table 1

Preliminary search strategy in OVID Medline (1950 to present)

The terms for the concept of ‘measurement properties’ were adapted from a search filter validated and published by Terwee et al 26 and optimised for Medline. A similar search will be employed for other databases, optimised for database-specific search interfaces. A filter to exclude certain publication types (eg, addresses, editorials, letters and newspaper articles) will be applied, as recommended by Terwee et al.26 References will be imported into DistillerSR software (Distiller SR Evidence Partners, Ottawa, Canada). All duplicate articles will be removed prior to screening. Reference lists of included studies will be scrutinised during the process of data extraction for additional articles meeting our criteria. Identified citations will be downloaded and subjected to the same de-duplication and screening process.

Study selection

Study selection will be undertaken in two steps. First, two reviewers will independently screen all titles and abstracts, and all citations selected by both reviewers will be retrieved for step two: full-text screening. In cases where there is disagreement between the two reviewers, any discrepancies will be resolved by consensus and/or consultation with a third member of the research team prior to making a final decision.

A screening tool will be developed and piloted for the title, abstract and full-text screening stages (see box). Inclusion/exclusion will be determined using the same tool throughout the process, although the definitions may be refined as conflicts arise and are discussed. The same questions will be used for both title and abstract as well as full-text screening, with a focus on sensitivity rather than specificity at these stages.

Box

Draft screening tool for use at the title, abstract and full-text review screening stages

Questions for all stages: title, abstract and the full-text (go from step 1 to 7)

  1. Is the study written in English?

    1. No—exclude

    2. Yes or uncertain—go to step 2

  2. Does the study deal with humans?

    1. No—exclude

    2. Yes or uncertain—go to step 3

  3. Does the study deal with adults (between 18 and 65 years of age)?

    1. No—exclude

    2. Yes or uncertain—go to step 4

  4. Does the article represent primary study (ie, no letters to the editor, book reviews, published study designs or trial protocols)?

    1. No—exclude

    2. Yes or uncertain—go to step 5

  5. Does the study assess the spine (cervical, thoracic, lumbar spine, including neck and low back)?

    1. No—exclude

    2. Yes or uncertain—go to step 6

  6. Does the study measure permanent functional impairment (PFI; permanent impairment, physical impairment, functional impairment or disability)?

    1. No—exclude

    2. Yes or uncertain—go to step 7

  7. Is the study designed to evaluate measurement properties of measurement instruments/tools (eg, validity, reliability, responsiveness)?

    1. No—exclude

    2. Yes or uncertain—choose one of the following options:

      1. Title and abstract screening stage—include

      2. Full-text screening stage—go to step 8

Additional question for Full-text stage only

  1. Does this study investigate individuals with spinal conditions not caused by congenital and developmental abnormalities, neoplasm, infection disorders and systemic inflammatory disorders?

    1. No—exclude

    2. Yes or uncertain—go to step 9

  2. Does this study have clinical measures of impairment (eg, range of motion, muscle performance) or objective measures of function (eg, functional impairment evaluation)?

    1. No—exclude

    2. Yes or uncertain—go to step 10

  3. Does this study have a PFI definition that includes measures of chronic or permanent symptoms’ duration as well as measures of functional limitation?

    1. No—exclude

    2. Yes or uncertain—go to step 11

  4. Does this study use at least one statistical test between the measures listed on question number ‘9’ and on question number ‘10’?

    1. No—exclude

    2. Yes or uncertain—Include

Data extraction process

We will develop data extraction forms, as outlined in table 2, which will be pilot tested on three studies not included in the review. Two reviewers will extract data independently. Any disagreement will be resolved through a third party adjudication. The nature of performance measures employed by included studies will drive the nature of the data extraction, as we anticipate that there may be some heterogeneity in the evaluation of measurement instruments.

Table 2

Draft extraction tool

To avoid analysing the same data from multiple articles, we will compare author names, sample sizes and outcomes. In the case of companion papers (eg, reports) of a primary study, all available data will be simultaneously evaluated, to maximise the yield of information for the primary study. Study authors may be contacted for clarifications. If contact with authors is unsuccessful, we will use the most complete and up-to-date data source available, thus avoiding the risk of double counting.

Data items

We will extract information relating to: (1) the measurement properties (ie, reliability, validity and responsiveness) of the measurement instruments for assessing PFI of the spine (eg, assessment of RoM, muscle strength, coordination, endurance and sensation, as well as ability to perform basic daily living and work activities, and duration of the impairment); (2) the type of inferential test and (3) the test results for each article. The extraction process will be tracked using a standardised data extraction form.

Articles will be separated into subgroups based on the instruments used to assess PFI of the spine. The different subgroups will be analysed separately based on impairment-based measures (ie, RoM, muscle strength, coordination, endurance and sensation) and functional limitation assessment (ie, self-report instruments of physical function and functional performance measures). For clarity, we are likely to isolate articles in subgroups to better differentiate the instruments used to assess PFI of the spine, especially when considering that RoM has predominantly been used for assessing impairment of the spine.5 6 9 11

Quality assessment of individual studies

Included studies will be appraised independently by two reviewers as to their methodological quality using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria.27 28 Any disagreement will be resolved by consultation with a third member of the research team. The COSMIN checklist is a consensus-based tool designed to evaluate the methodological quality of studies investigating measurement properties. The instrument shows appropriate levels of agreement27 and, based on its content validity, is a recommended tool for assessing the methodological quality of studies evaluating measurement properties of outcome measures within a systematic review.28

The tool will evaluate the following measurement constructs: reliability; measurement error; content validity; structural validity; hypotheses testing; cross-cultural validity and criterion validity, responsiveness, and interpretability with 5–18 items concerning methodological standards for how each measurement property should be assessed (see table 3). The methodological quality of a study will be considered adequate if all items in a measurement property are considered adequate. Each item is scored on a four-point rating scale (ie,  ‘poor’, ‘fair’, ‘good’ or ‘excellent’).

Table 3

COSMIN definitions of domains, measurement properties and aspects of measurement properties

In the context of this review, validity, in general, defines how well the instrument under evaluation measures the construct it purports to measure. Criterion validity is the degree to which measurements are an adequate reflection of a previously used ‘gold standard’. Content validity is an adequate reflection of the construct to be measured; construct validity is based on an assumption that the instrument truly measures what it is meant to and structural validity implies the scores of an instrument is an adequate reflection of the dimensionality of the construct to be assessed. Reliability refers to the extent to which scores for individuals who have not changed are the same for repeated measurement under several conditions. These include using different sets of items from the same instrument (internal consistency); over time (test–retest); by different assessors on the same occasion (inter-rater) or by the same assessors (ie, raters or responders) on different occasions (intrarater). Responsiveness is the ability of the measurement instrument to detect change over time in the construct to be measured.27

Planned methods of analysis

Where it is possible, mean values of statistical analysis (eg, Cronbach’s alpha, intraclass correlation coefficient, SE of measurement, smallest detectable change, effect sizes, etc) will be calculated from pooled data from methodologically similar studies and the results statistically summarised via meta-analysis into forest plots with estimates of heterogeneity. In addition, sensitivity analysis will be pursued by comparing results from studies with high-quality and low-quality ranking. We will then weight such meta-analyses using both the study’s sample size and their quality assessment as determined by the COSMIN checklist.28 However, we predict some heterogeneity will be identified in the various assessments of spinal PFI measures, which will likely make a meta-analysis difficult to apply.

In the event meta-analysis is not possible, descriptive and narrative syntheses will be presented in text and table formats, with the intent of summarising and discussing the sample and methodological aspects, as well as the findings regarding measurement properties of the included studies assessing PFI in individuals with spinal conditions. Tables will provide general and comparative information of these heterogeneous and disparate studies (ie, authors, country and population parameters, such as age, gender distribution, setting etc), and will summarise reliability, validity and responsiveness data with associated study quality indicators (COSMIN checklist).

In order to determine the best available method for measuring PFI of the spine, each identified instrument will be ranked using a range of measurement performance metrics identified in the COSMIN checklist. The findings will be presented and possible hypotheses for the results will be generated and discussed. In addition, gaps in the literature will be identified and discussed.

Ethics and dissemination

A substantial number and variety of spine impairment ratings are calculated internationally and used for financial award decisions, in both legal and compensation contexts.29–34 However, a systematic review in this area to support the use of instruments currently being used to determine PFI is lacking. The application of reliable and valid measurement instruments for assessing PFI of the spine is considered crucial.

This systematic review offers a feasible means for synthesising the evidence specific to spinal PFI assessment; and our results will likely provide unique insights concerning the breadth and depth of literature in the area. Outcomes of this review will be applicable to clinicians, policy-makers, worker’s compensation boards and health and safety organisations. In particular, findings will likely provide a foundation and direction in terms of research priorities for assessing PFI of the spine. Summarising the nature and strength of the evidence regarding the reliability, validity and responsiveness of spinal PFI measures will also inform future research and policy in this field.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.

Footnotes

  • Contributors SMG and SM conceptualised the initial review protocol and led the development of this manuscript. The entire team (SMG, SM, CB, CMT, BB, DCR, PH, LC and XZ) collaboratively drafted the manuscript, followed by numerous iterations and substantial input and appraisal from all authors. All authors have approved the final version of this manuscript.

  • Funding The Workers' Compensation Board of British Columbia (WorkSafeBC grant number RS2015-SR01).

  • Competing interests None declared.

  • Patient consent Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.