Development and initial testing of a Health Confidence Score (HCS)

Stage 1 development

The stage one questionnaire evolved through more than 30 distinct iterations over several months, with small-scale testing and refinement. Initially, we proposed to use the question how confident do you feel? with four response options: very confident, quite confident, not very confident and not at all confident.

Four dimensions were identified:

• Knowledge—how much you understand about your health and treatment, including health literacy.

• Self-management——perceived ability to manage your health, treatment and lifestyle.

• Access to help—social proficiency to navigate the health and care system to obtain the services you need.

• Shared decision-making—participation in clinical decisions and how well staff understand your wishes.

Stage 1 version of the Health Confidence Score (January 2015) is shown in figure 1. However, in testing (see Results), we found that the distribution of scores in study 1 was narrower than hoped.

Stage 2 development

Consequently, we recast the questionnaire into an agree–disagree format and the wording evolved through 30 further iterations. The final questionnaire is shown in figure 2.

The four items are as follows:

• I know enough about my health (short term: knowledge).

• I can look after my health (short term: self-management).

• I can get the right help if I need it (short term: access).

• I am involved in decisions about me (short term: shared decision-making).

We used four response options, recognising the skewed nature of the expected distribution: strongly agree, agree, neutral (the version used in study 2 used neither agree nor disagree), and disagree. All responses are optional, there is no don’t know category. The combination of four items and four options creates a 4×4 matrix with 256 combinations.

For analysis, each option is allocated a score on a 0–3 scale, where strongly agree=3, agree=2, neutral=1 and disagree=0. Higher is better. A summary score is calculated by adding the scores for each item, giving a 13-point scale with a range from the floor 0 (4×disagree) to the ceiling 12 (4×strongly agree).

For reporting group results, the mean scores are transformed linearly to a scale from 0 to 100, where 0 indicates that all respondents chose the lowest score and 100 that all chose the highest. A common 0 to –100 scale is familiar and lets people compare item and summary mean scores on the same scale.

The UK Terminology Centre has issued SNOMED CT and Read Codes for the HCS so that the information can be recorded in the patient’s electronic health record (EHR).34

Length and readability

Questionnaire length and complexity impact response rates and data quality.35 The number of items and words in the HCS was compared with six other instruments that measure related constructs: My Health Confidence (MHC),19 Patient Health Engagement Scale,31 Stanford self-efficacy for managing chronic disease 6-item scale,36 Patient Activation Measure (PAM-13),37 ICECAP-A33 and the Health Literacy Questionnaire.38 The texts analysed are those reported by the original authors.

Reading age was measured with the Flesch-Kincaid Readability Grade (FKG), provided as part of Microsoft Word.39 In general, patients should not be asked to complete questionnaires with reading age more than 10, corresponding approximately to FKG=5.40

Testing and validation

Study 1 tested the stage 1 questionnaire and was conducted in March 2015 through YouGov using a nationally representative online sample in the UK, as part of a study on online cancer information.41 Panel members were paid a small fee for surveys completed. The four HCS questions were presented on screen (without graphics) early in the survey.

Study 2 tested the stage 2 questionnaire. During the summer of 2015, an interviewer approached a convenience sample of members of the public around Newbury, an English market town, and asked them, with their consent, to complete a short anonymous survey, presented on a two-sided laminated sheet. The interviewer entered responses into a web-based survey app using a smart-phone.

In addition to the HCS, each respondent was asked to complete the MHC survey,19 and the howRu health status measure.22HowRu has four items: (1) pain or discomfort, (2) feeling low or worried, (3) limited in what you can do and (4) require help from others; each item has four responses: none, a little, quite a lot and extreme.

We also recorded respondent’s gender, age in decades, ethnicity (White British or other), if they had children (yes or no), took regular medication (none, 1 or 2 and 3 or more medications each day)and age at which they completed education (under 17, 17–19 and over 19).

We hoped to find a broader distribution of responses in study 2 than in study 1. We expected a ceiling effect, but in spite of no strongly disagree response option, did not anticipate a floor effect.

We expected inter-item correlations in the range r=0.4–0.6 and Cronbach’s alpha between 0.7 and 0.9, which would justify the use of a summary score.42

Exploratory factor analysis, using the JASP statistics programme, was applied to the whole study 2 data set (using an oblique rotation, Promax, as we expected constructs to be correlated) for the individual questions in HCS, howRu and the two MHC questions.

Construct validity is the degree to which the scores of an instrument are consistent with hypotheses, such as relationships to scores of other instruments or differences between relevant groups, based on the assumption that the instrument validly measures the construct to be measured.43

We hypothesised that in a study of this size:

• Health confidence would be moderately associated with health status, as measured by howRu.

• There would be little difference in health confidence between men and women, being a parent, education and ethnicity.

• Health confidence would be lower in people as they became older and take more medication.

• HCS would correlate more strongly with the MHC scale than with the MHC information scale.

Ethics statement

No data were collected unless participants consented. There was no risk of substantial damage or distress to individual participants.44

Length and readability

The number of items (questions), words, FKG and reading age for the final HCS and other instruments are shown in table 1. The HCS has a low word count and reading age.

Study 1

Study 1 (using the stage 1 questionnaire) had 1031 responses. The score distribution was narrower than hoped for, with mean summary score 8.0 on the 0–12 scale, SD 2.0, range 0–12, IQR 7–9, negative skew −0.5 and kurtosis 1.1. Of note, 62% chose the same response (quite confident). This led to revision of the instrument (stage 2).

Study 2

Study 2 used the final questionnaire. In total, 381 people completed the survey. Five surveys (1.3%) contained any missing values for any item. Only complete data sets were analysed further.

The frequency distribution of each item is shown in table 2. Frequency varies from 1% (disagree that I can get the right help if I need it) to 53% (strongly agree that I am involved in decisions about me).

The mean HCS summary score on the 0–12 scale is 9.2, SD 2.4, IQR 8–11, negative skew −0.9 and kurtosis 0.4. When transposed to a 0–100 scale, the mean HCS is 76.7 (95% CI 74.6 to 78.8, SD 20.4). The ceiling state (strongly agree on all four items) accounted for 23% of ratings; the floor state (disagree on all four items) occurred only once (0.3%).

The internal structure was explored by examining the correlations between each pair of items. All inter-item correlations are in the range 0.41 to –0.59, implying that they are measuring related but different things. Cronbach’s alpha=0.82 is in the middle of the desired range.

Exploratory factor analysis results are shown in table 3. A scree plot implies two factors, while Kaiser’s criterion implies only one. This supports the view that HCS measures the same concept as the MHC question, while howRu measures a separate concept (health status). The MHC information question measures a different concept.

For demographic and independent variables, the frequency distribution, mean HCSs (0–100 scale) and CIs are shown in table 4.

The HCSs for men are lower (74.9) than for women (78.0), but not significantly (t(371) = 1.4, p=0.20). HCS is flat between the ages of 30 and 60, but then falls with age (figure 3A) (Spearman’s correlation, r=−0.22, p<0.0001).

People taking three or more medicines also have lower HCS (Spearman’s correlation, r=−0.29, p<0.0001), a result that is confounded with age.

There are no significant differences in HCSs according to ethnicity (t(370) = 0.8, p=0.4), having children (t(369) = 0.5, p=0.7) or education level (Spearman’s correlation, r=0.10, p=0.059).

HCS is moderately correlated with health status (howRu) (figure 3B) (Spearman’s correlation, r=0.49, p<0.0001).

Correlation between the HCS and the MHC Confidence item is high (figure 3C) (Spearman’s correlation, r=0.76, p<0.0001). The MHC rating has a mean of 8.4 on the 0–10 scale. The MHC scale has a narrower dispersion than HCS (Brown-Forsythe test, F(1, 751)=141, p<0.00001).45 The MHC SD is 1.3, range 0–10, IQR 8–9, negative skew −1.6 and kurtosis 6.9.

The correlation between HCS and the MHC Information item is not significant (figure 3D) (Spearman’s correlation, r=0.08, p=0.10).

Strengths and limitations

This is the first full account of the HCS, which is a short generic person-reported measure of people’s perceived confidence in managing aspects of their own health and care. It is readily usable on smartphones and tablets. It is designed for use across all health and social care sectors, together with other modules of the R-Outcomes family.

HCS is shorter than comparable measures, with a lower reading age.

Michie’s COM-B model46 suggests that behaviour change depends on people’s capability, opportunity and motivation to change what they do. The first HCS item (I know enough about my health) relates to capability. People need to understand how their behaviour impacts their health. The second item (I can look after my health) relates to both capability and motivation—they may not be able or wish to do things. The third and fourth dimensions (I can get the right help if I need it, and I am involved in decisions that affect me) relate to opportunity and how local health and care services support them.

Inter-item correlations and Cronbach’s alpha are within the desired range. These results, together with factor analysis, suggests that, while each item measures a different aspect and is useful independently, the aggregate HCS is a meaningful summary measure. A recently published study, using the HCS with measures of PWS, health status (howRu) and patient experience (howRwe) in social prescribing, confirms this.24

For populations, the 0 to –100 scale is easy to understand and facilitates comparison of mean scores for items and summary scores. The HCS summary score has a broad distribution of responses on a 13-point scale, with low kurtosis. This compares favourably with the MHC score kurtosis.

HCS is strongly correlated with the MHC score but not with the MHC information score (which measures the value of health information provided). We found no significant relationships between HCS and gender, ethnicity, education or region. Hypothesised associations are as expected, supporting the case for construct validity.

One limitation is that this study was undertaken among a convenience sample of the public which may not be representative of the general population. Using a single interviewer to collect the data may have created some bias, but this is unknown.

Comparison with existing literature

Health confidence is a broad concept, encompassing aspects of self-efficacy,47 patient activation,17 health literacy,48 self-management,49 shared decision-making,50 capability51 and empowerment.52 The focus of HCS is on how people feel (perception), not on what they do (activity). Health confidence is about the person, not about a clinical encounter, unlike measures such as CollaboRATE.53 It does not measure how well clinicians or providers engage with patients.

The HCS scope is broader than the PAM. HCS is shorter than PAM (50 vs 299 words) with lower reading age (8 vs 13 years). Independent evaluation has reported that PAM has issues in terms of implementation, complexity and understandability.54

HCS provides a summary score plus a score for each item, while the PAM groups patients into four levels and a single overall score in the range 0 to 100. For interpretation, the mean HCS item and summary scores, on 0–100 scale, are often related to arbitrary thresholds of 80–100 (high), 60–79 (moderate), 40–59 (low) and 0–39 (very low).

HCS is not a conventional outcome measure, which should always be maximised. Health confidence is sensitive to each person’s existing health problems, as is activation.55 For example, an individual without any health problems is likely to score higher (better) than one with multiple conditions. Case mix, health status and social context must always be taken into account.

The concepts of health confidence, activation and health literacy can help us understand whether interventions reach and help their target participants, or whether only those who are already health literate are being reached.56 Health confidence includes aspects of both a mind set and a skill set. Activation is mainly a mindset, while health literacy is a skill set.57 Both are moderately correlated with physical and mental health but not with each other.58

Implications for practice

At the individual level, HCS results may help clinicians to tailor decisions to patient’s needs, be shared with patients and carers and incorporated into EHRs.59 60 Availability of Read codes and SNOMED CT codes makes HCS straightforward to integrate with EHRs used in hospitals and primary care.34

At the aggregate level, HCS item and summary scores can be integrated into interactive real-time dashboards to provide rapid feedback for clinicians, managers and commissioners. The What Matters Index, which uses MHC with four other questions (pain, distress, polypharmacy and adverse drug effects), has advantages over computer-generated risk models at predicting and controlling hospital costs.61 It is likely that HCS could be used in a similar way.

The HCS meets the need for a short easy to use measure of health confidence. It can be collected on smartphones and other devices that people already use. The HCS has already been used in more than 40 evaluation and quality improvement studies, usually as part of a longer questionnaire, with measures of health status (howRu), patient experience (howRwe) and PWS, to support and evaluate patient-centred care.24 62 Surveys are administered digitally or on a single sheet of A4 paper and include a text box to allow respondents to expand on their answers.

Two variants of the HCS, which measure carers’ confidence and staff job confidence, respectively, have also been used with success. Work may be needed to validate all these measures in other institutions, population and geographic zones, although experience to date has been encouraging.