Are “friends and family tests” useful: agree, disagree, neither, don’t know?
BMJ 2013; 346 doi: https://doi.org/10.1136/bmj.f2960 (Published 14 May 2013) Cite this as: BMJ 2013;346:f2960
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Dear Editor
The latest statistical data regarding the Friends and Family Test (FFT) has just been published by NHS England (1) covering both A&E and Inpatient data. The FFT is a survey which asks patients whether they would recommend the NHS service they have received to friends and family who need similar treatment or care. The responses to the FFT question are used to produce a score that can be aggregated to ward, site, specialty and trust level. The scores can also be aggregated to national level. The scores are calculated by analysing responses and categorising them into promoters, detractors and neutral responses. The proportion of responses that are promoters and the proportion that are detractors are calculated and the proportion of detractors is then subtracted from the proportion of promoters to provide an overall ‘net promoter’ score.
On the 16th of July 2013 Prof. Sir Bruce Keogh published a report covering 14 hospitals with persistently high mortality rates (2). His report pointed out that these 14 organisations have been ‘trapped in mediocrity’. He further added that though all these organisations showed ‘pockets’ of excellent practice, there is also significant scope for improvement. We attempted to analyse the FFT scores for these 14 trusts to determine whether the complex factors in a health care facility that could potentially lead to high mortality are reflected in the FFT scores. In the 14 trusts the response rates varied over 15 folds; ranging from 2.5% in Blackpool Teaching Hospital NHS Foundation Trust to 30.1 in North Cumbria University Hospital NHS Trust. The response rates were much higher for inpatients FFT data. No clear pattern emerged both for A&E and in patient FFT scores for the 14 trusts. One of the trusts had an A&E FFT score of 16 compared to the national average of 54. However another of the 14 had a high score of 89 for A&E but with a very low response rate. Table 1 shows the FFT scores data for 14 trusts covered in Keogh review.
The FFT score is a very blunt instrument to identify trusts with high mortality rates. However once standardised data collection procedures are adopted and response rates improved FFT scores could become a useful tool to gauge the pulse of a health care institution.
1.First results from NHS Friends and Family test published, NHS England July 2013 http://www.england.nhs.uk/2013/07/30/nhsfft accessed on 30th July 2013.
2.Review into the quality of care and treatment provided by 14 hospital trusts in England: overview report http://www.nhs.uk/NHSEngland/bruce-keogh-review/Documents/outcomes/keogh... accessed on 30th July 2013.
Imon Sultana, Foundation, Year 2 doctor
Padmanabhan Badrinath, Consultant in Public Health Medicine
Public Health Suffolk, Suffolk County Council, Endeavour House, 8, Russell Road, Ipswich IP1 2BX.
Disclaimer: The views expressed in this letter are solely of the authors in their private capacity and do not in any way represent the views of the Suffolk County Council.
Competing interests: No competing interests
NHS England released the first national data for the Friends and Family Test (FFT) on 30th July 2013 (1). Proponents of the test argue that it injects the consumer voice into the NHS, provides patients with understandable information to make choices and gives information much faster than conventional national patient surveys (2) (3). Opponents suggest that the test is not sufficiently validated, has inadequate methodological consistency (hospitals can choose how to collect responses) and asks an inappropriate question (for example: who would ever recommend a cancer service?) (4,5).
In an early analysis, we have compared the first 3 month’s findings of the FFT for inpatients with the findings of the most recent (2012) NHS Inpatient Survey (using the overall score question) at the hospital level, obtained from the UK Data Service (6). We used Spearman’s rank correlation coefficient to compare unadjusted scores for the 156 acute hospitals with results from FFT ratings. The correlation coefficient was 0.46 (P<0.001).
We also compared FFT ratings with the latest results for the Summary Hospital Mortality Indicator (SHMI) for January to December 2012, obtained from Health and Social Care Information Centre (7). For the 142 hospitals with FFT ratings, the correlation coefficient was 0.21(P=0.01).
These results suggest that there is relationship between the Friends and Family Test and other measures of hospital quality, but the strengths of associations are only mild to moderate, possibly as a result of sampling and response bias, and caution should be taken in using them as comparative measure of hospital performance.
References:
1. NHS England. Friends and Family Test Data. 2013. Available from: http://www.england.nhs.uk/statistics/statistical-work-areas/friends-and-...
2. NHS England. Introduction to the Friends and Family Test. 2013. Available from: http://www.england.nhs.uk/ourwork/pe/fft/
3. Winnett R. The wards to avoid... as decided by patients. Telegraph. 2013. Available from: http://www.telegraph.co.uk/health/healthnews/10210072/The-wards-to-avoid...
4. Patientexperience.co.uk. Criticism of the Friends and Family Test. Patientexperience.co.uk. 2013. Available from: http://www.patientexperience.co.uk/friends-and-family-test/criticism-fri...
5. Lynn P. The friends and family test is unfit for purpose | Healthcare Professionals Network | Guardian Professional. Guardian. 2013. Available from: http://www.guardian.co.uk/healthcare-network/2013/apr/09/friends-family-...
6. UK Data Service. Acute Trusts: Adult Inpatients Survey, 2012 . Available from: http://discover.ukdataservice.ac.uk/catalogue/?sn=7273&type=Data catalogue
7. Health and Social Care Information Centre. Indicator Portal. 2013. Available from: https://indicators.ic.nhs.uk/webview/
Competing interests: No competing interests
The "family and friends" test where patients are asked whether they would recommend their hospital or ward to friends or relatives is now mandatory (1).
We believe that it is grossly unsuited to some groups of patients. We care for patients with life-threatening diagnoses including acute leukaemia and high grade lymphomas. Almost all will require chemotherapy as an inpatient or outpatient; some will require a bone marrow transplant meaning weeks of often harrowing (though curative) treatment in hospital. We have trialed the question on patients who have received such treatment on our ward. This typically leads to the response: "I would hope my friends and family would never have to endure such a diagnosis and treatment, in this hospital or any other." Some patients have been actively distressed by being asked to imagine their relatives or friends in such a situation.
There is clearly a difference in the treatment required for a sprained wrist and acute leukaemia. Having one question to assess the treatment of both is, we believe, inappropriate. We urge the department of health to reconsider the wording of this question, particularly for patients undergoing cancer treatment.
References
(1) BMJ 2013;346:f2960
Competing interests: No competing interests
Getting feedback on performance and areas of possible improvement is vital, especially for such a public facing service as the NHS. However, asking unwell patients in the emergency department or acute medical wards to rate their care doesn’t seem to be the best way of doing it. Unwell patients don’t want to fill in another tick box exercise. Some of them can barely complete their lunch menu.
Besides, people in general are more likely to comment if they have an extreme opinion. If care was exceptional patients tend to bring a box of chocolates and a thankyou card to the ward on their departure. If care was substandard (or perceived as such) then patients complain, either verbally or in writing. These patients are the ones who will complete the friends and family test. I suspect there will be more in one group than the other.
If care was adequate and the patient has no complaints, they may well tick ‘neither agree nor disagree’ or even ‘agree’ that care would be recommended, but they are unlikely to leave any meaningful comments to highlight areas of excellence or the need for improvement. That’s if they fill the form in at all. I wonder if anyone is looking at the completion rate compared to number of ED attendances/inpatients in the same situation. This may well affect the reliability of any results obtained.
Finally, in my experience patient and family expectations are incredibly high now. Reviewing a patient, taking a thorough history and examination, ordering appropriate investigations, discussing the results and arranging a management plan whilst being polite and communicating well isn’t enough. Patients want 24/7 access to immediate medical review, medications that fix every problem and family discussions almost daily. Staffing levels and financial pressures mean this isn’t feasible, and until we address the mindset of the population as to the function of the NHS I’m not sure the family and friends test will really give us the answers we’re looking for.
Competing interests: No competing interests
Re: Are “friends and family tests” useful: agree, disagree, neither, don’t know?
Since April 2012 the Friends and Family Test (FFT) has become a mandatory requirement for all acute providers that cover NHS funded inpatient adult care and patients discharged from Accident and Emergency. The FTT which asks patients: ‘how likely are you to recommend our hospital ward/A&E department to friends and family if they needed similar care?’ aims to provide greater transparency about the quality of healthcare services to patients and serves as means of identifying those hospitals/hospital wards that are ‘poor performers’ in an attempt to encourage them to ‘up their game’1,2.
In July 2013 the first data from the hospital, trust and ward level results of the FTT became publically available. In order for patients to be able to use this data to make informed decisions about their care it needs to be presented in a way that is easy to understand and is not misleading in any manner. I would argue that the current method of scoring and presenting this data to the public on NHS Choices fails on both these grounds. Using St Mary’s hospital (where I undertake research) as an example the FFT test score is presented as 68 (from 173 responses) and the score for the hospital is ranked in the ‘normal range’ (http://www.nhs.uk/Services/hospitals/Overview/DefaultView.aspx?id=1672). Further details on the individual breakdown of responses (i.e. those that answered: ‘extremely likely’ (N= 121) ‘likely’ (N=48) ‘neither likely nor unlikely’ (N=1) ‘unlikely’ (N=1) ‘extremely unlikely’ (N=2) or ‘don’t know’ (N=0) is also made available for those that wish to view it.
I believe there are a number of issues in presenting the data in its current format. First, no information is provided on this webpage on how the FFT score is calculated. In particular, no mention is given to the fact that the current scoring method ignores those patients answering they would be ‘likely’ to recommend and instead treats them as ‘neutral’ responses in the calculation of the score. If patients do not know this, which I would argue the majority will not, this could lead to considerable confusion especially if patients then try and work out how the test score was generated based on the breakdown of responses by category. Second, no information is provided on the range that the test score should fall within (which is anywhere between -100 to +100); without knowing this what does a score of 68 really mean to patients? Third, if we consider the current method of scoring against the idea of presenting percentages to patients, findings from the general public show that the latter is much easier to understand and less open to interpretation2. Telling patients that their hospital scored 68 (out of 173 responses) could be interpreted in many ways. Conversely, telling patients that 98% of patients would recommend a hospital (response categories: ‘extremely likely’ and ‘likely’) is not only self-explanatory but also far less misleading as it takes into consideration all those patients that give favourable responses to the question.
Given the above I would argue that the current method of presenting data to patients on the FFT is far from patient-centred and would strongly suggest that the Department of Health reconsider the way in which this information on care is conveyed to the public.
1. Department of Health. The NHS Friends and Family Test Publication Guidance. 2013 https://www.gov.uk/government/uploads/system/uploads/attachment_data/fil...
2. Ipsos MORI. Scoring and presenting the Friends and Family Test: A review of options. 2012. https://www.gov.uk/government/uploads/system/uploads/attachment_data/fil...
Competing interests: No competing interests