Response on behalf of the REEACT collaborative Re: Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial
CCBYOpen access
Rapid response to:
Research
Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial
Response on behalf of the REEACT collaborative Re: Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial
Response by Gilbody and colleagues on behalf of the REEACT collaborative
We thank all the correspondents who took the time to read and comment on the REEACT trial.1 Our trial has provoked important discussions around the place of computerised therapy in modern care systems and the challenges of researching such interventions. Our trial is not the last word on cCBT but does illustrate that the value of computer delivered psychological therapy cannot be assumed and that evidence of efficacy does not necessarily translate into real world effectiveness. The REEACT trial demonstrated that two computer programmes that are still routinely offered in UK NHS primary care mental health services and which are recommended by NICE did not, on average, show any benefit for people with depression when added to routine NHS primary care.
Several correspondents recognised the value of the REEACT trial but some important issues were raised. Space prevents us from responding to each, but there were a number of points which deserve some comment and response.
Efficacy versus effectiveness
Several correspondents would have preferred if we had conducted another efficacy study, rather than a test of real world effectiveness.2 There is ample randomised evidence to show that cCBT can be efficacious when delivered under ideal conditions; when compared to nothing at all or when delivered with very high levels of support (often guided by experienced psychological therapists).3 Our trial adds little to this body of literature, but there remain important uncertainties relating the real world effectiveness of cCBT and to which REEACT speaks. The majority of patients with depression are treated entirely in primary care, and the REEACT trial is one of the first (and certainly one of the largest) trials to test the effectiveness (rather than efficacy) of this technology in a primary care setting.
For GPs, patients and commissioners of services a very important question is ‘what is the benefit I can expect to see if a person with depression is offered cCBT in addition to the usual care which they already receive?’ This is a question relating to effectiveness rather than efficacy and was the question posed to the UK research community by the body which commissioned the REEACT trial – the National Institute of Health Research (NIHR) Health Technology Assessment Programme. We think our results are informative and a number of criticisms relate to the choice of a pragmatic rather than explanatory design and the primary care setting of our trial. We defend our research in the face of this criticism.
Lack of engagement with cCBT
Several correspondents have sought to explain away the negative results of the REEACT trial with reference to a number of factors which were inherent in our pragmatic design. People consented to partake in a trial of cCBT and we can therefore assume that they were interested in cCBT as a therapeutic option. We sought to engage people with weekly phone contact and reminders to use the programmes. The vast majority logged on to the systems to look at the programmes. We know this since we were able to check computer records. However very few people returned to complete a second session, and the proportion of people who completed all sessions was very small indeed. We maintain this is a very important finding and the most significant contribution of the REEACT trial. We note that none of the correspondents suggested that the level of support that was offered in the REEACT trial was atypical of that which is offered in routine NHS practice and to that end we succeeded in delivering a fair test of real world effectiveness. Our research suggests that a higher level of support than was delivered in REEACT is needed to enhance uptake. This would require a re-design of current NHS services and a level of support that is greater than that which is recommended in the two programmes that were trialed. The effectiveness of enhanced levels of support needs to be tested in large scale effectiveness trials and we have our own programme of ongoing research which speaks to this issue. In the meantime we can be confident that cCBT as delivered in UK primary care did not demonstrate uptake or effectiveness. This remains an important finding.
Cross over and dilution of effect
One explanation for the negative result offered by some correspondents was the reported use of computer resources by a small proportion of participants in the usual care arm, which might have diluted the effect of cCBT. We think this is a red herring and a diversion from the finding of lack of engagement. There would needed to have been a very substantial cross over with high levels of engagement in the usual care arm, and a very powerful effect from only a very small number of sessions for cCBT to have been effective, but for this effect to have been diluted by crossover. We do not think this explanation stands up to scrutiny, and crossover effects are to be expected in pragmatic trials. The effects of cCBT were likely to be minimal in both intervention and usual care arms based on the lack of engagement alone. Invoking Occam’s razor, this is a more parsimonious explanation.
Compared to what?
REEACT was an effectiveness trial and is one of the first trials to evaluate cCBT in terms of the actual value that can be expected under real world conditions and when compared to usual GP care. Several correspondents commented that the outcomes under usual care make it difficult to demonstrate the benefits of cCBT, and would have preferred a ‘do nothing’ comparator arm. Such studies have already been undertaken and we have noted the results of these, but they are not a sufficient level of evidence in establishing the value of new technology. Demonstrating effectiveness is always more difficult than demonstrating efficacy, but this is the level of evidence that is demanded in health technology assessment. Judging the value of treatments in the face of discordant evidence between efficacy and effectiveness studies is an unenviable task for bodies such as NICE.
Opening up the black box to understand how people engage with cCBT
The REEACT benefited from a concurrent qualitative evaluation which has been published in BMJ Open by Knowles and colleagues.4 So whilst many correspondents speculated as to why cCBT worked for some but not for others, we have the benefit of some real data to illuminate this question. The main themes that emerged were the fact that people with depression are often demotivated by this condition and they found it difficult to maintain their enthusiasm for a computer-based treatment modality. Participant experience was on a continuum, with some patients unable or unwilling to accept psychological therapy without interpersonal contact while others appreciated the enhanced anonymity and flexibility of cCBT. The majority of patients were ambivalent, recognizing the potential benefits offered by cCBT but struggling with challenges posed by the severity of their illness, limited support and lack of personalization of programme content. Both positive and ambivalent patients perceived a need for a greater level of monitoring or follow-up to support completion, while negative patients reported deliberate non-adherence due to dissatisfaction with the programme. We also learned that many people valued the regular contact provided via telephone support. We knew that several participants continued to accept support phone calls despite no longer using the computer programmes. This raises the question about what level of support is needed over and above that currently offered in the NHS for cCBT to work. This is an interesting question but is a different question from that which was asked in the REEACT trial.
(B)leading edge technology?
One correspondent pointed out that Beating the Blues and MoodGYM were not now considered to be leading edge computer technologies. We have sympathy with this position, since inherent in computer technology is the need for continual innovation. However we counter this argument with the fact that there was existing trial based evidence that these technologies could work,5 and these were NICE-recommended treatments at the time of design of the REEACT trial.
The lack of engagement and absence of a clinically important effect were so clear that the REEACT trial raises fundamental questions about the way in which people with depression interact with computers or indeed other self-help technologies. This goes beyond the choice of computer programme. We would also assert that it remains important that all new technologies should be trialed since effectiveness can never be assumed. Such technologies are never resource free, and there is an opportunity cost when healthcare systems invest in unevaluated technologies which later turn out to be ineffective.
What is to be done?
We remain optimistic that cCBT does have a role in the management of depression but the REEACT trial shows that the expectations of cCBT do not readily translate into tangible patient benefits under conditions of routine care and delivery. Our main conclusion is that the level of support that needs to be offered in the context of routine care is much higher than is currently the case in publicly funded healthcare systems such as the NHS. Our qualitative research4 and systematic review evidence3 supports this assertion, and suggests that greater levels of guidance and support might result in patient benefit. This is an empirical question which needs to be answered in a well-designed pragmatic effectiveness study with a model of care which exceeds usual levels of NHS support. Fortunately we have undertaken such a study (REEACT2 ISRCTN55310481) and the results of this research will be available in 2016.
Finally we thank the correspondents for an interesting debate, and the interest in our trial reflects the importance of this topic and the need for effective low intensity interventions for depression that can be delivered at scale in the NHS. We also note that our trial has generated a debate about the value of computer-mediated psychological therapy which has not hitherto taken place.
1. Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ 2015;351:h5627.
2. Schwartz D, Lelloch J. Explanatory and pragmatic attitudes in therapeutic trials. Journal of Chronic Diseases 1967;20:637-48.
3. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: A meta-analysis. Cog Behav Ther 2009;38(4):196-205.
4. Knowles SE, Lovell K, Bower P, Gilbody S, Littlewood E, Lester H. Patient experience of computerised therapy for depression in primary care. BMJ open 2015;5(11):e008581.
5. Proudfoot J, Goldberg DP, Mann A, Everitt B, Marks IM, Gray JA. Computerised, interactive, multimedia cognitive behavioural therapy for anxiety and depression in general practice. Psychol Med 2003;33:217-27.
Competing interests:
No competing interests
14 December 2015
Simon Gilbody
Professor
Dr Liz Littlewood, Prof Peter Bower, Dr Sarah Knowles, on behalf of the REEACT collaborative
Rapid Response:
Response on behalf of the REEACT collaborative Re: Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial
Response by Gilbody and colleagues on behalf of the REEACT collaborative
We thank all the correspondents who took the time to read and comment on the REEACT trial.1 Our trial has provoked important discussions around the place of computerised therapy in modern care systems and the challenges of researching such interventions. Our trial is not the last word on cCBT but does illustrate that the value of computer delivered psychological therapy cannot be assumed and that evidence of efficacy does not necessarily translate into real world effectiveness. The REEACT trial demonstrated that two computer programmes that are still routinely offered in UK NHS primary care mental health services and which are recommended by NICE did not, on average, show any benefit for people with depression when added to routine NHS primary care.
Several correspondents recognised the value of the REEACT trial but some important issues were raised. Space prevents us from responding to each, but there were a number of points which deserve some comment and response.
Efficacy versus effectiveness
Several correspondents would have preferred if we had conducted another efficacy study, rather than a test of real world effectiveness.2 There is ample randomised evidence to show that cCBT can be efficacious when delivered under ideal conditions; when compared to nothing at all or when delivered with very high levels of support (often guided by experienced psychological therapists).3 Our trial adds little to this body of literature, but there remain important uncertainties relating the real world effectiveness of cCBT and to which REEACT speaks. The majority of patients with depression are treated entirely in primary care, and the REEACT trial is one of the first (and certainly one of the largest) trials to test the effectiveness (rather than efficacy) of this technology in a primary care setting.
For GPs, patients and commissioners of services a very important question is ‘what is the benefit I can expect to see if a person with depression is offered cCBT in addition to the usual care which they already receive?’ This is a question relating to effectiveness rather than efficacy and was the question posed to the UK research community by the body which commissioned the REEACT trial – the National Institute of Health Research (NIHR) Health Technology Assessment Programme. We think our results are informative and a number of criticisms relate to the choice of a pragmatic rather than explanatory design and the primary care setting of our trial. We defend our research in the face of this criticism.
Lack of engagement with cCBT
Several correspondents have sought to explain away the negative results of the REEACT trial with reference to a number of factors which were inherent in our pragmatic design. People consented to partake in a trial of cCBT and we can therefore assume that they were interested in cCBT as a therapeutic option. We sought to engage people with weekly phone contact and reminders to use the programmes. The vast majority logged on to the systems to look at the programmes. We know this since we were able to check computer records. However very few people returned to complete a second session, and the proportion of people who completed all sessions was very small indeed. We maintain this is a very important finding and the most significant contribution of the REEACT trial. We note that none of the correspondents suggested that the level of support that was offered in the REEACT trial was atypical of that which is offered in routine NHS practice and to that end we succeeded in delivering a fair test of real world effectiveness. Our research suggests that a higher level of support than was delivered in REEACT is needed to enhance uptake. This would require a re-design of current NHS services and a level of support that is greater than that which is recommended in the two programmes that were trialed. The effectiveness of enhanced levels of support needs to be tested in large scale effectiveness trials and we have our own programme of ongoing research which speaks to this issue. In the meantime we can be confident that cCBT as delivered in UK primary care did not demonstrate uptake or effectiveness. This remains an important finding.
Cross over and dilution of effect
One explanation for the negative result offered by some correspondents was the reported use of computer resources by a small proportion of participants in the usual care arm, which might have diluted the effect of cCBT. We think this is a red herring and a diversion from the finding of lack of engagement. There would needed to have been a very substantial cross over with high levels of engagement in the usual care arm, and a very powerful effect from only a very small number of sessions for cCBT to have been effective, but for this effect to have been diluted by crossover. We do not think this explanation stands up to scrutiny, and crossover effects are to be expected in pragmatic trials. The effects of cCBT were likely to be minimal in both intervention and usual care arms based on the lack of engagement alone. Invoking Occam’s razor, this is a more parsimonious explanation.
Compared to what?
REEACT was an effectiveness trial and is one of the first trials to evaluate cCBT in terms of the actual value that can be expected under real world conditions and when compared to usual GP care. Several correspondents commented that the outcomes under usual care make it difficult to demonstrate the benefits of cCBT, and would have preferred a ‘do nothing’ comparator arm. Such studies have already been undertaken and we have noted the results of these, but they are not a sufficient level of evidence in establishing the value of new technology. Demonstrating effectiveness is always more difficult than demonstrating efficacy, but this is the level of evidence that is demanded in health technology assessment. Judging the value of treatments in the face of discordant evidence between efficacy and effectiveness studies is an unenviable task for bodies such as NICE.
Opening up the black box to understand how people engage with cCBT
The REEACT benefited from a concurrent qualitative evaluation which has been published in BMJ Open by Knowles and colleagues.4 So whilst many correspondents speculated as to why cCBT worked for some but not for others, we have the benefit of some real data to illuminate this question. The main themes that emerged were the fact that people with depression are often demotivated by this condition and they found it difficult to maintain their enthusiasm for a computer-based treatment modality. Participant experience was on a continuum, with some patients unable or unwilling to accept psychological therapy without interpersonal contact while others appreciated the enhanced anonymity and flexibility of cCBT. The majority of patients were ambivalent, recognizing the potential benefits offered by cCBT but struggling with challenges posed by the severity of their illness, limited support and lack of personalization of programme content. Both positive and ambivalent patients perceived a need for a greater level of monitoring or follow-up to support completion, while negative patients reported deliberate non-adherence due to dissatisfaction with the programme. We also learned that many people valued the regular contact provided via telephone support. We knew that several participants continued to accept support phone calls despite no longer using the computer programmes. This raises the question about what level of support is needed over and above that currently offered in the NHS for cCBT to work. This is an interesting question but is a different question from that which was asked in the REEACT trial.
(B)leading edge technology?
One correspondent pointed out that Beating the Blues and MoodGYM were not now considered to be leading edge computer technologies. We have sympathy with this position, since inherent in computer technology is the need for continual innovation. However we counter this argument with the fact that there was existing trial based evidence that these technologies could work,5 and these were NICE-recommended treatments at the time of design of the REEACT trial.
The lack of engagement and absence of a clinically important effect were so clear that the REEACT trial raises fundamental questions about the way in which people with depression interact with computers or indeed other self-help technologies. This goes beyond the choice of computer programme. We would also assert that it remains important that all new technologies should be trialed since effectiveness can never be assumed. Such technologies are never resource free, and there is an opportunity cost when healthcare systems invest in unevaluated technologies which later turn out to be ineffective.
What is to be done?
We remain optimistic that cCBT does have a role in the management of depression but the REEACT trial shows that the expectations of cCBT do not readily translate into tangible patient benefits under conditions of routine care and delivery. Our main conclusion is that the level of support that needs to be offered in the context of routine care is much higher than is currently the case in publicly funded healthcare systems such as the NHS. Our qualitative research4 and systematic review evidence3 supports this assertion, and suggests that greater levels of guidance and support might result in patient benefit. This is an empirical question which needs to be answered in a well-designed pragmatic effectiveness study with a model of care which exceeds usual levels of NHS support. Fortunately we have undertaken such a study (REEACT2 ISRCTN55310481) and the results of this research will be available in 2016.
Finally we thank the correspondents for an interesting debate, and the interest in our trial reflects the importance of this topic and the need for effective low intensity interventions for depression that can be delivered at scale in the NHS. We also note that our trial has generated a debate about the value of computer-mediated psychological therapy which has not hitherto taken place.
1. Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ 2015;351:h5627.
2. Schwartz D, Lelloch J. Explanatory and pragmatic attitudes in therapeutic trials. Journal of Chronic Diseases 1967;20:637-48.
3. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: A meta-analysis. Cog Behav Ther 2009;38(4):196-205.
4. Knowles SE, Lovell K, Bower P, Gilbody S, Littlewood E, Lester H. Patient experience of computerised therapy for depression in primary care. BMJ open 2015;5(11):e008581.
5. Proudfoot J, Goldberg DP, Mann A, Everitt B, Marks IM, Gray JA. Computerised, interactive, multimedia cognitive behavioural therapy for anxiety and depression in general practice. Psychol Med 2003;33:217-27.
Competing interests: No competing interests