Wednesday, 25 January 2012

Science, Business, Politics or naivety in telehealth / telecare

Would the telehealth / telecare community and the evaluation team of the Whole System Demonstrator Programme please get their act together? If you are sceptical then be scientific. If you are frustrated at the slow uptake of technologies then be political. Don't try to mix the two.

The UK Department of Health (DH) funded research on telehealth and telecare, the Whole System Demonstrator programme, had its 'headline findings' published in December 2011. A press release described it as
"The Whole System Demonstrator (WSD) programme is the largest randomised control trial of telehealth and telecare in the world, involving 6191 patients, 238 GP practices across three sites, Newham, Kent and Cornwall.  WSD was set up to look at cost effectiveness, clinical effectiveness, organisational issues, effect on carers and workforce issues.  It focused on three conditions, diabetes, COPD and coronary heart disease.  The programme will provide a clear evidence base to support important investment decisions and show how technology supports people to live independently, take control and be responsible for their own health and care."
On 19th January 2012 Paul Birstow, Minister for Care Services, announced a Concordat with industry "to enhance the lives of three million people over the next 5 years by accelerating the roll-out of telehealth and telecare in the NHS and social care".

The headline findings of the Whole System Demonstrator, described by DH as one of the most complex trials it has undertaken designed to answer the question "Does the use of technology as a remote intervention make a difference?" are impressive.
"The early indications show that if used correctly telehealth can deliver a 15% reduction in A&E visits, a 20% reduction in emergency admissions, a 14% reduction in elective admissions, a 14% reduction in bed days and an 8% reduction in tariff costs. More strikingly they also demonstrate a 45% reduction in mortality rates."
However, there are a number of questions about the study:
  • 45% reduction in mortality rates! Really? So why is the data that has been hanging around for over a year not being scrutinised?
  • How come the DH can publish the results of a large Randomised Controlled Trial without peer review, quoting Relative Risk Reductions, no drop-out rates, and no confidence intervals? (Prof Trish Greenhalgh).
  • What incentives did the general practices have for putting the patients into the trial?
  • Apparently the detailed research is undergoing 'peer-review' so I wonder what the final report will look like. Is it presumptuous to talk about 'watershed' moments and "There's been more pilots in this space than British Airways. What we need to do now is drive forward at scale." Are you going to look at the evidence or not?
  • What is going to be presented in March 2012 at the Kings Fund meeting?
  • The last data was collected in September 2010, the results were asked for at a Kings Fund meeting on telehealth in March 2011 where the design was presented along with recruitment demographics and research themes. Then, later in March, the trial design was submitted (and published in August 2011) with an endnote saying that "In practice it was found that the majority of patients were exposed to one technology alone, and therefore the simpler design was eventually adopted." When was that 'eventually'? Looks like post-hoc. It is common practice to submit trial designs before recruiting patients. The GP practices were clearly biased to the fact that they thought the telehealth / telecare would be beneficial and it shouldn't be withheld from some patients.
  • Why is the Whole System Demonstrator variously described as a Randomised Controlled Trial, a field trial, a complex evaluation among other things?
  • The rumour from the test sites was that about 10% of the benefit was perhaps from the technology. Presumably the rest was from the engagement with the health professionals. If it is not all about the technology why does it appear the thrust of the investment conclusion is towards equipment? Healthcare staff will require new competencies for remote and asynchronous healthcare if they are to provide that 90% benefit.
I have a prior belief that telehealth and telecare is effective. I suspect it would be useful for certain types of patients but it may not be beneficial for all. For some it may be harmful by leaving them remotely managed but requiring face-to-face care. I'd like to be able to answer accurately the patient who asks me "so this remote monitoring equipment is going to work for me is it?" or the Medical Director who asks "Hey you're a techie type aren't you? How much of our budget should be assigned to this telehealth / telecare stuff?" I'd like to answer those uncertainties with some data.

See also:
Storify of Twitter discussion and links about #WSD
Paper by Davies & Newman on evaluating telecare and telehealth interventions including a description of 'pragmatic RCT'
Whole System Demonstrator (in Cornwall)


  1. Eight events were run from Jan 2009 to June 2011 to update people on the trial progress. Much of this was reported on the WSDAN web site (now consolidated into Kings Fund site). Most of these points raised by Dean have been discussed at the events. There is nothing really new here that has not been discussed before apart from the headline findings from December 2011.

    Detailed trial data has been analysed since Autumn 2010 and papers have been going through peer review (as always intended). A number of journal articles are expected. The outline programme for The King's Fund Congress in March is now available. Care needs to be taken on synchonisation of publication of journal articles and conference presentations.

    The trial design was adjusted early on - this has been covered at WSD events in 2008/9/10 - the telehealth and telecare combined group would have been too small for statistical significance, so it was adjusted (with ethics approval) to have separate telecare and telehealth intervention groups with their controls at an early stage. This would give the statistical significance - See Stan Newman video presentation from March 2011.

    There was a long documented protocol for contact with potential trialists who would not know whether they were in the intervention group or control group. Assessors could not make any reference to any potential benefits of the devices/services to potential trialists. Again, this process has been covered at at least 10 events I can think of.

    The trial was a cluster randomised trial as covered in the published protocol and Prof Stan Newman's presentation at the King's Fund last March (all references and links have been provided to Dean Jenkins). It was a 'field' (pragmatic) trial because each site worked in different ways with different technologies. It was a 'complex evaluation' because it involved 5 themes with 6 evaluation organisations and has been described as the world's largest RCT of telecare and telehealth. Tracking daily data, hospital admissions, GP and nursing visits to look at service utilisation change is very complex.

    The issue about technology and organisational change/culture has been well covered in the telehealth fields (see recent 2020Health report on VHA). Technology is an important part of the service but the wrap around clinician support and organisation is vital.

    Indications from telehealth programmes around the world are starting to provide more information about which groups are most likely to benefit. The drop out rates from the WSD programme and reasons have been presented by Prof Stan Newman at at least 2 conference last year.

    I suggest that Dean and others wait for the published journal articles and conference presentations to answer their questions rather than more speculation.


    Mike Clark
    Twitter: @clarkmike

  2. Thanks Mike - looking forward to the published papers. Thanks also for the storify of the discussion on Twitter. (I've added it as a hyperlink)