The NHS England's 'Board to Ward' patient feedback system with public transparency has gone public.
You can see the results for every Trust down to ward level on the Gov.UK website.
The calculation of the Friends and Family Test and guidance on how the results should be shared with staff and the public is also available. When trying to calculate the scores remember that the 'likely' response is included in the denominator but the 'don't know' is not.
The calculation is based on a net promoter mechanism (a customer satisfaction measure) popular in business but not without its criticism including for variability even in the industries where it is sometimes claimed to be superior to more robust survey methods.
What has been the reaction in the first month of national publication of the figures?
99 per cent are happy with NHS hospital care (in a survey where only 10 per cent actually responded). Usefulness of data from post-Stafford research already called into question. Independent
Family and Friends test 'at best meaningless'. The Government's family and friends test for NHS hospitals has been condemned as "at best meaningless, at worst misleading" as critics warned the system is "open to gaming". The Telegraph
Are you listening, Andy Burnham? NHS patients have been given a voice. The Friends and Family test improves transparency in the health service – and it's patients who will benefit the most. The Telegraph
I fear the gaming. Managers in NHS Trusts will put their own PR polish on their national results and not simply refer patients (and staff) to the transparent figures. They may add another layer of interpretation to the scores where none may exist. They may also try to shift the scores without making structural changes to the clinical areas themselves.
For example, a Trust may trumpet scores of 100 when the respondent numbers are very low, but then criticise a ward with scores nearer 50 when the respondent numbers are equally low. Low respondent numbers mean uncertain figures. Scores that conveniently support a particular Trust Board narrative may be selected for action in preference to others. Wards with low scores this month may have circulars to 'explain' the survey to patients and relatives so that 'the score can better reflect the true performance of the ward'.
Talk of ward scores should be responsibly done. I've been calculating the variance and margin of error on the ward scores using a discrete random variable approach. It gives an average margin of error of +/- 11.4 (range 1.1 to 43.8) for the Friends and Family score (based on the 68% of wards where variance can be estimated - the others are meaningless as numbers too small). [I've put the modified spreadsheet here ... column AA in the 'IP ward' sheet has the margin of error estimations where they are valid.]
You can see the results for every Trust down to ward level on the Gov.UK website.
The calculation of the Friends and Family Test and guidance on how the results should be shared with staff and the public is also available. When trying to calculate the scores remember that the 'likely' response is included in the denominator but the 'don't know' is not.
"Managers have widely embraced and adopted the Net Promoter metric, which noted loyalty consultant Frederick Reichheld advocates as the single most reliable indicator of firm growth compared with other loyalty metrics, such as customer satisfaction and retention. ... the research fails to replicate his assertions regarding the 'clear superiority' of Net Promoter" http://journals.ama.org/doi/abs/10.1509/jmkg.71.3.39Having said that, it is quite a simple tool and arguably more practical for patients and relatives to complete on NHS wards. More importantly, it is what we currently have and it is here to stay. It is among the mandatory central information for Monitor. It was announced by Prime Minister David Cameron last year to identify the 'best performing hospitals'. If it has a use it is possibly better for detecting change and stress within a clinical area rather than comparing between clinical or geographical areas. A short stay on intensive care where your life is saved is very different to a long stay on a stroke rehabilitation ward where you are being urged to be as physically independent as you can when you are not used to being told what to do.
What has been the reaction in the first month of national publication of the figures?
99 per cent are happy with NHS hospital care (in a survey where only 10 per cent actually responded). Usefulness of data from post-Stafford research already called into question. Independent
Family and Friends test 'at best meaningless'. The Government's family and friends test for NHS hospitals has been condemned as "at best meaningless, at worst misleading" as critics warned the system is "open to gaming". The Telegraph
Are you listening, Andy Burnham? NHS patients have been given a voice. The Friends and Family test improves transparency in the health service – and it's patients who will benefit the most. The Telegraph
PM: 1st ever #NHS friends & family stats out, giving patients far greater voice in NHS & putting a spotlight on care standards #NHSFFT
— UK Prime Minister (@Number10gov) July 30, 2013
Very strong believer in Friends and Family test because clinicians already using it to improve care. Tdy is big step on NHS journey #NHSFFT
— Tim Kelsey (@tkelsey1) July 30, 2013
I fear the gaming. Managers in NHS Trusts will put their own PR polish on their national results and not simply refer patients (and staff) to the transparent figures. They may add another layer of interpretation to the scores where none may exist. They may also try to shift the scores without making structural changes to the clinical areas themselves.
For example, a Trust may trumpet scores of 100 when the respondent numbers are very low, but then criticise a ward with scores nearer 50 when the respondent numbers are equally low. Low respondent numbers mean uncertain figures. Scores that conveniently support a particular Trust Board narrative may be selected for action in preference to others. Wards with low scores this month may have circulars to 'explain' the survey to patients and relatives so that 'the score can better reflect the true performance of the ward'.
Talk of ward scores should be responsibly done. I've been calculating the variance and margin of error on the ward scores using a discrete random variable approach. It gives an average margin of error of +/- 11.4 (range 1.1 to 43.8) for the Friends and Family score (based on the 68% of wards where variance can be estimated - the others are meaningless as numbers too small). [I've put the modified spreadsheet here ... column AA in the 'IP ward' sheet has the margin of error estimations where they are valid.]
No comments:
Post a Comment