41.2.1.1. Comparators & population

The guideline population is adults (age≥18) who have had an acute medical emergency (AME). It therefore exclude paediatric patients, maternity, trauma, surgery and people attending health services for non-urgent care. Our models focus primarily on interventions that occur in hospital to improve the flow of patients and patient outcomes:

1. RAT in the ED
2. Extended hours for consultants in AMU
3. Daily consultant review on medical wards
4. Extended access to therapy on wards
5. Extended access to therapy in the ED.

For 1 and 5 the population is people attending ED. For 2, its patients admitted to the AMU and for the others it is patients on medical wards (other than AMU).

The simulation model includes non-AME patients passing through the adult ED but the pathway for these patients is not specifically modelled after they have been processed by the ED.

41.2.1.2. Conceptual model

The health economics subgroup of the committee discussed the requirements of a simulation of a hospital that could evaluate costs, QALYs and explore the variation of performance over time.

Generally, the models were designed on the basis that

• Workload and case-mix (age and NEWS) is determined by season and day of the week and hour of the day. NEWS (National Early Warning Score) is a measure of acuity that uses 7 physiological parameters to determine a score ranging from 0 (low acuity) to 7 or more (critically ill).
• Case-mix (age and NEWS) determines baseline mortality, movements between locations and length of stay.
• Case-mix (age and CFS) determine average long-term survival and average utility. The Clinical Frailty Scale (CFS) uses a descriptive chart illustrating activity level. The scale ranges from 1 (very fit) to 9 (terminally ill).
• Age, NEWS and CFS are correlated.
• Interventions can affect many different outcomes:

  • length of stay which is influenced by clinical need, timely diagnosis, timely access to beds and specialist staff.
  • In-hospital mortality – sometimes a reduction in mortality is a real effect leading to substantial QALYS gained but sometimes patients will be discharged earlier so that they can die in a more preferable location.
  • Intensive care referral – we consider this an indicator of adverse events, other adverse events are captured by mortality and length of stay.
  • Medical outlying – an indicator of suboptimal care, associated with risk of death, adverse event and increased length of stay.
  • Queuing in ED – an indicator of the hospital being under stress and sub-optimal care.

Typical hospital pre-admission locations:

• Emergency Department (ED).
• Ambulatory Acute Medical Unit (AAMU) – acute medicine experts provides outpatient care for AME patients during daytime.
• Clinical Decision Unit (CDU) – short stay wards provided by emergency medicine experts. Although these are technically admissions, we have made a distinction, since they are part of the emergency pathway rather than medical pathway and in the hospital data sourced; these patients were not recorded on VitalPAC, which computes NEWS.

Typical hospital admission pathways/ locations:

• Acute Medical Unit (AMU) – where undifferentiated AME patients are assessed and managed usually for up to 48 hours.
• General medical wards (GMW) – provide level 1 care to medical patients, includes specialist wards such as gastroenterology, care of the elderly.
• Intensive care unit / high dependency unit (ICU/HDU) – the intensive medicine department providing level 2 and level 3 care.
• Specialist high care units (HCU) – level 2 care such as hyper-acute stroke unit and coronary care unit.
• Rehabilitation (Rehab) wards – longer stay wards involving occupational therapy and physiotherapy.
• Medical outliers – AME patients on non-medical (surgery, gynaecology, trauma) wards.
  Non-medical pathway – Patients that are admitted under a medical consultant but subsequently take an appropriately non-medical pathway.

41.2.1.3. Reference case

We have followed the NICE reference case.^131,135

The cost perspective taken is that of the NHS and personal social services. The health perspective was limited to the patients and not family members or staff.

We used a cost-effectiveness threshold of £20,000 per QALY in the base case. Between £20,000 and £30,000 per QALY the intervention could be considered cost effective if there are additional justifications. Future costs and QALYs were discounted at 3.5% per annum, and incremental analysis was conducted.

For our cohort analyses, we have not conducted probabilistic sensitivity analysis, since we have investigated uncertainty using a simulation model.

We have used a lifetime horizon.

41.2.2.1. RAT in the ED

In current UK practice, consultant oversight and advice is available in the ED, however, not all patients are routinely assessed with immediate consultant input. Rapid Assessment and Treatment (RAT) is where an immediate assessment by the consultant is given routinely for a subset of patients and is in addition to a subsequent (more comprehensive) assessment within the ED. The RAT assessment therefore uses additional resources in terms of consultant time and comes at an incremental cost to normal care.

In an average hospital (say, 50 medical admissions per day), a consultant would probably assess, on average, approximately 2 AME patients per hour, constituting about a third of the overall number of assessments of AME patients within ED (with the remainder focused on other presentations for example, minor injury and major trauma). If RAT assessment was in place, a consultant could potentially see 4 patients in an hour.

The likely rota arrangements which may be implemented to provide early consultant assessment within the ED are contingent on many factors, such as the numbers of patients, acuity of patients, time of day, day of week, number of consultants and middle grades available on recruitment and relative proportions of consultants/middle grades in a given department. Broadly speaking, an individual consultant might do 3 or 4 full (8 hour) clinical shifts in a week, a mixture of early (for example, 8am - 4pm), mid (for example, 11am – 9pm), or late (for example, 4pm - midnight). Consultants doing the RAT shift may see 16 patients in a 4 hour period. This is intensive work, probably broken down into shifts of no more than 4 hours in the busy periods.

Due to the potential variation in optimal staffing arrangement, the model costs patient contacts, and does not comment further on staffing arrangements.

The average full clinical assessment involves approximately 15 minutes of clinical contact time (range of 10 – 30 minutes) with a further non-clinical contact time (notes write-up and result checking) of 15 minutes. A RAT assessment is shorter, that is, 10 minutes for clinical assessment plus 5 minutes for write-up and organisation of investigations.

It was not felt necessary to stratify time spent with the patient by acuity. However, notably, very sick patients with NEWS above 6 will go to resuscitation, so are unlikely to have a RATing style assessment. Less sick patients will go to minors where RATing does not take place.

The specification of the modelled comparison is summarised in the above text box.

41.2.2.2. Extended hours for consultants

On AMU there should be a maximum of 45 patient contacts in a 12 hour day or 35 during an 8 hour day per consultant (please see Table 1 below, taken from the RCP acute care toolkit).¹⁶⁵ This equates to approximately 15 minutes per patient on average, however, for some patients the assessment may be longer (that is, 30 minutes). Generally, consultant assessment usually takes place between 8am and 8pm; however, the precise timings are variable between providers.

Table 1

Recommended number of consultants for AMU based on number of patient contacts.

Typically consultants would undertake overlapping shifts to provide such care (that is, from 8am - 5pm and 11am – 9pm or 12pm – 10pm). Due to the potential variation in optimal staffing arrangement, the model costs patient contacts and does not assume any particular staffing arrangement.

The specification of the comparison is summarised in the below text box.

41.2.2.3. Daily consultant review on medical wards

Throughout this chapter, we use the term general medical ward (GMW) to denote wards for medical patients that are not the AMU and are not high care or intensive care. These include wards that are dedicated to specific medical specialties, as well as ones that have a more generic medical population. On a GMW, a patient would be reviewed daily (weekdays) by ward staff but not necessarily with a consultant present. Nonetheless, there may be consultant input via ‘board round’ oversight rather than through direct bedside review. The additional ward rounds at the weekend would mean additional workload for junior doctors and a nurse, who support the consultant.

Daily review would increase the consultant’s familiarity with the patient and promote continuity. This would reduce the time it takes to do the review.

The specification of the comparison is summarised in the below text box.

41.2.2.4. Extended access to therapy

Hospitals generally have a dedicated physiotherapy and occupational therapy (PT/OT) service for acutely ill patients. The primary role of the therapist is to assess and improve the patient’s mobility/functioning, to make sure they are safe to go home and to avoid unnecessarily prolonged hospital stay. The therapists sometimes get involved in some of the social work function, for example, calling around to try to arrange emergency placements.

A REACT team typically consists of an OT, PT and an OT/PT support worker who cover the ED and AMU. The presence of a dedicated service on the wards and for outlying patients is more variable. In some hospitals, each medical ward will have a dedicated PT and OT, who would work Monday to Friday, 9am-5pm. At weekends, a number of patients on the ward would be highlighted for weekend input, but generally, there is very much a reduced service.

The initial assessment in ED typically takes between 30 minutes to 1 hour, with the time increased where discharge is planned. Up-skilling of both physiotherapists and occupational therapists mean that basic assessment and referral can be done by either staff member.

Once assessed, a management plan is drawn up. Typically, the patient will be reassessed once admitted on the ward (approximately 40 minutes of reassessment time) and then have 20 – 40 minutes of follow up reassessment and action of the management plan for each subsequent day on the ward. Ward based management plans are enacted by various members of the team dependent on the patient and their needs. We assume that any 1 member from a team of a physiotherapist (1 whole-time equivalent [WTE]) an assistant (0.5 WTE) or ward nurse (0.2 WTE) could be involved in any given session.

During the ward stay, the occupational therapist’s time spent on each patient will be variable, and predominantly used preparing the patient for discharge. This activity is varied and important but we have not costed this as part of the intervention, on the assumption that this activity would have to take place anyway.

The impact of extended PT/OT services is heavily reliant on the service provided in the community. The typical delay to discharge varies but is often due to capacity of care agencies at a weekend. In addition, the home environment of the patient might be unsuitable for early discharge without several adaptations.

The specification of the modelled comparison is summarised in the below text box.

Admitted patients

For the cohort model, the case mix (CFS and NEWS) by age of admitted patients was determined using a UK audit of 2990 patients attending Acute Medical Units (AMUs) – SAMBA 2013¹⁸⁸ – see Table 2 and Table 3. At the time, this was the most recent year of the annual audit that was available for bespoke analysis. The audit used a modified version of NEWS that omitted responsiveness (AVPU scale - alert, voice, pain, unresponsive).

Table 2

CFS distribution of admitted patients by age.

Table 3

NEWS distribution (%) of admitted patients by clinical frailty score.

For the simulation model, the case mix of age and NEWS were determined by data from the Queen Alexandra Hospital – see Appendix E. In the absence of specific CFS data, a CFS distribution was assumed for each age-NEWS group (0, 1-4, 5-6, 7+), using the SAMBA 2013 data. The Portsmouth data allowed calculation of the full NEWS score and ‘NEWS minus AVPU’. Therefore, at admission, we allocated each patient both a NEWS score and ‘NEWS minus AVPU’ score; a CFS score was then randomly allocated based on age and ‘NEWS minus AVPU’.

Patients discharged from the Emergency Department

We ascribed a CFS score to patients, using the age-CFS distribution in SAMBA 2013 – see Table 2. The patients being discharged from ED were less frail on average than those patients who were admitted to hospital since they were considerably younger.

We did not have NEWS data for patients discharged from the ED and therefore we assumed that the NEWS-CFS distribution by age was the same as for admitted patients, again using SAMBA 2013 – see Table 2. Hence, NEWS in ED was on average lower for patients discharged from ED, since they were considerably younger on average.

41.2.4.1. Timing and number of AME presentations

For the cohort model, we take English A&E attendance data from Hospital Episode Statistics (HES)⁷⁷ to estimate time and day of arrival distributions at ED - Table 4.

Table 4

Number of A&E attendances by hour of arrival, 2014-15.

For the simulation model, we use data from the Queen Alexandra Hospital, Portsmouth – see Appendix E. These presentations were also stratified by time of day, day of week and season. There was also data on the number and source of direct admissions (those not passing through the ED).

41.2.4.2. Admissions from ED

For the proportion of ED presentations arriving by ambulance, 30.5% was taken from national data¹¹⁸.

For the cohort model, admissions rates were derived from a sample of 5 hospitals (n=412,500)¹³²:

• Admission rate for patients arriving by ambulance, 42.6%.
• Admission rate overall for all ED attendances, 28.9%.
• Proportion of admissions that arrived by ambulance, 39.1%.

In the model, we made the simplifying assumption that those arriving by ambulance were dealt with in majors.

For the simulation model, admission rates were from the Queen Alexandra Hospital, Portsmouth, and they were stratified by age group, time of day, day of week and season – see Appendix E.

41.2.4.3. ED mortality and length of stay

For both models, mortality in the ED was taken from Hospital Episode Statistics and was 20,388/19,556,781 (0.1%).⁷⁶

ED length of stay features only in the simulation model; these data came from the Queen Alexandra Hospital, Portsmouth, and they were stratified by discharge destination (CDU, Ward, AAMU, discharge) – see Appendix E. The mean length of stay was 157 minutes (2.6 hours).

41.2.4.4. Inpatient mortality and length of stay

For the cohort model, inpatient mortality (5.8%) and average length of stay (6.4 days) were calculated by a NICE analyst in a bespoke analysis of HES data restricted to medical treatment specialty in the first finished consultant episode, adults and emergencies and excluding day cases.

For the simulation model, these data came from the Queen Alexandra Hospital, Portsmouth, and they were stratified by age, NEWS and current hospital location – see Appendix E. Length of stay was also stratified by next location. The probability that admitted patients die in AMU (1,039/110,995=0.9%) or GMW (6,194/97,521=6.4%) was also used in the cohort model.

41.2.4.5. Referral to intensive care and other movements within the hospital

The simulation model distinguishes between the following parts of the hospital:

• Emergency department (ED)
• Clinical decision unit (CDU)
• Ambulatory acute medical unit (AAMU)
• Acute medical unit (AMU)
• General medical wards (GMW)
• Intensive care unit / high dependency unit (ICU/HDU)
• Specialist high care units (HCUs)
• Medical outliers.
• Non-medical pathway.

Data on movements between these locations was from the Queen Alexandra Hospital, Portsmouth – see Appendix E. This was mainly used in the simulation model only. The probability that admitted patients go to the ICU/HDU from AMU (339/110,995=0.3%) and from GMW (866/97,521=0.9%) was also used in the cohort model.

41.2.4.6. Discharge

Data on discharge destination and time of discharge was from the Queen Alexandra Hospital, Portsmouth – see Appendix E. This data is not used in the cohort model.

41.2.5.1. RAT in the ED

41.2.5.2. Extended hours for consultants in AMU

41.2.5.3. Daily consultant review on medical wards

All these treatment effects apply to everyone who receives the intervention, therefore no adjustments need to be made to the MS Excel cohort model:

41.2.5.4. Extended access to therapy in the ED

41.2.5.5. Extended access to therapy on medical wards

41.2.6.1. Literature review

No study included within the guideline reviews reported survival rates for an undifferentiated AME population beyond 30 days.

A systematic search was conducted with the aim of finding long-term survival outcomes for a generic population. We were specifically interested in survival numbers/rates, survival curves or standardised mortality ratios (SMRs). An SMR is equal to the number of deaths in an AME population divided by deaths in the general population with the same age/sex distribution.

The search retrieved 1187 records. Titles and abstracts were sifted with the following exclusions:

• Publication date prior to 2006 (a 10 year publication cut off).
• Studies where population was not from North America, Australia or Europe.
• Studies with no indication from abstract or title that the population has had an acute event/emergency (that is, simply focused on chronic management).
• Studies looking at very specific subpopulations of 1 condition, that is, after a specific surgery, with a particular complication.
• Studies that had follow-up of less than 1 year.

From the search, only 1 paper was retrieved that reported long term survival of a generic AME population group.¹⁷¹ A search on Google Scholar, PubMed and the journal’s website for all citing papers retrieved a further 14 English language results, only 1 of which reported relevant outcomes for a non-condition specific medical emergency population.^72,73

The first study, a Swedish retrospective cohort study reported standardised mortality ratios for a population of non-surgical patients admitted after visiting the ED (n =6,263).¹⁷¹ Data was collected between 1995 and 1996, with follow up 10 years (median 9.6 years). The mean age of the cohort was 62.6. The main causes of death (SMR) were related to seizures (2.62), intoxications (2.51), asthma-like symptoms (1.84), hyperglycaemia (1.67) and chest pain (1.2). Authors note that reference population has lower than typical mortality for Sweden. The reported in-hospital mortality rate was 5.20%.

The second study, an Icelandic retrospective 6 year cohort study, reports standardised mortality ratios of a population of patients attending ED (n =19,259), with findings stratified by age and sex.^72,73 The hazard ratio calculated for the age group 80 to 84 was 1.33; however, for younger ages the hazard ratio was considerably higher. Data was collected between 1995 and 2001, with follow up at death or at study end for enrolled patients. The main causes of death (percent of all causes of death) were related to malignant neoplasm (32%), ischaemic heart disease (21%), cerebrovascular disease (10%) and chronic lower respiratory disease (5%).

To calculate survival curves we chose to use the SMRs from the Icelandic study since they were based on a larger cohort and were age group-specific, and therefore survival can be tailored more distinctly to case-mix and individual patients within the simulation model– see Table 7. Iceland has longer life expectancy than England therefore, we would expect crude mortality rates to be lower but it is not clear whether the SMRs would be an under or over-estimate.

Table 7

Aggregated standardised mortality ratios after an AME from Gunnarsdottir et al (2012) n=19,259.

41.2.6.2. Analysis of 90-day mortality using HES linked to ONS mortality

NHS digital has published linked HES-ONS mortality data aggregated by primary diagnosis (3 character ICD10). This reports mortality at 30, 60 and 90 days post admission for admitted patients in 1617 diagnostic categories:

• http://content.digital.nhs.uk/article/2677/Linked-HES-ONS-mortality-data

The most recent year published is 2013-2014:

• http://content.digital.nhs.uk/catalogue/PUB16081

We used this published data to calculate standardised mortality ratios (SMRs) for the first 90 days after admission for an adult AME by taking the following steps:

1. Removed diagnostic categories where emergency<50% or adult<50%.
2. Removed diagnostic categories which are non-medical (for details see below).
3. Added up number of deaths at each time point across the categories (a).
4. Extracted the age-sex profile of each included category.
   1. Had to assume sex split was the same for each age group (within a diagnostic category).
5. Calculated the expected deaths from ONS England life table for each age-sex group.¹⁴³
6. Added up number of expected deaths across all categories and all age-sex groups (b).
7. Calculated the standardised mortality ratio SMR=a/b and 95% confidence intervals.⁶⁶

To remove diagnostic categories that would not normally be dealt with through the adult medical pathway (trauma, surgery, gynaecology/obstetrics, paediatrics and psychiatry) – step 2 - 3 physicians from the guideline’s health economic subgroup went through the remaining diagnostic codes and marked them as being either i) likely to be medical, ii) unlikely to be medical or iii) uncertain / combination. There was complete agreement for 500 categories, a majority decision for 57 categories and 13 remained uncertain. It was decided to use a priori in the model; the SMRs based on diagnostic categories where there was complete agreement or a majority (Table 8) but we computed them separately for comparison (Table 9).

Table 8

Standardised mortality ratios used in base case.

Table 9

SMRs, by level of consensus around diagnostic inclusion.

The cohorts include some elective episodes and children and therefore this method certainly under-estimates the crude death rates of adults having an AME (Table 10). Whether it biases the SMRs is not clear – the inclusion of elective patients will under-estimate them but the inclusion of children might over-estimate them. Despite this, the mean stay was almost identical to what we have found by other means (Table 5).

Table 10

Cohorts used to calculate SMRs.

Table 5

In-hospital mortality and length of stay.

The ‘uncertain’ cohort was somewhat different to the base case (Table 10) in that there were proportionately fewer men, fewer emergencies and more day cases. This contributed to lower crude mortality. SMRs were comparable apart from the first 30 days, where they were substantially lower for the ‘uncertain’ cohort (Table 9). By far the largest diagnostic category in the ‘uncertain’ cohort was ‘abdominal or pelvic pain’ – these patients could take either a medical or a surgical/gynaecological pathway, depending on local hospital and patient factors. The ‘uncertain’ cohort was left out of the SMRs used in the model but including them would have made little difference, given the relatively small cohort size.

41.2.6.3. Calculating survival curves

A typical cohort model might use the mean age of the population and calculate life-years (mean survival) accordingly. However, for a patient level simulation, the expected life expectancy of an individual patient respective to their age (and case-mix) is required. In our models, therefore, expected life years and QALYs were modelled for each age between 18 and 100.

In the cohort models, life years and QALYs found for each specific age were then weighted by the age distribution of the population to find the expected average QALY for the cohort. Similarly, in the simulation model, the QALYs accrued by each patient are aggregated to find an average for the population.

Our approach was to produce survival curves for each age by multiplying together mortality rates taken from national life tables for England¹⁴³ with standardised mortality ratios (SMRs) for AME patients.

For all patients we used the SMRs in Table 8 for the first 90 days and then thereafter the age-specific SMRs in Table 7. To verify this approach we compared the 30-day mortality from our baseline model, 4.0%, with a published estimated for England based on 12.7 million ED attendances between April 2013 and February 2014, 4.3%¹¹⁸. We considered this to be reasonably close.

Figure 1 shows an example survival curve for a person aged 85 after an AME using this method compared with the general population of the same age. We calculate life-years as the area under the curve.

Figure 1

Survival of an 85-year-old after admission for an AME.

41.2.6.4. Capturing frailty

Figure 1 shows estimated survival for the cohort as a whole but some of the interventions we are evaluating are targeted at the frail elderly. The survival for these patients will be poorer than that for a similar cohort who are not frail. To avoid over-estimating QALYs gained, we attempted to estimate survival curves that were both age-specific and frailty-specific. As noted above, we have used the Clinical Frailty Score, since this has been used in the Society for Acute Medicine’s benchmarking audits – see 41.2.3. Rockwood and colleagues¹⁶³ analysed survival for a sample of 2305 elderly patients who participated in the second stage of the Canadian Study of Health and Aging (CSHA). They were aged over 65 (mean age 85). They estimated a mortality hazard ratio of 1.3 for each increment on the CFS (note that they also showed Kaplan-Meir curves for the cohort as a whole but we could not use these directly since, follow-up was only for 5 years and when we fitted curves to them, the best fit was the exponential function, which did not seem plausible for the longer-term, especially for the lower frailty scores).

We used the hazard ratio to estimate, for each patient age 65 and above, a survival curve that is both age and CFS-specific as follows:

• We have calculated a survival curve for all patients at a specific age (for example, Figure 1).
• We define each point on the survival curve as being a weighted average of the survival curves for each of the individual CFS scores.
• For the weights, the proportion of patients in each CFS score group at that age, we use the SAMBA 2013 (see Table 2).
• Using the hazard rate of 1.3, if we know the mortality for CFS1 then we also know it for the other CFS groups.
• At each point of the survival curve, given the specific set of weights and the hazard ratio of 1.3 there is a unique mortality for CFS1 that is consistent with the mortality for that age as a whole. We solved this for each point using the Goalseek tool in MS Excel.
• By joining up the CFS1 survival for each point gives a survival curve for CFS1, and so on for the other CFS score groups.

As an illustration, Figure 2 shows a set of survival curves for a person aged 85 after being admitted with an AME and for selected CFS scores. The CFS5 survival curve is similar the weighted average, since 5 is the median CFS at this age.

Figure 2

Survival curves for a person aged 85, by CFS.

41.2.6.5. Application of mortality treatment effect

To assess the treatment effects on mortality in the cohort model, we estimated impact on 30 day mortality of each intervention (41.2.5) and then re-calculated the survival curve for each age and then added up the life-years.

To assess the treatment effects on mortality in the simulation model, we took a slightly different approach. There was a mortality risk in each location within the hospital. These location-specific risks were modified according to the treatment effect (41.2.5). Post-discharge (up to 30 days from admission) the patients had a risk of death that was specific to their age and CFS score – this was estimated by subtracting age-specific in-hospital mortality from their age and CFS specific 30-day mortality. For the period beyond 30 days, each individual had a life expectancy, again related to his or her age and CFS score, using the method described above but omitting the first 30 days.

41.2.7.1. Identification of relevant evidence

Three systematic searches were conducted to find appropriate utilities to populate the model. The first was conducted for a general AME population and returned 662 titles, of which 12 papers were found to be suitable for review.^{3,9,45,60,67,85,86,164,170,175,192,193} The second search conducted aimed at finding any utilities reported for a population stratified by clinical frailty score. Of the 6 titles returned, 1 paper was reviewed for relevance.¹² The third search conducted aimed to find any utilities reported for a population stratified by NEWS, no titles were returned.

Of the 13 studies identified for relevance:

• Six studies were excluded due to poor applicability or quality that is, inappropriate quality of life measure employed.^{45,60,85,86,175,193}
• Two studies were conducted in the UK, both reporting EuroQol 5-Dimensions (EQ5D):

  • Goodacre et al. 2012 reports on quality of life experienced 30 days after admission by admitted patients who arrived by ambulance.⁶⁷
  • Round et al. 2004 reports quality of life at presentation and at 6 months for patients aged 70 and over who have experienced acute care.¹⁶⁴
• Two European studies report quality of life specifically for patients who have had an ICU admission, both reporting the EQ5D:

  • Sacanella et al. 2011 (Spain) reports on patients experiencing a medical condition and ICU aged 65 and over at the study start, discharge and 12 months.¹⁷⁰
  • Vainiola et al. 2011 (Finland) reports quality of life for emergency patients admitted to ICU/HDU at 6 and 12 months post treatment, stratifying by age.¹⁹²
• Three studies could be considered for longer term quality of life, all reporting use of EQ5D:

  • Bagshaw et al. 2014 (USA) reports quality of life experienced by people who had a critical care admission and stratifies by clinical frailty score.¹²
  • Ara and Brazier. 2011 (UK) report condition specific quality of life, stratified by age, using health surveys.⁹
  • Agborsangaya et al. 2013 (Canada) report quality of life experienced by people with a chronic condition within the last 12 months.³ This study was selectively excluded in light of similar evidence for a UK population.⁹

The reviewed quality of life papers are also summarised in Table 11 with rationale for inclusion and exclusion.

Table 11

Summary of utility evidence.

41.2.7.2. Quality of life after an AME

Utility values of those surviving 30 days post admission were taken from a UK study of patients recently admitted to hospital with a medical emergency.⁶⁷ The study uses responses to a EQ5D self-completed questionnaire. They report a utility of 0.45 (SD of 0.36) for the whole cohort where a utility of zero was given to non-survivors. Utilities of survivors only for application in the model were calculated and a breakdown by age is given in Table 12.

Table 12

Health utility estimates 30 days post admission stratified by age.

Utility values of those surviving 6 months post admission are reported by a UK prospective cohort study of patients aged over 70 with an acute illness requiring hospital admission.¹⁶⁴ The study uses responses to a EQ5D self-completed questionnaire. The findings are reported by either those attending a district general hospital or attending a community hospital. The utilities are reported for the study start point and a mean change score for 6 months is given in Table 13.

Table 13

Health utility estimates over six months.

The populations and findings from the 2 UK studies^67,164 appear comparable. Taking data from Goodacre et al. 2012⁶⁷, the weighted utility for patients 70 and over was 0.53 (at 30 days). Taking mid points of age categories, the mean age for this group was 81. Round et al¹⁶⁴ who studied patients aged 70 and over who were admitted with an acute illness, whose condition could have been fully treated in either a district general or community hospital. They found a mean utility of 0.36 at the start of the study (timing was undefined) and 0.57 at 6 months post admission. The median age of participants was 81.

A US study reports utility values for a population of critically ill patients, stratifying by clinical frailty score.¹² This study reported EuroQol visual analogue scale scores for each of 2 groups based on clinical frailty scores: 1 group with a score from 1 to 4 and the other group with a score greater than 4, representing the most frail group. We noted that those who have a CFS score > 3 have a utility 21% lower than the utility of those who were considered non-frail.

Table 14

Utilities by Clinical Frailty Scale score at 6 months.

41.2.7.3. Quality of life by age for people with chronic condition

Ara and Brazier⁹ report expected utilities stratified by age group and common health conditions for a UK population (Table 15). Utilities for a patient population without a history of any health condition are reported for comparison.

Table 15

Quality of life by age for the general population – with and without a history of a health condition. Ara and Brazier.

41.2.7.4. Application of utility data in the baseline scenario

Three studies were used to estimate baseline quality of life.

• Goodacre et al. 2012⁶⁷ reports applicable and complete data for quality of life experienced 30 days after admission by patients arriving by ambulance, however, the study did not report change in quality of life overtime.
• Bagshaw et al. 2014¹² indicates the difference in utility between frail and non-frail patients.
• Ara and Brazier 2011⁹ provide utilities by age group for people with chronic conditions.

Ara and Brazier⁹ report condition specific quality of life, stratified by age, using health surveys in a UK population. These represent upper estimates of long-term utility after an AME. We use these for utility for non-frail patients. Using this data, quality of life declines over time as the patient gets older. The committee were aware that for some patients, quality of life declines significantly after an AME, whereas others return to their usual quality of life. It is assumed in the model that those who are considered frail (CFS≥5) will have no utility improvement after an AME. Those who are not frail will have their utility linearly improve to the average age-specific quality of life described in Ara and Brazier⁹ for an individual with a health condition 1 year post AME.

Taking the above into account, the baseline utility used in the model is age dependent and informed by the proportion of that age group that are considered frail upon admission:

• Depending on the individual’s age, a utility value is taken from Goodacre et al, as described in Table 12.
• As this value represents the average utility for both frail and non-frail, it is then adjusted based on the assumption that those who are frail have a quality of life 23% lower than those who are not frail, as described in Bagshaw et al.
• If the individual is not frail then their quality of life will increase at a linear rate until 1 year when it reaches the age-specific quality of life of the general population, with a health condition, as described in Table 16.
• As the patient gets older, their quality of life changes in line with the values presented in Ara and Brazier but with the smoothing applied.
• If the patient is frail, it is assumed that their quality of life will remain unchanged for the remainder of their life.

Table 16

Utility over time in the baseline scenario for patient age 80.

This approach is illustrated in if the individual is not frail then their quality of life will increase at a linear rate until 1 year when it reaches the age-specific quality of life of the general population with a health condition, as described in Table 16.

41.2.7.5. Application of the quality of life treatment effect

The treatment effect for extended access to physiotherapy and occupational therapy was elicited from the experts of the committee’s health economics subgroup. These were multipliers and were applied for 1 year only in the base case analysis and for 5 years in a sensitivity analysis.

41.2.7.6. Quality of life within hospital

The models do not take into account incremental quality of life within the hospital period explicitly. There was no evidence for in-hospital quality of life improvement for the interventions we looked at and a modest gain in quality of life over the course of an admission would have a negligible impact on the long-term QALYs. To avoid over-estimating the benefits of reduced length of stay, we assumed the same utility in hospital as post-discharge up to 90 days.

41.2.8.1. Intervention (Staff) costs

Table 17 gives details of the staff time in the interventions, as decided by the Guideline’s health economics subgroup.

Table 17

Staff time.

The unit cost of staff were reported by the Personal and Social Services Research Unit.⁵⁰ These costs were adjusted to reflect on-call salary enhancements and whether the work was in premium or non-premium time. Standard NHS contract policy documents were consulted to determine any additional cost associated with out of hours and premium time, inclusive of enhancements to salary due to rota and on-call arrangements.^137–140 Since most of the interventions involve extending services further in to unsocial hours, it is important to capture the incremental costs associated with these hours. The full break down of these costs is shown in Table 18 and Table 19.

Table 18

annual wage costs used in the models.

Table 19

overhead costs associated with staff time.

Table 20

Cost of staff time.

41.2.8.2. Pathway and downstream costs

The models analysed the subsequent impact on hospital costs associated with the interventions. Table 21 below details the unit costs used.

Table 21

Unit costs of health care.

For post-discharge costs, we used the 3-month costs for patients followed up after being admitted to an AMU. In the base case analysis, we did not include other costs in extra months of life, since only disease-specific costs should be included in the NICE reference case. However, in a sensitivity analysis we included age-specific annual NHS costs calculated by the Nuffield Trust.^16,162

41.2.9.1. Interpreting the results

NICE’s report ‘Social value judgements: principles for the development of NICE guidance’¹³³ sets out the principles that committees should consider when judging whether an intervention offers good value for money. In general, an intervention was considered cost-effective if either of the following criteria applied (given that the estimate was considered plausible):

• The intervention dominated other relevant strategies (that is, it was both less costly in terms of resource use and more clinically effective compared with all the other relevant alternative strategies), or
• The intervention costs less than £20,000 per quality-adjusted life-year (QALY) gained compared with the next best strategy.

Where we compare several interventions, we use the net benefit to rank the strategies based on their relative cost-effectiveness. The highest net benefit identifies the optimal strategy at a willingness to pay of £20,000 per QALY gained.

41.4.1.1. Differences between the simulation model and the cohort model

By modelling hospital flow in the simulation model, we are able to estimate the incidence of medical outliers and the consequences for costs and health outcomes that are not assessed in the cohort model (41.3). The simulation model evaluates the same interventions as the cohort model. It is also being used to estimate the benefits of reducing delayed transfers of care for patients being transferred to a care home.

The cohort model can therefore be seen as the impact on costs and health outcomes if there were no changes to hospital flow arising from the interventions. This may be the case in some hospitals if they have few medical outliers.

By modelling individual patients, the simulation model can model some of the effects more precisely; since the effects can be applied directly to the transition probabilities (see 41.2.5). In addition, by modelling individual patients, the simulation model can better deal with the correlation between different patient characteristics (age, NEWS, CFS and mortality).

For some of the comparisons, the cohort model contained intervention costs in the baseline as well as in the intervention arm. For the simulation model, only the incremental intervention costs were included in the intervention scenarios and no intervention costs were included in the baseline scenario, on the assumption that they are incorporated within bed-day costs. The impact on cost effectiveness should be the same but it allowed the simulation model to have only a single Baseline scenario.

For the cohort models, results were reported per 1000 patients, whereas for the simulation model results are reported based on a single large DGH. Three different cohorts were used in the cohort analyses depending on the analysis (ED patients, AMU patients and GMW patients). For the simulation model, the population includes everyone presenting at ED plus direct non-elective medical admissions plus direct referrals to the ambulatory AMU. Hence mean QALYs and mean costs will reflect the cohort. However, this difference in approach should not affect the cost effectiveness result, such as the magnitude of the incremental cost per QALY gained.

The simulation model does utilise mode data that is specific to one hospital rather than national data (41.4.4) but that hospital was broadly similar to the national average in most respects (Appendix E).

During construction, the cohort model has been useful in checking the validity of the output of the (more complex) simulation model (see 41.4.8).

The run time of the simulation model has limited the number of sensitivity analyses that can be performed. Therefore, the cohort model has been useful in exploring the robustness of the model results (see 41.4.7).

41.4.4.1. Data

The data sources for the simulation model have been described above (41.2). Much of the data comes from a bespoke analysis of data from a large DGH, the Queen Alexandra Hospital, Portsmouth (Appendix E). The bed numbers were estimated as part of the bespoke analysis. However, the bed numbers used in the simulation were moderated to achieve a representative simulation of the hospital and processes not provided within the data analysis (see 41.4.6). GMW beds were adjusted until the model produced an average number of medical outliers within 1 year close to the 1800 seen in the data analysis. Once calibrated to achieve the correct number of medical outliers, the bed numbers and more detailed baseline results, including bed occupancy in the AMU and GMW, were discussed with the health economic subgroup as a sense check. ED trollies are the first constrained resource within the model. In times of pressure, the hospital flow will back up all the way to the queue for ED trollies. Therefore, the queue into the ED is the final choke point within the model. The ED queue can be affected by the flow of patients at other points within the hospital. The final bed numbers used can be found in Table 25.

Table 25

Bed/trolley numbers in the model.

A review of the effects of weekend admission on mortality was conducted (Appendix C). It is difficult to control for case-mix in this area. The studies that included ED presentations in addition to admissions suggested that case-mix could explain most of the observed weekend effect. Therefore, we decided not to include an explicit weekend effect, other than by varying case-mix (age and NEWS on admission) by day of week.

41.4.4.2. Sampling of probabilities

For patient movements, the model uses cumulative probabilities (see for example Table 26). Random numbers between 0 and 1 are generated to determine which route, so for the example in the table, a number of 0.6 would send the individual on to usual residence, whereas a value of 0.3 would send them to the GMW. The probabilities are stratified by: current location, age group, NEWS group and whether it is their first admitted location:

• Age groups

  • 16-44, 45-64, 65-75, 75-85, 85+
• NEWS groups

  • 0, 1-4, 5-6, 7+
  • Zero indicates normal healthy life signs. A score of 7+ indicates referral to critical care outreach.

Table 26

Transition probability for patients in AMU age group 16-44, NEWS group 1-4 and it is their first admitted location.

This approach is also used to determine:

• The arrival time of patients across the week.
• Discharge time of patient across the day.
• Patient case-mix (age, NEWS, CFS).
• Change in NEWS group over the stay in each location.
• The next location in the patient pathway.

The model controls for the case-mix of patients within the model by using identical random number streams for comparative runs. This means that for a given run, the number and case-mix of patients is identical for each scenario. However, the course that an individual patient can take can vary considerably, depending on:

• whether they receive the intervention,
• whether changes to system performance affect their pathway (indirectly caused by the intervention), and
• random variation.

41.4.4.3. Sampling of other inputs

For some variables in the model, the model creates distributions from which to sample. For example, patient length of stay in each location is determined by sampling from a lognormal distribution created using a mean and standard deviation from the data analysis found in a lookup table that is stratified by current location, next location, current NEWS group and age group. The sampled length of stay is capped at a maximum of one year for each location, to avoid sampling long lengths of stay that would not be captured in the model run time. The patient’s actual length of stay in a location in the model will differ from that which is initially sampled for them for a number of reasons:

• If their next destination is full then they might have to wait until a bed becomes available.
• If the GMW is full then they might be discharged slightly earlier (see Table 27).
• If GMW is full they might be made a medical outlier (see Table 27).
• If they are due to be discharged then their length of stay will be adjusted to fit the discharge time profile.
• They might receive an intervention that reduces their length of stay (Table 6).

Table 27

Decision rules built into the simulation model.

In other instances, probability profiles have been generated using data from the bespoke analysis. Probability profiles have been used where the patient needs to sample from a bespoke distribution. Probability profiles have been used for the following:

• Time presenting to hospital.
• Preadmission length of stay.
• Discharge time.

Post-discharge mortality up to 30-days and lifetime QALYs from 30-days, each by age and CFS were calculated in MS Excel in the manner described in section 41.2.7. These are then applied to patients in the simulation model using a lookup table.

Rapid Assessment and Treatment (RAT) in the ED

RAT cost £8,401 per 1,000 hospital presentations. RAT reduced admissions and in turn saw small reductions in ED length of stay, total hospital bed days, four-hour breeches, AMU bed occupancy and mortality. The impact on hospital flow and resource use did not offset the cost of RAT and did not return any QALY gain. As a result, RAT was dominated in the base case by standard care, due to an increase in costs (£7,973) with no QALY gain.

Extended access to therapy in the ED

Extended access to therapy in the ED cost £4,240 per 1,000 hospital presentations. Extended access reduced admissions with a small impact on ED length of stay, total hospital bed days, 4 hour breeches, AMU bed occupancy, medical outliers and mortality. The impact on hospital flow and resource use did not offset the cost of extended access to therapy in ED, with a total cost per 1,000 patients of £4,110. However, there was a mean QALY loss of 0.1 per 1000 patients. As a result, extended access to therapy in ED was deemed not cost-effective in the base case analysis.

Extended access to consultants in the Acute Medical Unit (AMU)

Extended access to consultant in the AMU cost £1,808 per 1000 hospital presentations. There were small reductions in total hospital bed days, 4 hour breeches, AMU bed occupancy, medical outliers and mortality. The impact on hospital flow and resource use did not completely offset the cost of the intervention, with a total cost per 1000 patients of £1,122. There was a mean QALY loss of 0.02 per 1000 patients. As a result, extended access to consultants in the AMU was deemed not cost-effective in the base case analysis.

Daily consultant review on the medical wards

Daily consultant review on the medical wards cost £12,457 per 1000 hospital presentations. There were large reductions in total hospital bed days, medical outliers and mortality. The impact on hospital flow and resource use did not offset the cost of the intervention, with a total cost per 1000 patients of £7,704 and there was a QALY gain of 0.07. the cost per QALY gained was £106,504 and therefore it was not cost effective in the base case analsis.

Extended access to therapy on the medical wards

Extended access to therapy in the medical wards cost £17,441 per 1000 hospital presentations. Extended access saw large reductions in total hospital bed days, four-hour breeches, AMU and GMW bed occupancy, medical outliers and mortality. The impact on hospital flow and resource use offset the cost of extended access to therapy in wards, with a total saving per 1000 patients of £7,209 driven by the large reduction in medical outliers. There was also a mean QALY gain of 0.85 per 1000 patients. As a result, extended access to therapy in the medical wards was deemed to be dominant in the base case, due to large cost savings and increase in QALYs.

RAT

The cohort model showed that the reduction in admissions from providing a RAT service would not compensate for the cost of providing the intervention. Given there were no predicted health outcomes from providing this service, it was dominated in the base case. In an optimistic scenario where the benefits of RAT were explored fully, the committee agreed that there might be a very modest reduction in ED mortality. However, even in this scenario, RAT was not cost effective with an ICER of £98k per QALY, which far exceeds the £20,000 per QALY threshold. The simulation model further explored the impact the intervention may have on hospital flow. The results showed that medical outliers were not reduced although 4-hour breeches were reduced. Therefore, the simulation model showed that even with the impact on hospital flow considered, the intervention still generated extra costs to the health service and provided no additional health benefits. Overall, the conclusion was that RAT would be a very expensive intervention for the health service to provide and it is unlikely to generate enough benefits to be considered a cost effective intervention.

Extended access to therapy in ED

The cohort model showed that the reduction in admissions from providing extended access to therapy in the ED would not fully compensate the cost of providing the service in the base case. In an ‘optimistic’ sensitivity analysis the additional admissions allowed the intervention to become cost saving although in a ‘conservative’ sensitivity analysis the net cost of providing the intervention became even higher. The simulation model indicated little impact on 4-hour breeches or medical outliers. In agreement with the cohort model, the simulation model also showed that the intervention would generate net costs to the health service.

Extended hours for consultants in AMU

The cohort model showed that the reduction in length of stay and ICU admissions did not provide enough cost savings to allow the intervention to provide a net saving to the health service. The intervention did provide health benefits in the form of mortality reduction in the AMU, however, these additional health benefits were not deemed cost effective in the base case with an ICER of £46k per QALY. Using optimistic estimates for the treatment effects the ICER decreased to £25k per QALY however, the intervention was dominated when more conservative treatment effects were applied. Although the cohort model found extended consultant hours in the AMU to not be cost effective in the base case, the additional health outcomes associated with improvements in hospital flow may provide enough additional benefits to allow the intervention to be cost effective. Unfortunately, the number of runs required was such that it was not possible to assess extended hours in AMU – there was too much noise to distinguish the signal. Therefore a definitive conclusion cannot be reached concerning its cost effectiveness.

Daily consultant review

The cohort model showed that the reduction in length of stay and ICU admissions did not provide enough cost savings to allow the intervention to provide a net saving to the health service. The intervention did provide health benefits in the form of mortality reduction seen in the GMW, however these additional health benefits were not deemed cost effective in the base case with an ICER of £48k per QALY. Using optimistic estimates for the treatment effects, the ICER decreased to £20k per QALY however, the intervention was dominated when conservative treatment effects were applied. The simulation model showed that medical outliers were reduced by a 450 per year for the hospital. However, the QALY gain was small and it cost a considerable £107k per QALY gained.

Overall, there is considerable uncertainty concerning the cost effectiveness of daily consultant reviews. Given the substantial cost of providing this intervention there would need to be considerable health benefits and/or cost savings to justify its implementation.

Therapy on medical wards

The cohort model showed that the reduction in length of stay provided enough cost savings to allow the intervention to provide a net saving to the health service of £88k per 1000 patients. The intervention also provided health benefits in the form of quality of life improvements for patients over 65 years of age with a CFS > 3 therefore making the intervention dominant. The treatment remained dominant even when conservative treatment effects were applied. The intervention would have to have significant negative impacts on hospital flow for the cost effectiveness of the intervention to be reversed. Therefore, from the cohort model alone it was considered highly likely that extended therapy access on the wards would be a cost effective and likely cost saving use of resources. The simulation model confirmed this result by also showing large cost savings along with QALY increases. Under all tested scenarios extended access to therapy remained cost effective across both models showing that the likelihood of it being a cost effective (and most likely a cost saving) intervention is very high.

Conclusions for all interventions

Overall RAT was the least likely to be cost effective and extended access to therapy on the wards was the most likely to be cost effective. There was considerable uncertainty concerning the cost effectiveness of all other strategies.

Consideration was given to how these interventions would interact with each other should they hypothetically all be provided at the same time. The 2 interventions in the ED would likely change the case-mix of individuals being admitted to AMU but would be unlikely to have an impact on GMW case mix as avoided admissions would be of low severity. Therefore, the cost effectiveness of interventions on the GMW would likely be independent of the 2 interventions assessed in the ED. The case mix of patients being admitted to the AMU may get worse, with the introduction of the ED interventions but the net impact on the cost effectiveness of extended consultant hours is not obvious. The ability of the consultant to discharge patients early would be reduced but the health outcomes might increase, since the consultant will be able to focus their attention on the more acutely ill patients.

The 2 interventions that would likely have the most impact on each other would be extended access to therapy on wards and daily consultant reviews. However, it is not clear how they would interact. On the one hand, it seems too optimistic to assume that the length of stay reductions from daily consultant review and extended access to therapy to be additive. However, the 2 interventions could be complementary –it is only possible to discharge a patient if they are signed off by both the therapist and the consultant. This should be a consideration when deciding to implement either service.

41.6.3.1. Treatment effects

The source of the treatment effects in the model were the expert opinion of the health economics subgroup of the committee. These opinions were informed by the guideline’s systematic review but also by the experience of the individuals and extensive discussion.

Although, the effects and their sizes were initially elicited through a formal consensus process, the subgroup did revise the estimates after extensive discussion, making the effect sizes more modest in each case. There was a deliberate attempt to make the analyses conservative by moderating the effect size (for example, RR=0.99), by targeting the effects on specific patient groups (for example, patients age>65 and CFS>2) and specific parts of the pathway (for example, AMU mortality). Conversely, we tried not to under-estimate intervention costs – these were applied to broad groups of patients and staff time were assumed to be incremental (there is an opportunity cost of the staff time required).

It was believed that the starting point of a hospital, could affect not just the baseline risks and case-mix but also the effect sizes themselves. For example, a hospital/ward that is operating effectively and efficiently with highly trained staff and access to critical care outreach might see much less benefit of daily consultant review than a hospital/ward that is less well-resourced or less well organised.

Analyses were conducted with more optimistic and more conservative effect sizes. In the case of extended therapy on medical wards, it remained cost saving but the other interventions were more sensitive to the magnitude of the treatment effects assumed.

The treatment effects incorporated in the model were those that the committee felt able to quantify. It was believed that these interventions could have other consequences that are not quantifiable. For example, the committee felt that, early consultant assessment in the ED is likely to lead to better quality/location of death for some patients, which are not captured in the model. There might also be reduced testing and fewer adverse events that are not captured.

Critical care outreach teams (CCOT) had been prioritised for modelling but the group decided that they could not estimate key consequences. For example, it was felt that one advantage of CCOT is that it relieves ward nurses and doctors of work but without a time and motions study it was unclear by how much. The only information obtained from the systematic review concerned the impact on cardiac arrests and in-hospital mortality. The committee felt that information on mortality could be misleading as in some instances the use of critical care outreach may be to improve the quality of death, an outcome which could not be captured using the QALY metric.

Overall, we have assessed the analyses as being directly applicable but with potentially serious limitations because the reporting of new trials or other evidence in this area could change the conclusions considerably.

41.6.3.2. Case-mix and baseline data

Since we were interested in the outcome of all non-elective medical patients being seen at an acute hospital, we chose to characterise patients by age, NEWS and CFS rather than diagnosis. In order to have data on patient movements and outcomes in relation to these characteristics, we had to do quite detailed analysis of data from a single large DGH. Had time allowed, we would have liked to repeat this analysis on data from at least one other Trust. Even in this case, we did not have CFS data from the same source as the other data and therefore we had to extrapolate using data from a national audit. In addition, we did not have data for patients in ED to the same level of detail as those admitted (for example, NEWS).

The case-mix of patients from the source hospital were similar to the national average but were slightly more severe. However, changing the case-mix of the population is something that could be dealt with by sensitivity analysis in the future.

We did not explicitly accounted for a weekend admission effect in the model but had we done so the effect might have been to increase the QALY gains from extended consultant hours in AMU and daily consultant review, due to increasing the baseline mortality and absolute reduction in mortality.

The short to long-term survival and quality of life of people who have had an acute medical problem or emergency was done using national data and epidemiological studies. However, this was fraught with difficulties because national statistics and epidemiological are usually either focused on specific diseases or else on the whole population so rarely can people having a specifically medical emergency be identified and followed up. For long-term survival, we found ourselves having to apply standardised mortality ratios to English national mortality data. We think that there is important research that could be done in terms of both:

• analysing the survival of AME patients, and
• cross-mapping utility scores with frailty scores.

41.6.3.3. Costs

Since staff rotas are complex and vary between hospitals, we did not attempt to model the staff numbers required but instead estimated contact time per patient and costed that time. This assumes that the time involved with the intervention would otherwise have been spent in a productive way.

With regard to the unit cost of staff time, we have based them on contracts in place at the time of analysis but we note that these will change as the move towards a 7-day NHS proceeds.

The majority of the intervention costs are either consultant time or therapist time. Implementation of these interventions will require such staff to be moved from other activity (such as outpatient work) or it would mean training of more staff. Therefore, there might be implications for Health Education England.

We have costed (occupied) bed days with a daily cost. We have costed medical outlying bed days the same as those on medical wards on the basis that there is an opportunity cost of a bed per se. This might not capture the cost of cancelled surgery neither from an NHS perspective nor from the perspective of Trust reimbursement. We have not attached a cost to an unoccupied bed day – although in the model these are relatively few in number, with GMW in particular operating at a very high occupancy level.

41.6.3.4. Simulation model

A patient-level simulation model allows interactions of complex systems, such as hospital pathways, to be explored in more depth than a cohort model. The simulation model simulates individual patients, their characteristics, outcomes and movements within the pathway. The individual patient outcomes can then be aggregated and averaged for results. Simulation models offer advantages over cohort models when⁹⁴:

• There is heterogeneity in the baseline characteristics of the eligible population and particularly where there is a non-linear relationship between characteristics and outcomes (for example, QALYs at the mean age might not equal the mean QALYs).
• Disease progression is a continuous process.
• Event rates vary by time.
• Prior events affect subsequent event rates.
• We want to explore the impact of an intervention in the context of fixed resources and queueing.

The interventions explored by our model specifically deal with timing of actions, such as timing and availability of staff interaction. Using a simulation model allows us to target interventions on specific patients and investigate the direct and indirect effects on the entire hospital pathway. A key characteristic of the simulation model is the dynamic use of resources, in this case hospital beds. The simulation model allows beds to be used throughout the pathway picked up and dropped by patients when needed. Having beds within the simulation model creates a flow to the hospital pathway that can be impacted upon positively or negatively by changes to the model, replicating a working hospital with the same pressures on capacity and solutions to accommodating patients. This adds to the cohort model as it allows saved bed days from interventions to be reallocated to other patients. An important outcome of the model, tied in with beds, is medical outliers. Medical outliers were generated as an outcome of the simulation model, resulting from blockages to hospital flow.

Hospitals are complex and our aim was to start with a simple but realistic model. With more time and more data, this model could be extended in the following ways. These modifications are unlikely to affect substantially our estimates of cost effectiveness but they could make certain parameters like bed occupancy and number of 4 hour breeches more realistic:

• More detailed specification of locations and patients.
  • Currently the model uses large locations to represent multiple wards within the hospital pathway. By not having to allocate patients to sex-specific wards or specialty-specific wards, a higher bed occupancy level is achievable in the model than would be in reality.
  • The model also does not include elective and non-medical patients and therefore does not capture their interactions with the acute medical emergency pathway. Simulating elective and non-medical patients would allow estimation of whole hospital occupancy, costs and consequences resulting from interventions in the medical emergency pathway.
• More refined transitions between locations.
  • The model updates NEWS when patients move to a new location, daily changes in patients’ NEWS scores and corresponding risks, such as mortality and ICU admission, could be implemented to capture variation in condition during a ward length of stay. If we were evaluating interventions that are triggered by NEWS then this would allow a precise estimation of the timing of the intervention.
  • Currently, patients move between beds within the model with no time delay. It has been assumed that the delay is built into the sampled length of stay. However, when patients are having length of stay adjusted through decision rules, this allows patients to move between beds immediately. Time to change beds between patients and delays could be implemented within the model when being forced to move beds, such as to an outlier ward, to capture the service delay in moving between beds.
  • As well as delays to movements between beds, timings of transfers may not always be realistic. The model adjusts sampled length of stay for those being discharged to represent realistic discharge times from hospital. However, it does not do this for transfers between wards. This means that the time distribution of patient transfers between wards is not taken into account when sampling length of stay and not be representative of a real hospital. The result of this could mean a greater proportion of patient transfers occur outside of normal working hours in the model.
  • Systematic reviews of the interventions investigated in the simulation model did not find a significant difference in readmissions. Furthermore, baseline readmission rates by age and CFS are not easily available. Readmissions were therefore not included in the simulated hospital pathway, although data from readmitted patients were not excluded from the data analysis. With the right data, this could be easily incorporated.
• More resource constraints

  • Resource constraints are used throughout the model to capture hospital capacity and investigate occupancy. However, not all the preadmission areas of the simulated hospital had constraints. The ambulatory acute medical unit could hold constraints. The ED could also be separated into locations for majors, minors and resuscitation, to add more detail and realistically represent a working ED. An additional step in the preadmission area would be to include ambulance queues prior to entry into the hospital, including costs and consequences to the first point of care in the acute medical emergency pathway.
  • The model uses staff time to generate unit costs for interventions. However, the model does not simulate individual members of staff and does not take into account their interactions with patients and each other. Including staff as a resource constraint would add a greater level of detail to the model and might allow conclusions on staffing levels to be explored but would probably not be generalisable.
• More scenarios

  • The model so far has looked at isolated interventions being implemented in the pathway. Some of the interventions target similar cohorts of patients. There is scope to investigate multiple interventions being implemented alongside each other to understand how they would interact. Many other service interventions could be evaluated as long as the pathway of the patients affected can be quantified.

The data used in the model for patient flow was from a single source, a large district general hospital, and so it was internally consistent. The data was stratified by age and NEWS so that correlation between outcomes and pathways could be reasonably estimated but this might have been achieved with greater precision had patient-level data for the whole pathway been used but this would be more complex and time-consuming to analyse.

Probabilities were used to model transitions and then time until the transition takes place. An alternative method would have been to use daily rates, with these rates changing by day of admission. However, our method ensured that mortality and length of stay were kept independent. This was important to avoid double counting of treatment effects, otherwise an intervention that reduced length of stay would inadvertently reduce mortality, even if this were not the intention of the committee.

We have tried to model the hospital to simulate what would happen at times of full capacity. This involved specifying decision rules about who is made a medical outlier and activating these rules when a hospital location is at full capacity. The main principle followed that patients in the early part of their stay would not be prioritised to be an outlier nor would patients with a high NEWS score or those going to rehab or a care home. However, by sampling length of stay from distributions that do not account for how busy the hospital is, the model will only be partially successful at mimicking practice for a number of reasons:

• It will not account for staff working more quickly when under greater pressure.
• In the case of the ED, admitted patients stay longer in the ED at times of stress, as they wait for a bed but those who are not admitted take the same time as when the hospital is busy.
• The model assumes increased risks for those who are made medical outliers reflected in their mortality, length of stay and referrals to ICU/HDU. However, it conservatively does not estimate the negative impact of over-occupancy on the patients that remain on the medical wards.

The simulation model holds a large amount of variability. Due to time constraints, the number of runs was limited. This was a significant limitation. It was concluded that the number of runs required was such that it was not possible to assess extended hours in AMU. For the other interventions the cost per QALY gained was very imprecise.. In order to speed up runtime, we used the parallel processing feature of Simul8, such that 8 runs were run in parallel. A side effect of this is that the exact results of an individual run may differ on different machines despite using an identical random number seed but this should not affect the statistical validity of the results.

The simulation model results do not include any probabilistic sensitivity analyses, such as distributions attached to input parameters. However, as the simulation model has conducted a large number of runs with variability, this may not be a major limitation. It is difficult to put a distribution around the relative treatment effects as these were based on expert opinion.

The model controls for case-mix of patients presenting in the simulated hospital. It would be desirable but not feasible to control further such that the same individual patients die in different scenarios of the same run. Controlling case-mix has reduced ‘noise’ in the analysis substantially but still random variations in mortality by case-mix group seem to be drowning out the effect sizes of interest.

41.6.3.5. Interventions not evaluated

Our modelling has focused on interventions that take place in the hospital. This arose because there were a number of interventions where we had evidence of effectiveness from the guideline’s systematic review but no published evidence of cost effectiveness. There was also reason to believe that the cost of these interventions is substantial. For interventions taking place outside of the hospital, on the other hand, either there was already, published evidence of cost effectiveness (for example, hospital at home) or else there was a lack of evidence of effectiveness (for example, GP home visits). For intermediate care, there were 15 published economic evaluations that were supportive including one based on a discrete event simulation. However, we have planned an analysis using the simulation model looking at the effects of reducing delayed transfers of care, to inform research around social care provision.

The model could be developed to evaluate other interventions both inside and outside the hospital.

41.6.4.1. Intervention evidence reviews

41.6.4.2. Discrete event simulations of acute medical services

We searched for discrete event simulation models that have evaluated acute medical care at the service level (rather than disease-specific models). We found 25 models that evaluated services within a hospital for acutely ill patients.^{13,14,42,46,51,58,59,70,71,81,82,93,98,100,103,106,107,124,151,153,154,169,172,176,190} Of these, 9 modelled flow beyond the ED.^{13,46,59,71,81,82,100,124,153}

Only one study¹²⁴ estimated costs and none looked at mortality or other health outcomes. We reported the results of this model in Chapter 12 on the alternatives to hospital. Our model is unique in terms of estimating QALYs, utility or cost effectiveness.

There are more examples that have used discrete event simulation to evaluate service delivery interventions in terms of costs and health outcomes but these have all focused on specific disease populations, such as heart failure¹⁷⁸ or stroke.¹³⁰

Our model is probably unique in modelling age, NEWS and clinical frailty score as primary characteristics of patients.

C.7.1. Weekend versus weekday admission

Figure 10Weekend versus weekday admission for predicting hospital mortality in acute medical admissions

Figure 11Weekend versus weekday admission for predicting hospital mortality in acute exacerbations of chronic obstructive pulmonary disease admissions

Figure 12Weekend versus weekday admission for predicting hospital mortality in emergency inpatient admissions

Figure 13Weekend versus weekday admission for predicting hospital mortality in emergency admissions

Figure 14Weekend versus weekday admission for predicting hospital mortality in acute subarachnoid haemorrhage admissions

Figure 15Weekend versus weekday admission for predicting hospital mortality in all admissions

Figure 16Weekend versus weekday admission for predicting hospital mortality in acute upper gastrointestinal bleeding admissions

Figure 17Weekend versus weekday admission for predicting hospital mortality in emergency admissions

Figure 18Weekend versus weekday admission for predicting hospital mortality in emergency medical and elderly admissions

Figure 19Weekend versus weekday admission for predicting hospital mortality in stroke admissions

Figure 20Weekend versus weekday admission for predicting hospital mortality in emergency admissions

Figure 21Weekend versus weekday admission for predicting hospital mortality in emergency admissions

Figure 22Weekend (8am-7.59pm) versus weekday admission for predicting 30 day survival in stroke admissions

Figure 23Weekend (8pm-7.59am) versus weekday admission for predicting 30 day survival in stroke admissions

Figure 24Weekend versus weekday admission for predicting 30 day mortality in stroke admissions

Figure 25Weekend versus weekday admission for predicting 30 day mortality in emergency admissions through A&E

Figure 26Weekend versus weekday admission for predicting 30 day mortality in direct emergency admissions

Figure 27Weekend versus weekday admission for predicting 30 day mortality in all admissions

Figure 28Weekend versus weekday admission for predicting avoidable adverse events (rebleeding) in acute upper gastrointestinal bleeding admissions

Figure 29Weekend versus weekday admission for predicting avoidable adverse events (surgery/radiology) in acute upper gastrointestinal bleeding admissions

Figure 30Weekend versus weekday admission for predicting avoidable adverse events (red cell transfusion) in acute upper gastrointestinal bleeding admissions

Figure 31Weekend versus weekday admission for predicting avoidable adverse events (inadequate clinical response to NEWS) in all admissions

Figure 32Weekend versus weekday admission for predicting avoidable adverse events (aspiration pneumonia) in stroke admissions

Figure 33Weekend versus weekday admission for predicting length of stay (discharge to usual place of residence within 56 days) in stroke admissions

C.7.2. Out of hours versus in hours admission

Figure 34Out of hours versus in hours admission for predicting hospital mortality in STEMI admissions

Figure 35Out of hours versus in hours admission for predicting 30 day mortality in stroke admissions

Figure 36Out of hours versus in hours admission for predicting 30 day mortality in STEMI admissions

Figure 37Out of hours versus in hours admission for predicting 30 day mortality in STEMI admissions

Figure 38Out of hours versus in hours admission for predicting 30 day mortality in all patients undergoing PPCI

Figure 39Out of hours versus in hours admission for predicting avoidable adverse events (bleeding complications) in STEMI admissions

Figure 40Out of hours versus in hours admission for predicting avoidable adverse events (major adverse cardiac events) in STEMI admissions

Summary of included studies

Narrative findings

Outlier versus non-outlier for predicting length of stay (days) (all admitted patients): median day (IC) outlying 8 (4-15); non-outlying 7 (4-13).

Outlier versus non-outlier for predicting ED length of stay (hours) (all admitted patients): median hour (25%-75%) outlying 9 (6-14); non-outlying 10 (6-16).

Clinical

• Five studies comprising 82,635 people evaluated the clinical outcomes of medical outliers in adults and young people admitted to hospital with a suspected or confirmed AME. The evidence suggested that being an outlier increased risk of length of stay and adverse events. The evidence for mortality was inconsistent across 4 studies. Two studies suggesting a benefit from being an outlier in terms of mortality were either in a specific population (congestive heart failure and cardiac arrhythmia patients) which may not be generalisable or graded very low quality. The other 2 studies suggested being an outlier had an increase in mortality. These studies were more generalisable populations and graded moderate quality.

D.7.1. Outlier versus non-outlier (adjusted for all key confounders)

Figure 42Serious adverse events (emergency calls)

D.7.2. Outlier versus non-outlier

Figure 43Mortality (hospital mortality)

Figure 44Mortality (hospital mortality)

Figure 45Mortality (hospital mortality)

Figure 46Mortality (90 day)

Figure 47Length of stay

Figure 48Serious adverse events (infection)

Figure 49Serious adverse events (haemorrhage)

Figure 50Serious adverse events (transfer to ICU)

E.2.1. Conceptual model

The health economics subgroup of the committee discussed the requirements of a simulation of a hospital that could evaluate costs, QALYs and explore the variation of performance over time.

Generally, the analyses were designed on the basis that workload and case-mix (age and NEWS) is determined by season and day of the week and hour of the day. Case-mix determines mortality, movements and length of stay.

It was agreed that to achieve this, the following characteristics would be essential.

• Patient characteristics:

  • Age group

    –

    16-44, 45-64, 65-75, 75-85, 85+

  • NEWS group

    –

    0, 1-4, 5-6, 7+

    –

    Zero indicates normal healthy life signs. A score of 7+ indicates referral to critical care outreach.

  • Frailty scores would have been desirable but were not recorded.
• Hospital pre-admission locations:

  • Emergency Department (ED)
  • Ambulatory Acute Medical Unit – acute medicine experts provides outpatient care for AME patients during daytime
  • Clinical Decision Unit – short stay wards provided by emergency medicine experts. Although these are technically admissions, we have made a distinction, since they are part of the emergency pathway rather than medical pathway and patients were not recorded on VitalPAC, which computes NEWS.
• Hospital admission locations

  • Acute Medical Unit (AMU) – where undifferentiated AME patients are assessed and managed usually for up to 24 hours
  • General medical wards (GMW) – provide level 1 care to medical patients, includes specialist wards such as gastroenterology, care of the elderly.
  • Intensive care unit / high dependency unit (ICU/HDU) – the intensive medicine department providing level 2 and level 3 care
  • Specialist high care units (HCU) – level 2 care in the hyper-acute stroke unit, coronary care unit, respiratory high care unit and renal high care unit.
  • Rehab wards – long stay wards
  • Medical outliers – AME patients on non-medical (surgery, gynaecology, trauma) wards
  • Non-medical pathway – Patients that are admitted under a medical consultant but subsequently take a non-medical pathway
• Discharge locations:

  • Usual residence
  • Care home (new admission) – a source of delayed transfers of care
  • NHS service
  • Other
• Outcomes:

  • Mortality – 30-day mortality data was not available; in-hospital mortality should be treated cautiously. Reduced in-hospital mortality might be due to reduced length of stay and could be offset by more deaths in the community. However, generally, death at home is considered preferable to patients and family members.
  • Length of stay (LOS) – excessive length of stay impedes flow and represents a cost to the NHS
  • ICU/HDU referral – we consider this an indicator of adverse events, other adverse events are captured by mortality and length of stay
  • Medical outlying – an indicator of suboptimal care
  • Queuing in ED – an indicator of the hospital being under stress and sub-optimal care.

E.2.2. Data

Data was extracted from the Queen Alexandra Hospital records and statistics computed by an experienced analyst from Portsmouth Hospitals Trust.

E.2.3. Analysis

For stays, mean, standard deviation and sample size were computed. For categorical outcomes, sample size and number of events were computed.

E.2.4. Validation

The guideline technical team checked that the numbers added up – for example, that the numbers leaving each destination were the same as the numbers entering.

The committee considered the face validity of the results in terms of their understanding of the pathway in their own hospitals. Generally, the results were considered generalisable. The one exception was the admission source, with Queen Alexandra having proportionately fewer patients coming from GPs and more patients coming from the ED and other NHS referrals than other hospitals.

E.3.1. Overview

Figure 51 and Figure 52 show the total activity analysed and the mean activity per day, respectively.

Figure 51Acute medical emergency activity 2010-2016

Figure 52Acute medical emergency activity per day

E.3.2. Pre-admission activity

The following statistics were extracted:

• ED attendances

  • By age group and whether admitted
• ED attendances

  • By time, quarter, day(Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday), admitted(y/n)
• ED attendances

  • By time & destination(CDU, Ward, AAMU, discharge)
• ED attendances

  • by week
• ED LOS mean SD and in 5 min intervals

  • By destination (CDU, Ward, AAMU, discharge)
• CDU discharges

  • by destination (Ward, AAMU, discharge)
• CDU LOS – mean, sd & n

  • By admitted(y/n)
• AAMU attendances

  • By hour, quarter, admitted(y/n)

The distribution of ED presentations can be seen by day of the week (Table 58) and hour of the day (Figure 53 and Figure 54). Presentations were highest on Sundays and Mondays, as were absolute numbers of admissions. But admission rates were lowest on these days.

Figure 53ED attendances by hour of the day

Figure 54Admission rate by hour of day

People presenting to ED were broken down by age group (Table 59). As expected, admission rates increased considerably with age from 17% in the lowest age group to 68% in the highest.

Patients spent an average 2.6 hours in the ED but this was close to the 4 hour target for those who were subsequently admitted (Table 60).

E.3.3. Admission activity

The following statistics were extracted:

• Admissions

  • By method of admission (ED, GP, outpatient, other)
• GMW^a stays where GMW was the first location

  • Next location, age group, NEWS group at beginning of GMW stay, NEWS at discharge from GMW.
• GMW^a stays where GMW was not the first location

  • Next location, age group, NEWS group at beginning of GMW stay, NEWS at discharge from GMW.
• Discharges

  • By destination & hour
• Mortality

  • by age group, NEWS group at admission, ITU stay (ICU/HDU, No but HCU, no), Medical outlier (yes at some point, no)
• LOS –mean, sd and n

  • by age group, NEWS group at admission, ITU stay (ICU/HDU, No but HCU, no)
• LOS –mean, sd and n

  • By current location, location, next location age group, news group at admission

Table 62 shows the admissions by first location and NEWS score at admission. Most patients admitted via the AMU but significant numbers go direct to GMW or HCU wards. 29.7% of patients had a NEWS score of zero (normal) at admission. NEWS was not recorded within the first 24 hours in 3.9% of patients - Table 62. This included ICU/HDU where it is not routinely recorded. However, most of the omissions were on the general medical ward. There are a number of reasons for these omissions including:

• Very short stay might mean it does not get recorded
• Patients admitted overnight might get it recorded on paper only
• Wards being refurbished
• Random selection
• Terminally ill patients – this seems to be borne out in Table 66 by the high mortality for patients who did not have a NEWS score recorded (after excluding patients on the ICU/HDU).

The proportion of patients with a NEWS at admission greater than 4 was more than double in the highest age group that of the lowest age group (Table 63).

The following were associated with high mortality:

• Higher NEWS – Table 64
• No NEWS recorded in first 24 hours - Table 64 and Table 66
• An admission to ICU - Table 65
• Older age - Table 63.