Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level...

106
Utilizzo dei modelli decisionali per la valutazione economica Gianluca Baio University College London Department of Statistical Science [email protected] http://www.ucl.ac.uk/statistics/research/statistics-health-economics/ http://www.statistica.it/gianluca https://github.com/giabaio IX Congresso Nazionale BIAS Parma Friday, 29 September 2017 Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 1 / 39

Transcript of Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level...

Page 1: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Utilizzo dei modelli decisionali per la valutazione economica

Gianluca Baio

University College LondonDepartment of Statistical Science

[email protected]://www.ucl.ac.uk/statistics/research/statistics-health-economics/

http://www.statistica.it/gianlucahttps://github.com/giabaio

IX Congresso Nazionale BIASParma

Friday, 29 September 2017

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 1 / 39

Page 2: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Outline

1. Health economic evaluation– What is it?– How does it work?

2. Statistical modelling– Individual-level vs aggregated data– The importance of being a Bayesian

3. Economic modelling & Decision analysis– Relevant quantities– Criteria for decision making

4. Uncertainty analysis– Rationale & main ideas– Value of information (hints)

5. Conclusions

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 2 / 39

Page 3: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Outline

1. Health economic evaluation– What is it?– How does it work?

2. Statistical modelling– Individual-level vs aggregated data– The importance of being a Bayesian

3. Economic modelling & Decision analysis– Relevant quantities– Criteria for decision making

4. Uncertainty analysis– Rationale & main ideas– Value of information (hints)

5. Conclusions

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 2 / 39

Page 4: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Outline

1. Health economic evaluation– What is it?– How does it work?

2. Statistical modelling– Individual-level vs aggregated data– The importance of being a Bayesian

3. Economic modelling & Decision analysis– Relevant quantities– Criteria for decision making

4. Uncertainty analysis– Rationale & main ideas– Value of information (hints)

5. Conclusions

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 2 / 39

Page 5: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Outline

1. Health economic evaluation– What is it?– How does it work?

2. Statistical modelling– Individual-level vs aggregated data– The importance of being a Bayesian

3. Economic modelling & Decision analysis– Relevant quantities– Criteria for decision making

4. Uncertainty analysis– Rationale & main ideas– Value of information (hints)

5. Conclusions

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 2 / 39

Page 6: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Outline

1. Health economic evaluation– What is it?– How does it work?

2. Statistical modelling– Individual-level vs aggregated data– The importance of being a Bayesian

3. Economic modelling & Decision analysis– Relevant quantities– Criteria for decision making

4. Uncertainty analysis– Rationale & main ideas– Value of information (hints)

5. Conclusions

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 2 / 39

Page 7: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Health technology assessment (HTA)

Objective: Combine costs & benefits of a given intervention into a rational scheme forallocating resources

Statisticalmodel

Economicmodel

Decisionanalysis

Uncertaintyanalysis

• Estimates relevant populationparameters θ

• Varies with the type ofavailable data (& statisticalapproach!)

• Combines the parameters to obtaina population average measure forcosts and clinical benefits

• Varies with the type of availabledata & statistical model used

• Summarises the economic modelby computing suitable measures of“cost-effectiveness”

• Dictates the best course ofactions, given current evidence

• Standardised process

• Assesses the impact of uncertainty (eg inparameters or model structure) on theeconomic results

• Mandatory in many jurisdictions (includingNICE, in the UK)

• Fundamentally Bayesian!

∆e = fe(θ)

∆c = fc(θ)

. . .

ICER = g(∆e,∆c)

EIB = h(∆e,∆c; k)

. . .

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 3 / 39

Page 8: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Health technology assessment (HTA)

Objective: Combine costs & benefits of a given intervention into a rational scheme forallocating resources

Statisticalmodel

Economicmodel

Decisionanalysis

Uncertaintyanalysis

• Estimates relevant populationparameters θ

• Varies with the type ofavailable data (& statisticalapproach!)

• Combines the parameters to obtaina population average measure forcosts and clinical benefits

• Varies with the type of availabledata & statistical model used

• Summarises the economic modelby computing suitable measures of“cost-effectiveness”

• Dictates the best course ofactions, given current evidence

• Standardised process

• Assesses the impact of uncertainty (eg inparameters or model structure) on theeconomic results

• Mandatory in many jurisdictions (includingNICE, in the UK)

• Fundamentally Bayesian!

∆e = fe(θ)

∆c = fc(θ)

. . .

ICER = g(∆e,∆c)

EIB = h(∆e,∆c; k)

. . .

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 3 / 39

Page 9: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Health technology assessment (HTA)

Objective: Combine costs & benefits of a given intervention into a rational scheme forallocating resources

Statisticalmodel

Economicmodel

Decisionanalysis

Uncertaintyanalysis

• Estimates relevant populationparameters θ

• Varies with the type ofavailable data (& statisticalapproach!)

• Combines the parameters to obtaina population average measure forcosts and clinical benefits

• Varies with the type of availabledata & statistical model used

• Summarises the economic modelby computing suitable measures of“cost-effectiveness”

• Dictates the best course ofactions, given current evidence

• Standardised process

• Assesses the impact of uncertainty (eg inparameters or model structure) on theeconomic results

• Mandatory in many jurisdictions (includingNICE, in the UK)

• Fundamentally Bayesian!

∆e = fe(θ)

∆c = fc(θ)

. . .

ICER = g(∆e,∆c)

EIB = h(∆e,∆c; k)

. . .

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 3 / 39

Page 10: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Health technology assessment (HTA)

Objective: Combine costs & benefits of a given intervention into a rational scheme forallocating resources

Statisticalmodel

Economicmodel

Decisionanalysis

Uncertaintyanalysis

• Estimates relevant populationparameters θ

• Varies with the type ofavailable data (& statisticalapproach!)

• Combines the parameters to obtaina population average measure forcosts and clinical benefits

• Varies with the type of availabledata & statistical model used

• Summarises the economic modelby computing suitable measures of“cost-effectiveness”

• Dictates the best course ofactions, given current evidence

• Standardised process

• Assesses the impact of uncertainty (eg inparameters or model structure) on theeconomic results

• Mandatory in many jurisdictions (includingNICE, in the UK)

• Fundamentally Bayesian!

∆e = fe(θ)

∆c = fc(θ)

. . .

ICER = g(∆e,∆c)

EIB = h(∆e,∆c; k)

. . .

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 3 / 39

Page 11: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Health technology assessment (HTA)

Objective: Combine costs & benefits of a given intervention into a rational scheme forallocating resources

Statisticalmodel

Economicmodel

Decisionanalysis

Uncertaintyanalysis

• Estimates relevant populationparameters θ

• Varies with the type ofavailable data (& statisticalapproach!)

• Combines the parameters to obtaina population average measure forcosts and clinical benefits

• Varies with the type of availabledata & statistical model used

• Summarises the economic modelby computing suitable measures of“cost-effectiveness”

• Dictates the best course ofactions, given current evidence

• Standardised process

• Assesses the impact of uncertainty (eg inparameters or model structure) on theeconomic results

• Mandatory in many jurisdictions (includingNICE, in the UK)

• Fundamentally Bayesian!

∆e = fe(θ)

∆c = fc(θ)

. . .

ICER = g(∆e,∆c)

EIB = h(∆e,∆c; k)

. . .

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 3 / 39

Page 12: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

1. (“Standard”) Statistical modelling — Individual level data

• The available data usually look something like this:

Demographics HRQL data Resource use dataID Trt Sex Age . . . u0 u1 . . . uJ c0 c1 . . . cJ

1 1 M 23 . . . 0.32 0.66 . . . 0.44 103 241 . . . 802 1 M 21 . . . 0.12 0.16 . . . 0.38 1 204 1 808 . . . 8773 2 F 19 . . . 0.49 0.55 . . . 0.88 16 12 . . . 22. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

and the typical analysis is based on the following steps:

1 Compute individual QALYs and total costs as

ei =J∑j=1

(uij + uij−1)δj

2and ci =

J∑j=0

cij ,[

with: δj =Timej − Timej−1

Unit of time

]

2 (Often implicitly) assume normality and linearity and model independently individualQALYs and total costs by controlling for baseline values

ei = αe0 + αe1u0i + αe2Trti + εei [+ . . .], εei ∼ Normal(0, σe)

ci = αc0 + αc1c0i + αc2Trti + εci [+ . . .], εci ∼ Normal(0, σc)

3 Estimate population average cost and effectiveness differentials and use bootstrap toquantify uncertainty

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 4 / 39

Page 13: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

1. (“Standard”) Statistical modelling — Individual level data

• The available data usually look something like this:

Demographics HRQL data Resource use dataID Trt Sex Age . . . u0 u1 . . . uJ c0 c1 . . . cJ

1 1 M 23 . . . 0.32 0.66 . . . 0.44 103 241 . . . 802 1 M 21 . . . 0.12 0.16 . . . 0.38 1 204 1 808 . . . 8773 2 F 19 . . . 0.49 0.55 . . . 0.88 16 12 . . . 22. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

and the typical analysis is based on the following steps:

1 Compute individual QALYs and total costs as

ei =J∑j=1

(uij + uij−1)δj

2and ci =

J∑j=0

cij ,[

with: δj =Timej − Timej−1

Unit of time

]

Time (years)

Qu

alit

y o

f lif

e (

sca

le 0

-1)

0 2 4 6 8 10

0.0

0.2

0.4

0.6

0.8

1.0

δj

uij+uij−12

QALYi = “Area under the curve”

2 (Often implicitly) assume normality and linearity and model independently individualQALYs and total costs by controlling for baseline values

ei = αe0 + αe1u0i + αe2Trti + εei [+ . . .], εei ∼ Normal(0, σe)

ci = αc0 + αc1c0i + αc2Trti + εci [+ . . .], εci ∼ Normal(0, σc)

3 Estimate population average cost and effectiveness differentials and use bootstrap toquantify uncertainty

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 4 / 39

Page 14: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

1. (“Standard”) Statistical modelling — Individual level data

• The available data usually look something like this:

Demographics HRQL data Resource use dataID Trt Sex Age . . . u0 u1 . . . uJ c0 c1 . . . cJ

1 1 M 23 . . . 0.32 0.66 . . . 0.44 103 241 . . . 802 1 M 21 . . . 0.12 0.16 . . . 0.38 1 204 1 808 . . . 8773 2 F 19 . . . 0.49 0.55 . . . 0.88 16 12 . . . 22. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

and the typical analysis is based on the following steps:

1 Compute individual QALYs and total costs as

ei =J∑j=1

(uij + uij−1)δj

2and ci =

J∑j=0

cij ,[

with: δj =Timej − Timej−1

Unit of time

]

2 (Often implicitly) assume normality and linearity and model independently individualQALYs and total costs by controlling for baseline values

ei = αe0 + αe1u0i + αe2Trti + εei [+ . . .], εei ∼ Normal(0, σe)

ci = αc0 + αc1c0i + αc2Trti + εci [+ . . .], εci ∼ Normal(0, σc)

3 Estimate population average cost and effectiveness differentials and use bootstrap toquantify uncertainty

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 4 / 39

Page 15: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

1. (“Standard”) Statistical modelling — Individual level data

• The available data usually look something like this:

Demographics HRQL data Resource use dataID Trt Sex Age . . . u0 u1 . . . uJ c0 c1 . . . cJ

1 1 M 23 . . . 0.32 0.66 . . . 0.44 103 241 . . . 802 1 M 21 . . . 0.12 0.16 . . . 0.38 1 204 1 808 . . . 8773 2 F 19 . . . 0.49 0.55 . . . 0.88 16 12 . . . 22. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

and the typical analysis is based on the following steps:

1 Compute individual QALYs and total costs as

ei =J∑j=1

(uij + uij−1)δj

2and ci =

J∑j=0

cij ,[

with: δj =Timej − Timej−1

Unit of time

]

2 (Often implicitly) assume normality and linearity and model independently individualQALYs and total costs by controlling for baseline values

ei = αe0 + αe1u0i + αe2Trti + εei [+ . . .], εei ∼ Normal(0, σe)

ci = αc0 + αc1c0i + αc2Trti + εci [+ . . .], εci ∼ Normal(0, σc)

3 Estimate population average cost and effectiveness differentials and use bootstrap toquantify uncertainty

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 4 / 39

Page 16: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

1. (“Standard”) Statistical modelling — Aggregated data

1 Build a population level model (eg decision tree/Markov model)

NStandard/new

treatment

N − SEtNo side effects

(1− πt)

SEtBlood-related

side effects(πt)

AtAmbulatory care

(γ)

SEt −AtHospital admission

(1− γ)

et = 0, ct = cdrugt + camb

et = 0, ct = cdrugt + chosp

et = 1, ct = cdrugt

Economic outcomes

NB: in this case, the “data” are typically represented by summary statistics for theparameters of interest θ = (πt, γ, ...)

2 Use point estimates for the parameters to build the “base-case” (average) evaluation

3 Use resampling methods (eg bootstrap) to propage uncertainty in the pointestimates and perform uncertainty analysis

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 5 / 39

Page 17: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

1. (“Standard”) Statistical modelling — Aggregated data

1 Build a population level model (eg decision tree/Markov model)

NStandard/new

treatment

N − SEtNo side effects

(1− πt)

SEtBlood-related

side effects(πt)

AtAmbulatory care

(γ)

SEt −AtHospital admission

(1− γ)

et = 0, ct = cdrugt + camb

et = 0, ct = cdrugt + chosp

et = 1, ct = cdrugt

Economic outcomes

NB: in this case, the “data” are typically represented by summary statistics for theparameters of interest θ = (πt, γ, ...)

2 Use point estimates for the parameters to build the “base-case” (average) evaluation

3 Use resampling methods (eg bootstrap) to propage uncertainty in the pointestimates and perform uncertainty analysis

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 5 / 39

Page 18: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

1. (“Standard”) Statistical modelling — Aggregated data

1 Build a population level model (eg decision tree/Markov model)

NStandard/new

treatment

N − SEtNo side effects

(1− πt)

SEtBlood-related

side effects(πt)

AtAmbulatory care

(γ)

SEt −AtHospital admission

(1− γ)

et = 0, ct = cdrugt + camb

et = 0, ct = cdrugt + chosp

et = 1, ct = cdrugt

Economic outcomes

NB: in this case, the “data” are typically represented by summary statistics for theparameters of interest θ = (πt, γ, ...)

2 Use point estimates for the parameters to build the “base-case” (average) evaluation

3 Use resampling methods (eg bootstrap) to propage uncertainty in the pointestimates and perform uncertainty analysis

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 5 / 39

Page 19: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

“Standard” approach to HTA — “Two-stage”

Statisticalmodel

Economicmodel

Decisionanalysis

Uncertaintyanalysis

• Estimates relevant populationparameters θ

• Varies with the type ofavailable data (& statisticalapproach!)

• Combines the parameters to obtaina population average measure forcosts and clinical benefits

• Varies with the type of availabledata & statistical model used

• Summarises the economic model bycomputing suitable measures of“cost-effectiveness”

• Dictates the best course of actions,given current evidence

• Standardised process

• Assesses the impact of uncertainty (egin parameters or model structure) onthe economic results

• Mandatory in many jurisdictions(including NICE, in the UK)

• Fundamentally Bayesian!

1. Estimation (base-case)

θ

yp(y | θ)

θ̂ = f(Y )

“Two-stage approach” (Spiegelhalter, Abrams & Myles, 2004)

2. Probabilistic sensitivity analysis

θp(θ) ! g(θ̂)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 6 / 39

Page 20: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

“Standard” approach to HTA — “Two-stage”

Statisticalmodel

Economicmodel

Decisionanalysis

Uncertaintyanalysis

• Estimates relevant populationparameters θ

• Varies with the type ofavailable data (& statisticalapproach!)

• Combines the parameters to obtaina population average measure forcosts and clinical benefits

• Varies with the type of availabledata & statistical model used

• Summarises the economic model bycomputing suitable measures of“cost-effectiveness”

• Dictates the best course of actions,given current evidence

• Standardised process

• Assesses the impact of uncertainty (egin parameters or model structure) onthe economic results

• Mandatory in many jurisdictions(including NICE, in the UK)

• Fundamentally Bayesian!

1. Estimation (base-case)

θ

yp(y | θ)

θ̂ = f(Y )

“Two-stage approach” (Spiegelhalter, Abrams & Myles, 2004)

2. Probabilistic sensitivity analysis

θp(θ) ! g(θ̂)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 6 / 39

Page 21: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

2./3. Economic modelling+Decision analysis — base-case scenario

Cost-effectiveness plane

Effectiveness differential

Cos

t diff

eren

tial

∆e

∆c

∆e = E[e | θ̂1]︸ ︷︷ ︸µ̂e1

− E[e | θ̂0]︸ ︷︷ ︸µ̂e0

∆c = E[c | θ̂1]︸ ︷︷ ︸µ̂c1

− E[c | θ̂0]︸ ︷︷ ︸µ̂c0

ICER =E[∆c]

E[∆e]=µ̂c1 − µ̂c0µ̂e1 − µ̂e0

= Cost per outcome

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 7 / 39

Page 22: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

2./3./4. Economic modelling+Decision analysis+Uncertainty analysis

Cost-effectiveness plane

∆e

∆c

Effectiveness differential

Cos

t diff

eren

tial

∆e = E[e | θ1]︸ ︷︷ ︸µe1

− E[e | θ0]︸ ︷︷ ︸µe0

∆c = E[c | θ1]︸ ︷︷ ︸µc1

− E[c | θ0]︸ ︷︷ ︸µc0

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 7 / 39

Page 23: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

What’s wrong with this?...

• Potential correlation between costs & clinical benefits [Both ILD and ALD]

– Strong positive correlation — effective treatments are innovative and result fromintensive and lengthy research ⇒ are associated with higher unit costs

– Negative correlation — more effective treatments may reduce total care pathway costse.g. by reducing hospitalisations, side effects, etc.

– Because of the way in which standard models are set up, bootstrapping generally onlyapproximates the underlying level of correlation — MCMC does a better job!

• Joint/marginal normality not realistic [Mainly ILD]

– Costs usually skewed and benefits may be bounded in [0; 1]– Can use transformation (e.g. logs) — but care is needed when back transforming to

the natural scale– Should use more suitable models (e.g. Beta, Gamma or log-Normal) — generally

easier under a Bayesian framework

• Particularly as the focus is on decision-making (rather than just inference), we needto use all available evidence to fully characterise current uncertainty on the modelparameters and outcomes [Mainly ALD]

– A Bayesian approach is helpful in combining different sources of information– Propagating uncertainty is a fundamentally Bayesian operation!

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 8 / 39

Page 24: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

What’s wrong with this?...

• Potential correlation between costs & clinical benefits [Both ILD and ALD]

– Strong positive correlation — effective treatments are innovative and result fromintensive and lengthy research ⇒ are associated with higher unit costs

– Negative correlation — more effective treatments may reduce total care pathway costse.g. by reducing hospitalisations, side effects, etc.

– Because of the way in which standard models are set up, bootstrapping generally onlyapproximates the underlying level of correlation — MCMC does a better job!

• Joint/marginal normality not realistic [Mainly ILD]

– Costs usually skewed and benefits may be bounded in [0; 1]– Can use transformation (e.g. logs) — but care is needed when back transforming to

the natural scale– Should use more suitable models (e.g. Beta, Gamma or log-Normal) — generally

easier under a Bayesian framework

• Particularly as the focus is on decision-making (rather than just inference), we needto use all available evidence to fully characterise current uncertainty on the modelparameters and outcomes [Mainly ALD]

– A Bayesian approach is helpful in combining different sources of information– Propagating uncertainty is a fundamentally Bayesian operation!

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 8 / 39

Page 25: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

What’s wrong with this?...

• Potential correlation between costs & clinical benefits [Both ILD and ALD]

– Strong positive correlation — effective treatments are innovative and result fromintensive and lengthy research ⇒ are associated with higher unit costs

– Negative correlation — more effective treatments may reduce total care pathway costse.g. by reducing hospitalisations, side effects, etc.

– Because of the way in which standard models are set up, bootstrapping generally onlyapproximates the underlying level of correlation — MCMC does a better job!

• Joint/marginal normality not realistic [Mainly ILD]

– Costs usually skewed and benefits may be bounded in [0; 1]– Can use transformation (e.g. logs) — but care is needed when back transforming to

the natural scale– Should use more suitable models (e.g. Beta, Gamma or log-Normal) — generally

easier under a Bayesian framework

• Particularly as the focus is on decision-making (rather than just inference), we needto use all available evidence to fully characterise current uncertainty on the modelparameters and outcomes [Mainly ALD]

– A Bayesian approach is helpful in combining different sources of information– Propagating uncertainty is a fundamentally Bayesian operation!

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 8 / 39

Page 26: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Why does it matter?

Example: survival analysis in health economic evaluation

• Survival data are often the main outcome in clinical studies relevant for HTA– Cancer drugs (progression-free/overall survival time): ≈ 40% of NICE appraisals!– Need to extrapolate, for economic modelling purposes. BUT: Limited follow up from

trials, not consistent with time horizon of economic model

time

Sur

viva

l

as.factor(arm)=0as.factor(arm)=1

0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

1.0

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 9 / 39

Page 27: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Why does it matter? Median survival time

Example: survival analysis in health economic evaluation

• Survival data are often the main outcome in clinical studies relevant for HTA– Cancer drugs (progression-free/overall survival time): ≈ 40% of NICE appraisals!– Need to extrapolate, for economic modelling purposes. BUT: Limited follow up from

trials, not consistent with time horizon of economic model

time

Sur

viva

l

as.factor(arm)=0as.factor(arm)=1

0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

1.0 Kaplan Meier

Weibull

● ●8.33

11.54

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 9 / 39

Page 28: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Why does it matter? Mean survival time

Example: survival analysis in health economic evaluation

• Survival data are often the main outcome in clinical studies relevant for HTA– Cancer drugs (progression-free/overall survival time): ≈ 40% of NICE appraisals!– Need to extrapolate, for economic modelling purposes. BUT: Limited follow up from

trials, not consistent with time horizon of economic model

time

Sur

viva

l

as.factor(arm)=0as.factor(arm)=1

0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

1.0 Kaplan Meier

Weibull

9.0910.34

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 9 / 39

Page 29: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Why does it matter?

Example: survival analysis in health economic evaluation

• Survival data are often the main outcome in clinical studies relevant for HTA– Cancer drugs (progression-free/overall survival time): ≈ 40% of NICE appraisals!– Need to extrapolate, for economic modelling purposes. BUT: Limited follow up from

trials, not consistent with time horizon of economic model

• When there is strong correlation among the survival parameters, the results ofuncertainty analysis may be (strongly) biased under a more simplisticfrequentist model

– This matters most in health economics, because this bias carries over the economicmodelling, optimal decision making and assessment of the impact of parametricuncertainty!

• For more complex models, MLE-based estimates may fail to converge– This may be an issue for multi-parameter models, where limited data (not compounded

by relevant prior information) are not enough to fit all the model parameters– NB: you would normally need to fit more complex models for cases where the survival

curves are “strange” and so the usual parametric models fail to provide sufficient fit

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 9 / 39

Page 30: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Why does it matter?

Example: survival analysis in health economic evaluation

• Survival data are often the main outcome in clinical studies relevant for HTA– Cancer drugs (progression-free/overall survival time): ≈ 40% of NICE appraisals!– Need to extrapolate, for economic modelling purposes. BUT: Limited follow up from

trials, not consistent with time horizon of economic model

Model fit for the Generalised F model , obtained using Flexsurvreg(Maximum Likelihood Estimate). Running time: 1.157 seconds

mean se L95% U95%mu 2.29139696 0.0798508 2.13489e+00 2.44790e+00sigma 0.58729598 0.0725044 4.61076e-01 7.48069e-01Q 0.84874994 0.2506424 3.57500e-01 1.34000e+00P 0.00268265 0.0902210 6.33197e-32 1.13655e+26as.factor(arm)1 0.34645851 0.0877892 1.74395e-01 5.18522e-01

Model fit for the Generalised F model , obtained using Stan(Bayesian inference via Hamiltonian Monte Carlo). Running time: 26.692 seconds

mean se L95% U95%mu 2.256760 0.3455163 0.0897086 0.0865904sigma 0.507861 0.0762112 0.3608566 0.6582047Q 0.700062 0.3358360 0.0786118 1.3880582P 1.131968 0.5837460 0.3908284 2.6342762as.factor(arm)1 0.345516 0.0865904 0.1745665 0.5176818

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 9 / 39

Page 31: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian modelling

Existing knowledge• Population registries

• Observational studies

• Small/pilot RCTs

• Expert optionsp(θ)

0.0 0.2 0.4 0.6 0.8 1.0

theta

Prior

Encode the assumption that a drug has a response rate between 20 and 60%

Current data

• Large(r) scale RCT

• Observational study

• Relevant summaries

p(y | θ)

Updated knowledge

p(θ | y)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 10 / 39

Page 32: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian modelling

Existing knowledge• Population registries

• Observational studies

• Small/pilot RCTs

• Expert optionsp(θ)

Current data

• Large(r) scale RCT

• Observational study

• Relevant summariesp(y | θ)

0.0 0.2 0.4 0.6 0.8 1.0

theta

LikelihoodPrior

Observe a study with 150 responders out of 200 patients given the drug

Updated knowledge

p(θ | y)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 10 / 39

Page 33: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian modelling

Existing knowledge• Population registries

• Observational studies

• Small/pilot RCTs

• Expert optionsp(θ)

Current data

• Large(r) scale RCT

• Observational study

• Relevant summariesp(y | θ)Updated knowledge

p(θ | y)

0.0 0.2 0.4 0.6 0.8 1.0

theta

Posterior

LikelihoodPrior

Update knowledge to describe revised “state of science”

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 10 / 39

Page 34: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian HTA in action — Decision analytic models (influenza)

Yes(p1)

Cost with NIs +cost influenza

Yes Influenza?

No(1 − p1)

Cost with NIs

Prophylactic NIs?

Yes(p0)

Cost influenza

No Influenza?

No(1 − p0)

Cost with no NIsµe1 = −lp1

µe0 = −lp0

µc1 =(cNI + cInf

)p1 + cNI (1− p1)

µc0 =(cNI + cInf

)p0 + cNI (1− p0)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 11 / 39

Page 35: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian HTA in action — Decision analytic models (influenza)

xh

βh mh

γh

µγ σ2γ

p0

Module 1: Influenza incidence

• H studies reporting number ofpatients who get influenza (xh) inthe sample (mh)

• βh = population probability ofinfluenza (from study h)

logit(βh) = γh ∼ Normal(µγ , σγ)

• µγ ∼ Normal(0, v) = pooledaverage probability of infection (onlogit scale!)

⇒ p0 =exp(µγ)

[1 + exp(µγ)]

p1

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 11 / 39

Page 36: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian HTA in action — Decision analytic models (influenza)

xh

βh mh

γh

µγ σ2γ

p0

r(0)s

π(0)s n(0)

s

r(1)s

π(1)s n(1)

s

αs δs

µδ σ2δ

ρ

Module 2: Prophylaxis effectiveness

• S studies reporting number of

infected patients r(t)s in a sample

made of n(t)s subjects

• π(t)s = study- and

treatment-specific chance ofcontracting influenza

logit(π(0)s

)= αs ∼ Normal(0, 10)

logit(π(1)s

)= αs + δs

δs ∼ Normal(µδ, σδ) =study-specific treatment effect

• µδ ∼ Normal(0, v) = pooledlog-odds ratio of influenza giventreatment ⇒ ρ = exp(µδ)

p1

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 11 / 39

Page 37: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian HTA in action — Decision analytic models (influenza)

xh

βh mh

γh

µγ σ2γ

p0

r(0)s

π(0)s n(0)

s

r(1)s

π(1)s n(1)

s

αs δs

µδ σ2δ

ρ

p1Can combine modules 1 and 2

p1 =ρp0/(1− p0)

1 + ρp0/(1− p0)

h = 1, . . . , H s = 1, . . . , S

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 11 / 39

Page 38: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian HTA in action — Decision analytic models (influenza)

Inference for Bugs model at "EvSynth.txt",Current: 2 chains , each with 10000 iterations (first 9500 discarded), n.thin = 20Cumulative: n.sims = 1000 iterations saved

mean sd 2.5% 25% 50% 75% 97.5% Rhat n.effp0 = p1 0.059 0.023 0.024 0.045 0.056 0.071 0.113 1.004 460p1 = p2 0.013 0.008 0.004 0.008 0.012 0.016 0.031 1.006 440

...µc0 = mu.c[1] 20.000 0.412 19.360 19.730 19.940 20.190 20.971 1.001 1000µc1 = mu.c[2] 19.226 0.142 19.070 19.140 19.200 19.280 19.550 1.011 1000µe0 = mu.e[1] -0.489 0.203 -1.004 -0.589 -0.461 -0.353 -0.178 1.003 510µe1 = mu.e[2] -0.111 0.070 -0.267 -0.134 -0.097 -0.068 -0.030 1.009 970

●●●

●●

●●

●●

●●

●●

● ●●

● ●

●●

●●

●●

●●

●●

●●

●●

● ●

● ●

● ●

●●

●●

●●

●●

●●

●●

●●

● ● ●

●●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

● ●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

● ●

●●

● ●

●●

● ●

●●

●●

●●

●●

● ●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●

● ●

● ●

●●

●●

●●

●●

● ●

●●

● ●

●●

● ●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

mu.e[1]

mu.

e[2]

−1.4 −1.2 −1.0 −0.8 −0.6 −0.4 −0.2 0.0

−1.

0−

0.8

−0.

6−

0.4

−0.

20.

0

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

● ●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

mu.c[1]

mu.

c[2]

19 20 21 22 23

19.0

19.5

20.0

20.5

21.0

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 12 / 39

Page 39: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian approach to HTA

Statisticalmodel

Economicmodel

Decisionanalysis

Uncertaintyanalysis

• Estimates relevant populationparameters θ

• Varies with the type ofavailable data (& statisticalapproach!)

• Combines the parameters to obtaina population average measure forcosts and clinical benefits

• Varies with the type of availabledata & statistical model used

• Summarises the economic model bycomputing suitable measures of“cost-effectiveness”

• Dictates the best course of actions,given current evidence

• Standardised process

• Assesses the impact of uncertainty (egin parameters or model structure) onthe economic results

• Mandatory in many jurisdictions(including NICE, in the UK)

• Fundamentally Bayesian!

Estimation & PSA (one stage)

θ

yp(y | θ)

p(θ) p(θ | y)

“Integrated approach” Spiegelhalter, Abrams & Myles (2004)Baio, Berardi & Heath (2017)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 13 / 39

Page 40: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian approach to HTA [p(θ | y) vs gi(θi)]

Parameters Model structure Decision analysis

0.0 0.2 0.4 0.6 0.8 1.0

0.0 0.5 1.0 1.5 2.0

0.0 0.2 0.4 0.6 0.8 1.0

0 2000 6000 10000

π0

ρ

γ

chosp

Old chemotherapy

A0Ambulatory care

(γ)99K camb

SE0Blood-relatedside effects

(π0)

H0Hospital admission

(1 − γ)99Kchosp

cdrug0 L99N

Standardtreatment

A0Ambulatory care

(γ)99K camb

N − SE0No side effects

(1 − π0)

H0Hospital admission

(1 − γ)99Kchosp

New chemotherapy

A1Ambulatory care

(γ)99K camb

SE1Blood-relatedside effects(π1 = π0ρ)

H1Hospital admission

(1 − γ)99Kchosp

cdrug1 L99NNew

treatment

A1Ambulatory care

(γ)99K camb

N − SE1No side effects

(1 − π1)

H1Hospital admission

(1 − γ)99Kchosp

Old chemotherapyBenefits Costs

743.1 656 644.6

New chemotherapyBenefits Costs

743.1 656 644.6

ICER =276 468.6

58.3

ICER = 6 497.1

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 14 / 39

Page 41: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian approach to HTA [p(θ | y) vs gi(θi)]

Parameters Model structure Decision analysis

0.0 0.2 0.4 0.6 0.8 1.0

0.0 0.5 1.0 1.5 2.0

0.0 0.2 0.4 0.6 0.8 1.0

0 2000 6000 10000

π0

ρ

γ

chosp

x

x

x

x

Old chemotherapy

A0Ambulatory care

(γ)99K camb

SE0Blood-relatedside effects

(π0)

H0Hospital admission

(1 − γ)99Kchosp

cdrug0 L99N

Standardtreatment

A0Ambulatory care

(γ)99K camb

N − SE0No side effects

(1 − π0)

H0Hospital admission

(1 − γ)99Kchosp

New chemotherapy

A1Ambulatory care

(γ)99K camb

SE1Blood-relatedside effects(π1 = π0ρ)

H1Hospital admission

(1 − γ)99Kchosp

cdrug1 L99NNew

treatment

A1Ambulatory care

(γ)99K camb

N − SE1No side effects

(1 − π1)

H1Hospital admission

(1 − γ)99Kchosp

Old chemotherapyBenefits Costs

741 670 382.1

743.1 656 644.6

New chemotherapyBenefits Costs

732 1 131 978

743.1 656 644.6

ICER =276 468.6

58.3

ICER = 6 497.1

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 14 / 39

Page 42: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian approach to HTA [p(θ | y) vs gi(θi)]

Parameters Model structure Decision analysis

0.0 0.2 0.4 0.6 0.8 1.0

0.0 0.5 1.0 1.5 2.0

0.0 0.2 0.4 0.6 0.8 1.0

0 2000 6000 10000

π0

ρ

γ

chosp

x

x

x

x

Old chemotherapy

A0Ambulatory care

(γ)99K camb

SE0Blood-relatedside effects

(π0)

H0Hospital admission

(1 − γ)99Kchosp

cdrug0 L99N

Standardtreatment

A0Ambulatory care

(γ)99K camb

N − SE0No side effects

(1 − π0)

H0Hospital admission

(1 − γ)99Kchosp

New chemotherapy

A1Ambulatory care

(γ)99K camb

SE1Blood-relatedside effects(π1 = π0ρ)

H1Hospital admission

(1 − γ)99Kchosp

cdrug1 L99NNew

treatment

A1Ambulatory care

(γ)99K camb

N − SE1No side effects

(1 − π1)

H1Hospital admission

(1 − γ)99Kchosp

Old chemotherapyBenefits Costs

741 670 382.1699 871 273.3

New chemotherapyBenefits Costs

732 1 131 978664 1 325 654

ICER =276 468.6

58.3

ICER = 6 497.1

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 14 / 39

Page 43: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Bayesian approach to HTA [p(θ | y) vs gi(θi)]

Parameters Model structure Decision analysis

0.0 0.2 0.4 0.6 0.8 1.0

0.0 0.5 1.0 1.5 2.0

0.0 0.2 0.4 0.6 0.8 1.0

0 2000 6000 10000

π0

ρ

γ

chosp

x

x

x

x

Old chemotherapy

A0Ambulatory care

(γ)99K camb

SE0Blood-relatedside effects

(π0)

H0Hospital admission

(1 − γ)99Kchosp

cdrug0 L99N

Standardtreatment

A0Ambulatory care

(γ)99K camb

N − SE0No side effects

(1 − π0)

H0Hospital admission

(1 − γ)99Kchosp

New chemotherapy

A1Ambulatory care

(γ)99K camb

SE1Blood-relatedside effects(π1 = π0ρ)

H1Hospital admission

(1 − γ)99Kchosp

cdrug1 L99NNew

treatment

A1Ambulatory care

(γ)99K camb

N − SE1No side effects

(1 − π1)

H1Hospital admission

(1 − γ)99Kchosp

Old chemotherapyBenefits Costs

741 670 382.1699 871 273.3. . . . . .726 425 822.2

716.2 790 381.2

New chemotherapyBenefits Costs

732 1 131 978664 1 325 654. . . . . .811 766 411.4

774.5 1 066 849.8

ICER =276 468.6

58.3

ICER = 6 497.1

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 14 / 39

Page 44: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

2. Economic modelling

Cost-effectiveness plane

Effectiveness differential

Cos

t diff

eren

tial

∆e

∆c

∆e = E[e | θ1]︸ ︷︷ ︸µe1

− E[e | θ0]︸ ︷︷ ︸µe0

∆c = E[c | θ1]︸ ︷︷ ︸µc1

− E[c | θ0]︸ ︷︷ ︸µc0

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 15 / 39

Page 45: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

3. Decision analysis

Cost-effectiveness plane∆e = E[e | θ1]︸ ︷︷ ︸

µe1

− E[e | θ0]︸ ︷︷ ︸µe0

∆c = E[c | θ1]︸ ︷︷ ︸µc1

− E[c | θ0]︸ ︷︷ ︸µc0

Effectiveness differential

Cos

t diff

eren

tial

∆e

∆c

ICER =E[∆c]

E[∆e]=

E[µc1]− E[µc0]

E[µe1]− E[µe0]

= Cost per outcome

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 15 / 39

Page 46: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

3./4. Decision analysis+Uncertainty analysis

Cost effectiveness plane

New Chemotherapy vs Old Chemotherapy

Effectiveness differential

Cost diffe

rential

−200 −100 0 100 200

−200000

0200000

600000

• ICER=6497.10

k = 1000

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 15 / 39

Page 47: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

3./4. Decision analysis+Uncertainty analysis

Cost effectiveness plane

New Chemotherapy vs Old Chemotherapy

Effectiveness differential

Cost diffe

rential

−200 −100 0 100 200

−200000

0200000

600000

• ICER=6497.10

k = 25000

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 15 / 39

Page 48: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis — Cost-effectiveness acceptability curve

0 10000 20000 30000 40000 50000

0.0

0.2

0.4

0.6

0.8

1.0

Cost EffectivenessAcceptability Curve

Willingness to pay

Pro

babi

lity

of c

ost e

ffect

iven

ess

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 16 / 39

Page 49: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 1. Multiparameter evidence synthesis

• Unusual for a policy question to be informed by a single study– Must use all available and relevant evidence

• Multiparameter evidence synthesis (often called Network Meta-Analysis)– Learning about more than one quantity from combination of direct and indirect

evidence

• Simplest example:– New treatment C: been trialled against old treatment B, but not to A– For health economic evaluation need to compare A/B/C together– ⇒ Learn about C/A effect from C/B and B/A trial data

• Also called mixed treatment comparisons– since can also “mix” direct and indirect data on same comparison

• Common in UK health technology assessment, but require some statistical skills

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 17 / 39

Page 50: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Smoking cessation trial data: network meta-analysis

Comparison A: No intervention B: Self-help C: Individual counselling D: Group counselling Baseline treatment

AB79/702 77/694 A18/671 21/535 A8/116 19/149 A

AC

75/731 363/714 A2/106 9/205 A58/549 237/1561 A0/33 9/48 A3/100 31/98 A1/31 26/95 A6/39 17/77 A64/642 107/761 A5/62 8/90 A20/234 34/237 A95/1107 143/1031 A15/187 36/504 A78/584 73/675 A69/1177 54/888 A

ACD 9/140 23/140 10/138 A

AD 0/20 9/20 A

BC 20/49 16/43 B

BCD 11/78 12/85 29/170 B

BD 7/66 32/127 B

CD12/76 20/74 C9/55 3/26 C

• 24 trials; outcome: successfully quit smoking by 6-12 months

• Network of comparisons involving 4 interventions

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 18 / 39

Page 51: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Network of comparisons: smoking cessation

A: Nointervention

D: Groupcounselling

B: Self-help

C: Individualcounselling

2 trials15 trials

3 trials

2 trials

4 trials

1 trial

N=12846

N=2867

N=318

N=255

N=441

N=764

• All comparisons haveat least one trial withdirect data

• We wish to enhancedirect with indirectevidence

• e.g. A-D comparison(2 direct trials)improved by includingA-C, C-D trials

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 19 / 39

Page 52: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Networks with missing direct comparisons

Treatment A

Treatment D

Treatment B

Treatment C

2 trials15 trials

3 trials

2 trials

?

?

• In other applications,might want to learnabout comparisonswith no direct trialevidence

• e.g. how much betterthan current treatmentC is new treatment D?

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 19 / 39

Page 53: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Results: Comparing direct and mixed evidence

Direct-only odds ratios (CIs) from classical analysis of pooled individual data

• Precision of D/A estimate improvedby indirect C/A, C/D data

• Strong direct data for othercomparisons, so not improved byindirect evidence

• C/B estimate from one direct study→ pulled towards much biggerindirect C/A, B/A data

– Evidence of heterogeneity —should consider “random effect”modelling. . .

Odds ratio

0 1 2 3 4 5

Pooled N

B: Self−help / A: None

C: Individual / A: None

D: Group / A: None

C: Individual / B: Self−help

D: Group / B: Self−help

D: Group / C: Individual

2867

12846

318

255

441

764

Fixed effects mixedFixed effects direct

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 20 / 39

Page 54: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

• Assess long-term cost-effectiveness based only on short-term data

• State-transition (usually Markov) models for clinical histories– Commonly implemented in Excel, or specialized software (e.g. TreeAge)

• Assume a set of S “clinically relevant” states– Exhaustive and mutually exclusive

• The structure (links among nodes) describes the dynamics of disease history– Arrows connecting two states encode the assumption that a transition from the one

where the arrow originates to the one reached by it is possible– Absence of an arrow between two states implies that the transition from one to the

other is not allowed by our model

• From one period to the next, subjects can move among the states according to therules specified by the arrows

• Movements occur according to suitable transition probabilities

πj = πj−1Λj

where– πj is the vector of probabilities for each state at time j– Λj = [Λj;s,s′ ] is a transition matrix describing the probability of moving from state s

to state s′ at time j

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 21 / 39

Page 55: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

• Assess long-term cost-effectiveness based only on short-term data

• State-transition (usually Markov) models for clinical histories– Commonly implemented in Excel, or specialized software (e.g. TreeAge)

• Assume a set of S “clinically relevant” states– Exhaustive and mutually exclusive

• The structure (links among nodes) describes the dynamics of disease history– Arrows connecting two states encode the assumption that a transition from the one

where the arrow originates to the one reached by it is possible– Absence of an arrow between two states implies that the transition from one to the

other is not allowed by our model

• From one period to the next, subjects can move among the states according to therules specified by the arrows

• Movements occur according to suitable transition probabilities

πj = πj−1Λj

where– πj is the vector of probabilities for each state at time j– Λj = [Λj;s,s′ ] is a transition matrix describing the probability of moving from state s

to state s′ at time j

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 21 / 39

Page 56: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

• Assess long-term cost-effectiveness based only on short-term data

• State-transition (usually Markov) models for clinical histories– Commonly implemented in Excel, or specialized software (e.g. TreeAge)

• Assume a set of S “clinically relevant” states– Exhaustive and mutually exclusive

• The structure (links among nodes) describes the dynamics of disease history– Arrows connecting two states encode the assumption that a transition from the one

where the arrow originates to the one reached by it is possible– Absence of an arrow between two states implies that the transition from one to the

other is not allowed by our model

• From one period to the next, subjects can move among the states according to therules specified by the arrows

• Movements occur according to suitable transition probabilities

πj = πj−1Λj

where– πj is the vector of probabilities for each state at time j– Λj = [Λj;s,s′ ] is a transition matrix describing the probability of moving from state s

to state s′ at time j

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 21 / 39

Page 57: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

1. Define a structure

Disease

In health Death

Recoveryxyyxxyyxxyyxxyyxxyyxyyxx

xyyxxyyxxyyxxyyxxyyxyyxx

xyyx

xy

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 22 / 39

Page 58: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

2. Estimate the transition probabilities

Disease

In health Death

Recovery

λ22

λ11

λ12 λ24

λ14λ44

λ23λ31 λ34

xyyxxyyxxyyxxyyxxyyxyyxx

xyyxxyyxxyyxxyyxxyyxyyxx

xyyx

xy

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 22 / 39

Page 59: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

3. Run the simulation: time j = 0

Disease

In health Death

Recovery

λ22

λ11

λ12 λ24

λ14λ44

λ23λ31 λ34

xyyxxyyxxyyxxyyxxyyxyyxx

xyyxxyyxxyyxxyyxxyyxyyxx

xyyx

xy

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 22 / 39

Page 60: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

3. Run the simulation: time j = 1

Disease

In health Death

Recovery

λ22

λ11

λ12 λ24

λ14λ44

λ23λ31 λ34

xyyxxyyxxyyxxyyxxyxyyxyx

xyyxxyyxxyyxxyyxxyyxyx

xyy

xy

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 22 / 39

Page 61: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

3. Run the simulation: time j = 2

Disease

In health Death

Recovery

λ22

λ11

λ12 λ24

λ14λ44

λ23λ31 λ34

xyyxxyyxxyyxxyyxxyxyyxyx

xyyxxyyxxyyxxyyxxyyxyx

xyxy

x

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 22 / 39

Page 62: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

3. Run the simulation: time j = 3

Disease

In health Death

Recovery

λ22

λ11

λ12 λ24

λ14λ44

λ23λ31 λ34

xyyxxyyxxyyxxyyxxyxyyxyx

xyyxxyyxxyyxxyyxxyyxyx

xyx

xx

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 22 / 39

Page 63: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

3. Run the simulation: time j = J

Disease

In health Death

Recovery

λ22

λ11

λ12 λ24

λ14λ44

λ23λ31 λ34

xyyxxyyxxyyxxyyxxyxyyxyx

xyyxxyyxxyyxxyyxxyyxyyxx

xyx

xx

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 22 / 39

Page 64: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

• MMs are very popular in modelling non-communicable diseases (eg cancer)

• Usually, they are run using (yet again...) a “hybrid” approach

1 Estimate relevant parameters (eg based on survival analysis), using proper statisticalsoftware (Stata, SAS, R,. . . )

2 Import/copy table of results in an Excel spreadsheet3 Use methods to approximate correlation across survival parameters (eg Cholesky

decomposition)4 Simulate a large number of survival curves (based on parameters simulations)5 Use the survival curves to determine the transition probabilities in the Markov model

• This is often a recipe for disaster!– Correlation among survival parameters crucial — need to properly account for that!– Mixing fdifferent software not always ideal– Programming Excel for MMs can be easy (for very simple models), but can become

highly inefficient!

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 23 / 39

Page 65: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

• MMs are very popular in modelling non-communicable diseases (eg cancer)

• Usually, they are run using (yet again...) a “hybrid” approach1 Estimate relevant parameters (eg based on survival analysis), using proper statistical

software (Stata, SAS, R,. . . )

2 Import/copy table of results in an Excel spreadsheet3 Use methods to approximate correlation across survival parameters (eg Cholesky

decomposition)4 Simulate a large number of survival curves (based on parameters simulations)5 Use the survival curves to determine the transition probabilities in the Markov model

• This is often a recipe for disaster!– Correlation among survival parameters crucial — need to properly account for that!– Mixing fdifferent software not always ideal– Programming Excel for MMs can be easy (for very simple models), but can become

highly inefficient!

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 23 / 39

Page 66: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

• MMs are very popular in modelling non-communicable diseases (eg cancer)

• Usually, they are run using (yet again...) a “hybrid” approach1 Estimate relevant parameters (eg based on survival analysis), using proper statistical

software (Stata, SAS, R,. . . )2 Import/copy table of results in an Excel spreadsheet

3 Use methods to approximate correlation across survival parameters (eg Choleskydecomposition)

4 Simulate a large number of survival curves (based on parameters simulations)5 Use the survival curves to determine the transition probabilities in the Markov model

• This is often a recipe for disaster!– Correlation among survival parameters crucial — need to properly account for that!– Mixing fdifferent software not always ideal– Programming Excel for MMs can be easy (for very simple models), but can become

highly inefficient!

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 23 / 39

Page 67: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

• MMs are very popular in modelling non-communicable diseases (eg cancer)

• Usually, they are run using (yet again...) a “hybrid” approach1 Estimate relevant parameters (eg based on survival analysis), using proper statistical

software (Stata, SAS, R,. . . )2 Import/copy table of results in an Excel spreadsheet3 Use methods to approximate correlation across survival parameters (eg Cholesky

decomposition)

4 Simulate a large number of survival curves (based on parameters simulations)5 Use the survival curves to determine the transition probabilities in the Markov model

• This is often a recipe for disaster!– Correlation among survival parameters crucial — need to properly account for that!– Mixing fdifferent software not always ideal– Programming Excel for MMs can be easy (for very simple models), but can become

highly inefficient!

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 23 / 39

Page 68: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

• MMs are very popular in modelling non-communicable diseases (eg cancer)

• Usually, they are run using (yet again...) a “hybrid” approach1 Estimate relevant parameters (eg based on survival analysis), using proper statistical

software (Stata, SAS, R,. . . )2 Import/copy table of results in an Excel spreadsheet3 Use methods to approximate correlation across survival parameters (eg Cholesky

decomposition)4 Simulate a large number of survival curves (based on parameters simulations)

5 Use the survival curves to determine the transition probabilities in the Markov model

• This is often a recipe for disaster!– Correlation among survival parameters crucial — need to properly account for that!– Mixing fdifferent software not always ideal– Programming Excel for MMs can be easy (for very simple models), but can become

highly inefficient!

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 23 / 39

Page 69: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

• MMs are very popular in modelling non-communicable diseases (eg cancer)

• Usually, they are run using (yet again...) a “hybrid” approach1 Estimate relevant parameters (eg based on survival analysis), using proper statistical

software (Stata, SAS, R,. . . )2 Import/copy table of results in an Excel spreadsheet3 Use methods to approximate correlation across survival parameters (eg Cholesky

decomposition)4 Simulate a large number of survival curves (based on parameters simulations)5 Use the survival curves to determine the transition probabilities in the Markov model

• This is often a recipe for disaster!– Correlation among survival parameters crucial — need to properly account for that!– Mixing fdifferent software not always ideal– Programming Excel for MMs can be easy (for very simple models), but can become

highly inefficient!

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 23 / 39

Page 70: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Advanced decision modelling 2. Markov models

• MMs are very popular in modelling non-communicable diseases (eg cancer)

• Usually, they are run using (yet again...) a “hybrid” approach1 Estimate relevant parameters (eg based on survival analysis), using proper statistical

software (Stata, SAS, R,. . . )2 Import/copy table of results in an Excel spreadsheet3 Use methods to approximate correlation across survival parameters (eg Cholesky

decomposition)4 Simulate a large number of survival curves (based on parameters simulations)5 Use the survival curves to determine the transition probabilities in the Markov model

• This is often a recipe for disaster!– Correlation among survival parameters crucial — need to properly account for that!– Mixing fdifferent software not always ideal– Programming Excel for MMs can be easy (for very simple models), but can become

highly inefficient!

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 23 / 39

Page 71: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Uncertainty analysis — beyond the CEAC

• Decision models usually contain a very large number of parameters — so it’simportant to fully assess the resulting uncertainty in the decision-making process

• BUT: the CEAC only deals with the probability of making the “right decision”

• But it does not account for the payoff/penalty associated with making the“wrong” one!

• Example 1: Intervention t = 1 is the most cost-effective, given current evidence– Pr(t = 1 is cost-effective) = 0.51– If we get it wrong: Increase in costs = £3

If we get it wrong: Decrease in effectiveness = 0.000001 QALYs– Large uncertainty/negligible consequences ⇒ can afford uncertainty

• Example 2: Intervention t = 1 is the most cost-effective, given current evidence– Pr(t = 1 is cost-effective) = 0.999– If we get it wrong: Increase in costs = £1 000 000 000

If we get it wrong: Decrease in effectiveness = 999999 QALYs– Tiny uncertainty/dire consequences ⇒ probably should think about it...

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 24 / 39

Page 72: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Uncertainty analysis — beyond the CEAC

• Decision models usually contain a very large number of parameters — so it’simportant to fully assess the resulting uncertainty in the decision-making process

• BUT: the CEAC only deals with the probability of making the “right decision”

• But it does not account for the payoff/penalty associated with making the“wrong” one!

• Example 1: Intervention t = 1 is the most cost-effective, given current evidence– Pr(t = 1 is cost-effective) = 0.51– If we get it wrong: Increase in costs = £3

If we get it wrong: Decrease in effectiveness = 0.000001 QALYs– Large uncertainty/negligible consequences ⇒ can afford uncertainty

• Example 2: Intervention t = 1 is the most cost-effective, given current evidence– Pr(t = 1 is cost-effective) = 0.999– If we get it wrong: Increase in costs = £1 000 000 000

If we get it wrong: Decrease in effectiveness = 999999 QALYs– Tiny uncertainty/dire consequences ⇒ probably should think about it...

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 24 / 39

Page 73: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Uncertainty analysis — beyond the CEAC

• Decision models usually contain a very large number of parameters — so it’simportant to fully assess the resulting uncertainty in the decision-making process

• BUT: the CEAC only deals with the probability of making the “right decision”

• But it does not account for the payoff/penalty associated with making the“wrong” one!

• Example 1: Intervention t = 1 is the most cost-effective, given current evidence– Pr(t = 1 is cost-effective) = 0.51– If we get it wrong: Increase in costs = £3

If we get it wrong: Decrease in effectiveness = 0.000001 QALYs– Large uncertainty/negligible consequences ⇒ can afford uncertainty

• Example 2: Intervention t = 1 is the most cost-effective, given current evidence– Pr(t = 1 is cost-effective) = 0.999– If we get it wrong: Increase in costs = £1 000 000 000

If we get it wrong: Decrease in effectiveness = 999999 QALYs– Tiny uncertainty/dire consequences ⇒ probably should think about it...

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 24 / 39

Page 74: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

The rationale for PSA / research prioritisation

do not gather

additional data

keep

t = 0

switch to

t = 1

y

y

temporarily keep

t = 0 & gather

additional data

E

z

z

switch to

t = 1

keep

t = 0

D

u(y)

u(y)

u(z)

u(z)

decisions

random events

sampling costs

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 25 / 39

Page 75: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

The rationale for PSA / research prioritisation

do not gather

additional data

keep

t = 0

switch to

t = 1

y

y

temporarily keep

t = 0 & gather

additional data

E

z

z

switch to

t = 1

keep

t = 0

D

u(y)

u(y)

u(z)

u(z)

decisions

random events

sampling costs

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 25 / 39

Page 76: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

The rationale for PSA / research prioritisation

do not gather

additional data

keep

t = 0

switch to

t = 1

y

y

temporarily keep

t = 0 & gather

additional data

E

z

z

switch to

t = 1

keep

t = 0

D

u(y)

u(y)

u(z)

u(z)

decisions

random events

sampling costs

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 25 / 39

Page 77: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Perfect Information

Parameters simulations Expected utility Maximum OpportunityIter/n π0 ρ . . . γ NB0(θ) NB1(θ) net benefit loss

1 0.365 0.076 . . . 0.162 19 214 751 19 647 706 19 647 706 0

2 0.421 0.024 . . . 0.134 17 165 526 17 163 407 17 165 526 2 119.3

3 0.125 0.017 . . . 0.149 18 710 928 16 458 433 18 710 928 2 252 495.5

4 0.117 0.073 . . . 0.120 16 991 321 18 497 648 18 497 648 0

5 0.481 0.008 . . . 0.191 19 772 898 18 662 329 19 772 898 1 110 569.3

6 0.163 0.127 . . . 0.004 17 106 136 18 983 331 18 983 331 0

. . . . . . . . . . . .

1000 0.354 0.067 . . . 0.117 18 043 921 16 470 805 18 043 921 1 573 116.0

Average 18 659 238 19 515 004 19 741 589 226 585

• Characterise uncertainty in the model parameters– In a full Bayesian setting, these are draws from the joint posterior distribution of θ– In a frequentist setting, these are typically Monte Carlo draws from a set of univariate

distributions that describe some level of uncertainty around MLEs (two-step/hybrid)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 26 / 39

Page 78: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Perfect Information

Parameters simulations Expected utility Maximum OpportunityIter/n π0 ρ . . . γ NB0(θ) NB1(θ) net benefit loss

1 0.365 0.076 . . . 0.162 19 214 751 19 647 706 19 647 706 0

2 0.421 0.024 . . . 0.134 17 165 526 17 163 407 17 165 526 2 119.3

3 0.125 0.017 . . . 0.149 18 710 928 16 458 433 18 710 928 2 252 495.5

4 0.117 0.073 . . . 0.120 16 991 321 18 497 648 18 497 648 0

5 0.481 0.008 . . . 0.191 19 772 898 18 662 329 19 772 898 1 110 569.3

6 0.163 0.127 . . . 0.004 17 106 136 18 983 331 18 983 331 0

. . . . . . . . . . . .

1000 0.354 0.067 . . . 0.117 18 043 921 16 470 805 18 043 921 1 573 116.0

Average 18 659 238 19 515 004 19 741 589 226 585

• Uncertainty in the parameters induces a distribution of decisions– Typically based on the net benefits: NBt(θ) = kµet − µct– In each parameters configuration can identify the optimal strategy

• Averaging over the uncertainty in θ provides the overall optimal decision, givencurrent uncertainty (= choose the intervention associated with highestexpected utility)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 26 / 39

Page 79: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Perfect Information

Parameters simulations Expected utility Maximum OpportunityIter/n π0 ρ . . . γ NB0(θ) NB1(θ) net benefit loss

1 0.365 0.076 . . . 0.162 19 214 751 19 647 706 19 647 706 0

2 0.421 0.024 . . . 0.134 17 165 526 17 163 407 17 165 526 2 119.3

3 0.125 0.017 . . . 0.149 18 710 928 16 458 433 18 710 928 2 252 495.5

4 0.117 0.073 . . . 0.120 16 991 321 18 497 648 18 497 648 0

5 0.481 0.008 . . . 0.191 19 772 898 18 662 329 19 772 898 1 110 569.3

6 0.163 0.127 . . . 0.004 17 106 136 18 983 331 18 983 331 0

. . . . . . . . . . . .

1000 0.354 0.067 . . . 0.117 18 043 921 16 470 805 18 043 921 1 573 116.0

Average 18 659 238 19 515 004 19 741 589 226 585

• Summarise uncertainty in the decision, eg via the Expected Value of “Perfect”Information (EVPI)

– Defined as the average Opportunity Loss– Can also be computed as the difference between the average maximum expected

utility under “perfect” information and the maximum expected utility overall —in formula:

EVPI = Eθ

[maxt

NBt(θ)

]−max

tEθ [NBt(θ)]

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 26 / 39

Page 80: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Partial Perfect Information

• θ = all the model parameters; can be split into two subsets– The “parameters of interest” φ, e.g. prevalence of a disease, HRQL measures, length

of stay in hospital, ...– The “remaining parameters” ψ, e.g. cost of treatment with other established

medications,

• We are interested in quantifying the value of gaining more information on φ, whileleaving the current level of uncertainty on ψ unchanged

• In formulæ:– First, consider the expected utility (EU) if we were able to learn φ but not ψ– If we knew φ perfectly, best decision = the maximum of this EU– Of course we cannot learn φ perfectly, so take the expected value– And compare this with the maximum expected utility overall– This is the EVPPI!

EVPPI = Eφ

[maxt

Eψ|φ [NBt(θ)]

]−max

tEθ [NBt(θ)]

• That’s the difficult part!

– Can do nested Monte Carlo, but takes forever to get accurate results– Recent methods based on Gaussian Process regression very efficient

Strong et al (2014)Heath et al (2016)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 27 / 39

Page 81: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Partial Perfect Information

• θ = all the model parameters; can be split into two subsets– The “parameters of interest” φ, e.g. prevalence of a disease, HRQL measures, length

of stay in hospital, ...– The “remaining parameters” ψ, e.g. cost of treatment with other established

medications,

• We are interested in quantifying the value of gaining more information on φ, whileleaving the current level of uncertainty on ψ unchanged

• In formulæ:– First, consider the expected utility (EU) if we were able to learn φ but not ψ

– If we knew φ perfectly, best decision = the maximum of this EU– Of course we cannot learn φ perfectly, so take the expected value– And compare this with the maximum expected utility overall– This is the EVPPI!

EVPPI = Eφ

[maxt

Eψ|φ [NBt(θ)]

]−max

tEθ [NBt(θ)]

• That’s the difficult part!

– Can do nested Monte Carlo, but takes forever to get accurate results– Recent methods based on Gaussian Process regression very efficient

Strong et al (2014)Heath et al (2016)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 27 / 39

Page 82: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Partial Perfect Information

• θ = all the model parameters; can be split into two subsets– The “parameters of interest” φ, e.g. prevalence of a disease, HRQL measures, length

of stay in hospital, ...– The “remaining parameters” ψ, e.g. cost of treatment with other established

medications,

• We are interested in quantifying the value of gaining more information on φ, whileleaving the current level of uncertainty on ψ unchanged

• In formulæ:– First, consider the expected utility (EU) if we were able to learn φ but not ψ

– If we knew φ perfectly, best decision = the maximum of this EU– Of course we cannot learn φ perfectly, so take the expected value– And compare this with the maximum expected utility overall– This is the EVPPI!

EVPPI = Eφ

[maxt

Eψ|φ [NBt(θ)]

]−max

tEθ [NBt(θ)]

• That’s the difficult part!

– Can do nested Monte Carlo, but takes forever to get accurate results– Recent methods based on Gaussian Process regression very efficient

Strong et al (2014)Heath et al (2016)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 27 / 39

Page 83: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Partial Perfect Information

• θ = all the model parameters; can be split into two subsets– The “parameters of interest” φ, e.g. prevalence of a disease, HRQL measures, length

of stay in hospital, ...– The “remaining parameters” ψ, e.g. cost of treatment with other established

medications,

• We are interested in quantifying the value of gaining more information on φ, whileleaving the current level of uncertainty on ψ unchanged

• In formulæ:– First, consider the expected utility (EU) if we were able to learn φ but not ψ– If we knew φ perfectly, best decision = the maximum of this EU

– Of course we cannot learn φ perfectly, so take the expected value– And compare this with the maximum expected utility overall– This is the EVPPI!

EVPPI = Eφ

[maxt

Eψ|φ [NBt(θ)]

]−max

tEθ [NBt(θ)]

• That’s the difficult part!

– Can do nested Monte Carlo, but takes forever to get accurate results– Recent methods based on Gaussian Process regression very efficient

Strong et al (2014)Heath et al (2016)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 27 / 39

Page 84: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Partial Perfect Information

• θ = all the model parameters; can be split into two subsets– The “parameters of interest” φ, e.g. prevalence of a disease, HRQL measures, length

of stay in hospital, ...– The “remaining parameters” ψ, e.g. cost of treatment with other established

medications,

• We are interested in quantifying the value of gaining more information on φ, whileleaving the current level of uncertainty on ψ unchanged

• In formulæ:– First, consider the expected utility (EU) if we were able to learn φ but not ψ– If we knew φ perfectly, best decision = the maximum of this EU– Of course we cannot learn φ perfectly, so take the expected value

– And compare this with the maximum expected utility overall– This is the EVPPI!

EVPPI = Eφ

[maxt

Eψ|φ [NBt(θ)]

]−max

tEθ [NBt(θ)]

• That’s the difficult part!

– Can do nested Monte Carlo, but takes forever to get accurate results– Recent methods based on Gaussian Process regression very efficient

Strong et al (2014)Heath et al (2016)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 27 / 39

Page 85: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Partial Perfect Information

• θ = all the model parameters; can be split into two subsets– The “parameters of interest” φ, e.g. prevalence of a disease, HRQL measures, length

of stay in hospital, ...– The “remaining parameters” ψ, e.g. cost of treatment with other established

medications,

• We are interested in quantifying the value of gaining more information on φ, whileleaving the current level of uncertainty on ψ unchanged

• In formulæ:– First, consider the expected utility (EU) if we were able to learn φ but not ψ– If we knew φ perfectly, best decision = the maximum of this EU– Of course we cannot learn φ perfectly, so take the expected value– And compare this with the maximum expected utility overall

– This is the EVPPI!

EVPPI = Eφ

[maxt

Eψ|φ [NBt(θ)]

]−max

tEθ [NBt(θ)]

• That’s the difficult part!

– Can do nested Monte Carlo, but takes forever to get accurate results– Recent methods based on Gaussian Process regression very efficient

Strong et al (2014)Heath et al (2016)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 27 / 39

Page 86: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Partial Perfect Information

• θ = all the model parameters; can be split into two subsets– The “parameters of interest” φ, e.g. prevalence of a disease, HRQL measures, length

of stay in hospital, ...– The “remaining parameters” ψ, e.g. cost of treatment with other established

medications,

• We are interested in quantifying the value of gaining more information on φ, whileleaving the current level of uncertainty on ψ unchanged

• In formulæ:– First, consider the expected utility (EU) if we were able to learn φ but not ψ– If we knew φ perfectly, best decision = the maximum of this EU– Of course we cannot learn φ perfectly, so take the expected value– And compare this with the maximum expected utility overall– This is the EVPPI!

EVPPI = Eφ

[maxt

Eψ|φ [NBt(θ)]

]−max

tEθ [NBt(θ)]

• That’s the difficult part!– Can do nested Monte Carlo, but takes forever to get accurate results– Recent methods based on Gaussian Process regression very efficient

Strong et al (2014)Heath et al (2016)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 27 / 39

Page 87: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Sample Information

EVSI = Eθ,d|θ

maxt

Eθ|d [NBt(θ)]︸ ︷︷ ︸Value of decision based on

sample information(for a given study design)

− max

tEθ [NBt∗(θ)]︸ ︷︷ ︸

Value of decision based oncurrent information

Prior predictivedistribution

(pre-posterior)

Posterior given data d

• Computationally complex– Requires specific knowledge of the model for (future/hypothetical) data collection– Again, recent methods have improved efficiency

• Can be used to drive design of new study (eg sample size calculations)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 28 / 39

Page 88: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Sample Information

0 500 1000 1500

0.0

0.5

1.0

1.5

Sample Size

Per

Per

son

EV

SI

0.0250.250.50.750.975EVPPI

5 10 15 20 250

2000

4000

6000

8000

1000

0

Probability of Cost−Effective Trial

Time Horizon

Inci

denc

e P

opul

atio

n

Prob=0

Prob=.5

Prob=1

https://github.com/giabaio/EVSI

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 29 / 39

Page 89: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

4. Uncertainty analysis Expected Value of Sample Information

20 40 60 80 100 120 140

−60

0−

400

−20

00

200

400

Economic Value of Study

Sample Size of XN

Eco

nom

ic v

alue

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 29 / 39

Page 90: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

General (Bayesian) process of HTA

NStandardtreatment

N − SEt

No side effects(1− πt)

SEt

Blood-relatedside effects

(πt)

At

Ambulatory care(γ)

SEt −At

Hospital admission(1− γ)

et = 0, ct = cdrugt + camb

et = 0, ct = cdrugt + chosp

et = 1, ct = cdrugt

Economic outcomes

# JAGS model (saved to ‘modelChemo.txt’)model {

pi[1] ∼ dbeta(a.pi,b.pi)pi[2] <- pi[1]*rhorho ∼ dnorm(m.rho,tau.rho)gamma ∼ dbeta(a.gamma,b.gamma)c.amb ∼ dlnorm(m.amb,tau.amb)c.hosp ∼ dlnorm(m.hosp,tau.hosp)for (t in 1:2) {

SE[t] ∼ dbin(pi[t],N)A[t] ∼ dbin(gamma,SE[t])H[t] <- SE[t] - A[t]

}}

# Creates the variables of cost & effectivenesse <- c <- matrix(NA,1000,2)e <- N - SEfor (t in 1:2) {

c[,t] <- c.drug[t]*(N-SE[,t]) +(c.amb+c.drug[t])*A[,t] +(c.hosp+c.drug[t])*H[,t]

}

# Calls JAGS in background to run the modellibrary(R2jags)data <- list("a.pi","b.pi","a.gamma","b.gamma",

"m.amb","tau.amb","m.hosp","tau.hosp","m.rho","tau.rho","N")

filein <- "modelChemo.txt"params <- c("pi","gamma","c.amb","c.hosp",

"rho","SE","A","H")inits <- function()

list(pi=c(runif(1),NA),gamma=runif(1),c.amb=rlnorm(1),c.hosp=rlnorm(1),rho=runif(1))

chemo <- jags(data,inits,params,n.iter=20000,model.file=filein,n.chains=2,n.burnin=9500,n.thin=42,DIC=FALSE)

print(chemo,digits=3,intervals=c(0.025, 0.975))attach.jags(chemo)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 30 / 39

Page 91: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

General (Bayesian) process of HTA

NStandardtreatment

N − SEt

No side effects(1− πt)

SEt

Blood-relatedside effects

(πt)

At

Ambulatory care(γ)

SEt −At

Hospital admission(1− γ)

et = 0, ct = cdrugt + camb

et = 0, ct = cdrugt + chosp

et = 1, ct = cdrugt

Economic outcomes

# JAGS model (saved to ‘modelChemo.txt’)model {

pi[1] ∼ dbeta(a.pi,b.pi)pi[2] <- pi[1]*rhorho ∼ dnorm(m.rho,tau.rho)gamma ∼ dbeta(a.gamma,b.gamma)c.amb ∼ dlnorm(m.amb,tau.amb)c.hosp ∼ dlnorm(m.hosp,tau.hosp)for (t in 1:2) {

SE[t] ∼ dbin(pi[t],N)A[t] ∼ dbin(gamma,SE[t])H[t] <- SE[t] - A[t]

}}

# Creates the variables of cost & effectivenesse <- c <- matrix(NA,1000,2)e <- N - SEfor (t in 1:2) {

c[,t] <- c.drug[t]*(N-SE[,t]) +(c.amb+c.drug[t])*A[,t] +(c.hosp+c.drug[t])*H[,t]

}

# Calls JAGS in background to run the modellibrary(R2jags)data <- list("a.pi","b.pi","a.gamma","b.gamma",

"m.amb","tau.amb","m.hosp","tau.hosp","m.rho","tau.rho","N")

filein <- "modelChemo.txt"params <- c("pi","gamma","c.amb","c.hosp",

"rho","SE","A","H")inits <- function()

list(pi=c(runif(1),NA),gamma=runif(1),c.amb=rlnorm(1),c.hosp=rlnorm(1),rho=runif(1))

chemo <- jags(data,inits,params,n.iter=20000,model.file=filein,n.chains=2,n.burnin=9500,n.thin=42,DIC=FALSE)

print(chemo,digits=3,intervals=c(0.025, 0.975))attach.jags(chemo)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 30 / 39

Page 92: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

General (Bayesian) process of HTA

NStandardtreatment

N − SEt

No side effects(1− πt)

SEt

Blood-relatedside effects

(πt)

At

Ambulatory care(γ)

SEt −At

Hospital admission(1− γ)

et = 0, ct = cdrugt + camb

et = 0, ct = cdrugt + chosp

et = 1, ct = cdrugt

Economic outcomes

# JAGS model (saved to ‘modelChemo.txt’)model {

pi[1] ∼ dbeta(a.pi,b.pi)pi[2] <- pi[1]*rhorho ∼ dnorm(m.rho,tau.rho)gamma ∼ dbeta(a.gamma,b.gamma)c.amb ∼ dlnorm(m.amb,tau.amb)c.hosp ∼ dlnorm(m.hosp,tau.hosp)for (t in 1:2) {

SE[t] ∼ dbin(pi[t],N)A[t] ∼ dbin(gamma,SE[t])H[t] <- SE[t] - A[t]

}}

# Creates the variables of cost & effectivenesse <- c <- matrix(NA,1000,2)e <- N - SEfor (t in 1:2) {

c[,t] <- c.drug[t]*(N-SE[,t]) +(c.amb+c.drug[t])*A[,t] +(c.hosp+c.drug[t])*H[,t]

}

# Calls JAGS in background to run the modellibrary(R2jags)data <- list("a.pi","b.pi","a.gamma","b.gamma",

"m.amb","tau.amb","m.hosp","tau.hosp","m.rho","tau.rho","N")

filein <- "modelChemo.txt"params <- c("pi","gamma","c.amb","c.hosp",

"rho","SE","A","H")inits <- function()

list(pi=c(runif(1),NA),gamma=runif(1),c.amb=rlnorm(1),c.hosp=rlnorm(1),rho=runif(1))

chemo <- jags(data,inits,params,n.iter=20000,model.file=filein,n.chains=2,n.burnin=9500,n.thin=42,DIC=FALSE)

print(chemo,digits=3,intervals=c(0.025, 0.975))attach.jags(chemo)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 30 / 39

Page 93: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

General (Bayesian) process of HTA

NStandardtreatment

N − SEt

No side effects(1− πt)

SEt

Blood-relatedside effects

(πt)

At

Ambulatory care(γ)

SEt −At

Hospital admission(1− γ)

et = 0, ct = cdrugt + camb

et = 0, ct = cdrugt + chosp

et = 1, ct = cdrugt

Economic outcomes

# JAGS model (saved to ‘modelChemo.txt’)model {

pi[1] ∼ dbeta(a.pi,b.pi)pi[2] <- pi[1]*rhorho ∼ dnorm(m.rho,tau.rho)gamma ∼ dbeta(a.gamma,b.gamma)c.amb ∼ dlnorm(m.amb,tau.amb)c.hosp ∼ dlnorm(m.hosp,tau.hosp)for (t in 1:2) {

SE[t] ∼ dbin(pi[t],N)A[t] ∼ dbin(gamma,SE[t])H[t] <- SE[t] - A[t]

}}

# Creates the variables of cost & effectivenesse <- c <- matrix(NA,1000,2)e <- N - SEfor (t in 1:2) {

c[,t] <- c.drug[t]*(N-SE[,t]) +(c.amb+c.drug[t])*A[,t] +(c.hosp+c.drug[t])*H[,t]

}

# Calls JAGS in background to run the modellibrary(R2jags)data <- list("a.pi","b.pi","a.gamma","b.gamma",

"m.amb","tau.amb","m.hosp","tau.hosp","m.rho","tau.rho","N")

filein <- "modelChemo.txt"params <- c("pi","gamma","c.amb","c.hosp",

"rho","SE","A","H")inits <- function()

list(pi=c(runif(1),NA),gamma=runif(1),c.amb=rlnorm(1),c.hosp=rlnorm(1),rho=runif(1))

chemo <- jags(data,inits,params,n.iter=20000,model.file=filein,n.chains=2,n.burnin=9500,n.thin=42,DIC=FALSE)

print(chemo,digits=3,intervals=c(0.025, 0.975))attach.jags(chemo)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 30 / 39

Page 94: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

BCEA: a package for Bayesian cost-effectiveness analysis

What is BCEA not?

• BCEA is not a package to automatically run a Bayesian analysis– It cannot build the health economic model for you– It does not prepare the data to be used in the model– It does not automatically run the MCMC simulations– It does not choose the prior distributions for you

So what is it, then?

• BCEA provides a set of specific functions to systematically post-process the outputof a (Bayesian) health economic model

• Uses R (http://cran.r-project.org/)– Very good at interacting with standard MCMC software• BUGS (www.mrc-bsu.cam.ac.uk/bugs/winbugs/contents.shtml)• JAGS (www.mcmc-jags.sourceforge.net/)• Stan (http://mc-stan.org/users/interfaces/rstan)

– Free and there is a very large community of contributors– Specifically designed for statistical analysis and has very good graphical capabilities

http://www.statistica.it/gianluca/BCEA

https://github.com/giabaio/BCEA

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 31 / 39

Page 95: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

BCEA: a package for Bayesian cost-effectiveness analysis

What is BCEA not?

• BCEA is not a package to automatically run a Bayesian analysis– It cannot build the health economic model for you– It does not prepare the data to be used in the model– It does not automatically run the MCMC simulations– It does not choose the prior distributions for you

So what is it, then?

• BCEA provides a set of specific functions to systematically post-process the outputof a (Bayesian) health economic model

• Uses R (http://cran.r-project.org/)– Very good at interacting with standard MCMC software• BUGS (www.mrc-bsu.cam.ac.uk/bugs/winbugs/contents.shtml)• JAGS (www.mcmc-jags.sourceforge.net/)• Stan (http://mc-stan.org/users/interfaces/rstan)

– Free and there is a very large community of contributors– Specifically designed for statistical analysis and has very good graphical capabilities

http://www.statistica.it/gianluca/BCEA

https://github.com/giabaio/BCEA

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 31 / 39

Page 96: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

The R package BCEA — Baio et al (2017)

bcea

plot

• ceplane.plot• contour• contour2• ceac.plot• evi.plot• eib.plot

• ib.plot

summary

evppiCreateInputsstruct.psa

plot

diag.evppi

multi.ce

mce.plot

ceaf.plot

ceef.plot

CEriskav plot

mixedAn

plot

summary

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 32 / 39

Page 97: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

SAVI — http://savi.shef.ac.uk/SAVI/

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 33 / 39

Page 98: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

BCEAweb — https://egon.stats.ucl.ac.uk/projects/BCEAweb/

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 34 / 39

Page 99: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Let’s go all the way?...

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 35 / 39

Page 100: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Let’s go all the way?...

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 35 / 39

Page 101: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Conclusions

• Bayesian modelling particularly effective in health economic evaluations

• Allows the incorporation of external, additional information to the current analysis– Previous studies– Elicitation of expert opinions

• In general, Bayesian models are more flexible and allow the inclusion of complexrelationships between variables and parameters

– This is particularly effective in decision-models, where information from differentsources may be combined into a single framework

– Useful in the case of individual-level data (eg from Phase III RCT)

• Using MCMC methods, it is possible to produce the results in terms of simulationsfrom the posterior distributions

– These can be used to build suitable variables of cost and benefit– Particularly effective for running “probabilistic sensitivity analysis”

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 36 / 39

Page 102: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Conclusions

• Bayesian modelling particularly effective in health economic evaluations

• Allows the incorporation of external, additional information to the current analysis– Previous studies– Elicitation of expert opinions

• In general, Bayesian models are more flexible and allow the inclusion of complexrelationships between variables and parameters

– This is particularly effective in decision-models, where information from differentsources may be combined into a single framework

– Useful in the case of individual-level data (eg from Phase III RCT)

• Using MCMC methods, it is possible to produce the results in terms of simulationsfrom the posterior distributions

– These can be used to build suitable variables of cost and benefit– Particularly effective for running “probabilistic sensitivity analysis”

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 36 / 39

Page 103: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Conclusions

• Bayesian modelling particularly effective in health economic evaluations

• Allows the incorporation of external, additional information to the current analysis– Previous studies– Elicitation of expert opinions

• In general, Bayesian models are more flexible and allow the inclusion of complexrelationships between variables and parameters

– This is particularly effective in decision-models, where information from differentsources may be combined into a single framework

– Useful in the case of individual-level data (eg from Phase III RCT)

• Using MCMC methods, it is possible to produce the results in terms of simulationsfrom the posterior distributions

– These can be used to build suitable variables of cost and benefit– Particularly effective for running “probabilistic sensitivity analysis”

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 36 / 39

Page 104: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Shameless self marketing

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 37 / 39

Page 105: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Some references

Baio, G. (2012).

Bayesian Methods in Health Economics.Chapman Hall, CRC Press, Boca Raton, FL.(https://www.crcpress.com/Bayesian-Methods-in-Health-Economics/Baio/p/book/9781439895559)

Baio, G., A. Berardi, and A. Heath (2017).

Bayesian Cost-Effectiveness Analysis with the R package BCEA.Springer, New York, NY. (http://www.springer.com/gp/book/9783319557168)

Briggs, A., M. Sculpher, and K. Claxton (2006).

Decision modelling for health economic evaluation.Oxford University Press, Oxford, UK.

Heath, A., I. Manolopoulou, and G. Baio (2016).

Estimating the expected value of partial perfect information in health economic evaluations using Integrated NestedLaplace Approximation.Statistics in Medicine 35(23), 4264-4280.

Spiegelhalter, D., K. Abrams, and J. Myles (2004).

Bayesian Approaches to Clinical Trials and Health-Care Evaluation.John Wiley and Sons, Chichester, UK. (http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471499757.html)

Strong, M., J. Oakley, and A. Brennan (2014).

Estimating Multiparameter Partial Expected Value of Perfect Information from a Probabilistic Sensitivity Analysis SampleA Nonparametric Regression Approach.Medical Decision Making 34(3), 311–326.

Welton, N., A. Sutton, N. Cooper, K. Abrams, and A. Ades (2012).

Evidence Synthesis for Decision Making in HealthcareJohn Wiley and Sons, Chichester, UK. (http://eu.wiley.com/WileyCDA/WileyTitle/productCd-047006109X.html)

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 38 / 39

Page 106: Utilizzo dei modelli decisionali per la valutazione economica · 2017-10-10 · {Individual-level vs aggregated data {The importance of being a Bayesian 3.Economic modelling & Decision

Thank you!

Gianluca Baio (UCL) Modelli decisionali in HTA – Part I Congresso BIAS, 29 Sep 2017 39 / 39