1 1 1

PharmD Julien Favresse

4ème année biologie clinique

1

Bayesian strategy in long-term IQC results management

NTHC 2018

(1) INTERNAL QUALITY CONTROL

(2) WESTGARD RULES

(3) CURRENT LIMITATION OF IQC

(4) ALTERNATIVES

(5) BAYESIAN APPROACH

(6) TAKE-HOME MESSAGES

Summary

Aim

• Detect analytical errors 1,2,3

• Avoid misdiagnosis 1,3

• Adopt a cost-effective strategy

±20€ /day /2 vials (aPTT, PT, Fg, AT and TT)

1. Internal Quality Control

1 Kitchen S et al. Quality in Laboratory Hemostasis and Thrombosis, 2nd Ed, Wiley-Blackwell, 2013, 283 pp

2 Westgard J. Basic QC Practice: training in statistical quality controls for medical laboratories. 4th Edition, 2016, 334 pp 3 Parvin C. JALM (2017) 2:5, 1-4

IQC material (recommEndations)

• Should be similar in properties to test samples commutability 1,2

• Should be stable for its intended period of use 2

• Limited lot-to-lot and vial-to vial variation 2

• At least 2 levels, every 8h or 1 level every 4h 2,3

• IQC levels should ideally be set according to clinical decision threshold 4

• After reagent addition, maintenance, calibration, … (before and after) 5

1. Internal Quality Control

1 Miller W. Clin Chem (2011) 57:1, 76-83 2 Kitchen S et al. Quality in Laboratory Hemostasis and Thrombosis, 2nd Ed, Wiley-Blackwell, 2013, 283 pp 3 CLSI H47-A2 4 Petersen PH. Eur J Clin Chem Biochem (1996) 34: 983-999 5 Jones G. Clin Biochem Rev (2015) 36:3, 87-95

Preparing a control chart

• Target (mean) and standard deviation (SD) provided by manufacturers should

be established locally (preliminary phase) 1,2

• Collect 20 measurements (>10 days) to assess the intrinsic variability 1,3

Target values and limits should be revised periodically 4

1. Internal Quality Control

1 Kitchen S et al. Quality in Laboratory Hemostasis and Thrombosis, 2nd Ed, Wiley-Blackwell, 2013, 283 pp 2 Shewhart WA (1931) Economic control of quality of manufactured product. New York, NY: Van Nostrand; 1931; Vol.. 3 Westgard J. Basic QC Practice: training in statistical quality controls for medical laboratories. 4th Edition, 2016, 334 pp 4 CLSI 24-A3

1. Internal Quality Control

1 MacFarland A. Int Jnl Lab Hem (2015) 37, 729-738

Westgard rules (1)

• Back in 1981:

– 12s/13s/22s/R4s/41s/10x 1

2. Westgard rules

1 Westgard S. ECAT Foundation, Quality in the Medical Laboratory, with a focus on blood coagulation, special issue (2016) 17-20

Westgard rules (2)

• The 12s rule is no more recommended (too strict) 1,2,3

• The 12s rule Pfr of 4.6%, 8.9% or 13.0% for 1, 2 or 3 QC, respectively 1,2,4

• However 55% of labs (n=682) are still using this rule 5

1 Parvin CA. Clin Chem (2012) 58:5, 925-929 2 Westgard S. ECAT Foundation, Quality in the Medical Laboratory, with a focus on blood coagulation, special issue (2016) 17-20 3 Westgard J. Basic QC Practice: training in statistical quality controls for medical laboratories. 4th Edition, 2016, 334 pp 4 MacFarland A. Int Jnl Lab Hem (2015) 37, 729-738 5 Westgard survey 2017 (https://www.westgard.com/great-global-qc-survey-results.htm)

2. Westgard rules

Westgard rules (3)

• Six sigma: allow the performance evaluation of laboratory tests on the Sigma scale 1

• Total analytic Error (TEa)

CLIA, RCPA, ECAT, …

Within-subject biological variation (<1.65 (0.50 CVi) + 0.250 (CVi²+CVg²)1/2) (>350 analytes) 2

• Imprecision: CV of the lab (SD/mean*100%)

• Bias: EQA survey, peer group means, target value of controls

Sigma-metric (σ)= (TEa% – Bias%)/CV%

2. Westgard rules

1 Gras J. Laboratory Quality Control and patient Safety, De Gruyter (2017) 94 pp 2 https://www.westgard.com/biodatabase1.htm

Westgard rules (3)

• Six sigma: allow the performance evaluation of laboratory tests on the Sigma scale 1

• Total analytic Error (TEa)

CLIA, RCPA, ECAT, …

Within-subject biological variation (<1.65 (0.50 CVi) + 0.250 (CVi²+CVg²)1/2) (>350 analytes) 2

• Imprecision: CV of the lab (formule sur le schéma)

• Bias: EQA survey, peer group means, target value of controls

Sigma-metric (σ)= (TEa% – Bias%)/CV%

2. Westgard rules

1 Gras J. Laboratory Quality Control and patient Safety, De Gruyter (2017) 94 pp 2 https://www.westgard.com/biodatabase1.htm

Westgard rules (3)

• A 6-sigma method requires only 1 rule (13s) excellent tests

• A 5-sigma method requires 3 rules (13s/22s/R4s) tests that are suited for purpose

• A 4-sigma method requires 4 rules (13s/22s/R4s/41s) tests that are suited for purpose

• A 3-sigma method requires 5 rules (13s/22s/R4s/41s/8x) poor performers

2. Westgard rules

1 Westgard S. ECAT Foundation, Quality in the Medical Laboratory, with a focus on blood coagulation, special issue (2016) 17-20

Westgard rules (3)

• Sigma-metric (σ) = (TEa% – Bias%)/CV%

• Examples CHU UCL Namur (STAR-Max²):

–aPTT (CK Prest): TEa = 5.3%, bias = 1.1% and CV = 0.7% Sigma-metric = 5.7 (3 rules)

–Fibrinogen (clauss): TEa = 20%, bias = 0.1% and CV = 4.0% Sigma-metric = 5 (3 rules)

–Protein C (chromogenic): TEa = 18.7%, bias = 2.3% and CV = 3.8% Sigma-metric = 4.3 (4 rules)

–PT (Readyplastin): TEa = 5.3%, bias = 2.2% and CV = 0.9% Sigma-metric = 3.4 (4 rules)

–Protein S (antigenic): TEa = 20.7%, bias = 3.8% and CV = 5.1% Sigma-metric = 3.3 (4 rules)

–Antithrombin (chromogenic): TEa = 8.3%, bias = 1.2% and CV = 2.5% Sigma-metric = 2.8 (5 rules)

2. Westgard rules

Sigma metric Category Proposed QC rule Proposed QC frequency

>6 Excellent tests 13.5s One QC per day

4-6 Tests that are suited for

purpose 12.5s Two levels of QC per day

3-4 Poor performers 13s, 22s, R4s, 41s, 10x Two levels of QC two

times a day

<3 Problematic tests 13s, 22s, R4s, 41s, 10x Three levels of QC three

times a day

Cooper G. Clin Chem Lab Med (2011) 49:5, 793-802

Problem with IQC

• 1. Cost 1,2

• 2. Reconstitution for lyophilized QC 1,2

• 3. Commutability matter 3

• 4. Storage 1,2

• 5. Liquide samples have to be ship frozen 1,2

• 6. Variation between QC lots need patient samples 1,2

• 7. Variation between shipments 1,2

• 8. Control deterioration 1,2

• 9. Not dynamic 1,2

3. Current limitations of IQC

1 Gras J. Laboratory Quality Control and patient Safety, De Gruyter (2017) 94 pp

2 Kazmierczak SC. Clin Chem Lab Med (2003) 41:5, 617-27 3 Miller W. Clin Chem (2011) 57:1, 76-83

Problem with IQC

• 10. Within-subject biological variation 1,2

Variation across studies

A checklist for critical appraisal of studies of biological variation has been recently

proposed 3

Lacking for hemostasis (D-dimer, dRVVT, ADAMTS13, …) 4,5

• 11. Only the analytical part of the process 6

3. Current limitations of IQC

1 Gras J. Laboratory Quality Control and patient Safety, De Gruyter (2017) 94 pp

2 Kazmierczak SC. Clin Chem Lab Med (2003) 41:5, 617-27 3 Bartlett. Clin Chem Lab Med (2015) 53:6, 879-885 4 de Maat MP. Clin Chem (2016) 62, 1639-1646 5 Shou W. Semin Thromb Hemost (2016) 42, 87-92 6 Kitchen S et al. Quality in Laboratory Hemostasis and Thrombosis, 2nd Ed, Wiley-Blackwell, 2013, 283 pp

Other approaches

4. Alternatives

1 Westgard J. Basic QC Practice: training in statistical quality controls for medical laboratories. 4th Edition, 2016, 334 pp 2 Gras J. Laboratory Quality Control and patient Safety, De Gruyter (2017) 94 pp 3 Carey RN. Clin Lab Med (2013) 33, 139-146 4 Badrick T. Clin Chem Lab Med (2018) 56:4, 549-553 5 Ceriotti F. Clin Chem Lab Med (2015) 53:6, 913-918 6 Tsiamyrtzis P. Technometrics (2005) 47:4, 446-456 7 Sobas F. Blood Coagul Fibrinolysis (2010) 21, 289-295 8 Tsiamyrtzis P. Blood Coagul Fibrinolysis (2015) 26, 590-596

Repeat patient test controls 1 Student’s t-test 1,2

Delta checks 2 Multiparametric checks 2

Average of normals 3 “Real time” external quality control 4

Follow-up of medians 2 Measurement uncertainty 5

Bayesian approach 6,7,8 …

1 Westgard survey 2017 (https://www.westgard.com/great-global-qc-survey-results.htm)

2 Kitchen S et al. Quality in Laboratory Hemostasis and Thrombosis, 2nd Ed, Wiley-Blackwell, 2013, 283 pp 3 Shewhart WA (1931) Economic control of quality of manufactured product. New York, NY: Van Nostrand; 1931; Vol.. 4 Westgard J. Basic QC Practice: training in statistical quality controls for medical laboratories. 4th Edition, 2016, 334 pp 5 CLSI 24-A3 6 Sobas F. Blood Coagul Fibrinolysis (2010) 21, 289-295

Rational

• Manufacturer’s data are too wide 1

• Target (mean) and range (SD) should be established locally during the

preliminary phase (20 days over at least a 10 day period) 2,3,4,5

• However, 20 days is judged to be insufficient to assess the “true” variability 3,6

• Main consequence Risk of high Pfr

5. Bayesian approach

1 Westgard survey 2017 (https://www.westgard.com/great-global-qc-survey-results.htm)

2 Kitchen S et al. Quality in Laboratory Hemostasis and Thrombosis, 2nd Ed, Wiley-Blackwell, 2013, 283 pp 3 Shewhart WA (1931) Economic control of quality of manufactured product. New York, NY: Van Nostrand; 1931; Vol.. 4 Westgard J. Basic QC Practice: training in statistical quality controls for medical laboratories. 4th Edition, 2016, 334 pp 5 CLSI 24-A3 6 Sobas F. Blood Coagul Fibrinolysis (2010) 21, 289-295

Bayesian model

• Predictive approach considering the mean and the SD as random variables

• The probability of a quality control value lying within acceptability limits (a-

posteriori probability) depends on the probability distribution of previous

control values (a-priori probability)

• No need for a preliminary phase

5. Bayesian approach

Bayesian model

• Dynamic model: each time new IQC results come in, prior information is

automatically updated

• A-priori knowledge obtained during the “validation phase” serves to draw up

the IQC control plan 1,2

• A-posteriori probability of not detecting a shift if the analytic system is under

statistical control calculated as follow:

5. Bayesian approach

P = a-posteriori probability of not detecting a shift (≧90% considered good)

Ɵn = mobile mean

M = cut-off value (concept of bio-clinical specification defining the size of the shift)

Xn = new individual value

1 Tsiamyrtzis P. Technometrics (2005) 47:4, 446-456 2 Sobas F. Blood Coagul Fibrinolysis (2010) 21, 289-295

• Same shift detection capacity as the 12s rule approach implemented after a

preliminary phase 1,2

5. Bayesian approach

1 Tsiamyrtzis P. Technometrics (2005) 47:4, 446-456 2 Sobas F. Blood Coagul Fibrinolysis (2010) 21, 289-295

P <0.9 out of

control

• Utilization of prior manufacturer specifications with Bayesian logic eludes

preliminary phase issues in quality control 1

• The laboratory environment is taken into account (continuous “retarget”) and

the weight of the manufacturer specifications tend to decrease

• As effective as the 1ks rule in detecting increases in both random and

systematic error after the minimal preliminary phase

5. Bayesian approach

1 Tsiamyrtzis P. Blood Coagul Fibrinolysis (2015) 26, 590-596

• Predictive control chart: manufacturer prior data (max CV = 10% and target =

99%) and inter-assay SD of the lab (4.67%)

• Conventional control chart

5. Bayesian approach (antithrombin)

5. Bayesian approach

Pros Cons

Dynamic (Ɵn) Complexity of calculation

No need for a “preliminary phase” Still a “preliminary phase” *

Rapid statistical control Compared to 12s **

Objective day-to-day calculation Need further validation *** Pfr? Ped?

« M » specification Same limitations of current IQC ****

Better with small SD

* A-priori knowledge

** Multiple rules and sigma-metrics?

*** Applications published only for factor VIII, PT and aPTT; lower Pfr?

**** Commutability? Preanalytics? Biological variation studies

1 Westgard J. Basic QC Practice: training in statistical quality controls for medical laboratories. 4th Edition, 2016, 334 pp 2 Parvin CA. Clin Chem (2012) 58:5, 925-929 3 Gras J. Laboratory Quality Control and patient Safety, De Gruyter (2017) 94 pp 4 Kitchen S et al. Quality in Laboratory Hemostasis and Thrombosis, 2nd Ed, Wiley-Blackwell, 2013, 283 pp

1. Concepts not widely used and developed for hemostasis laboratories

2. The 12s rule is no more recommended (high Pfr) 1,2

3. Strategies based on performance specifications 1,3

Sigma = (TEa% – Bias%)/CV%

Variation across studies (analyzers, statistical approach, …)

Lacking for hemostasis (dRVVT, ADAMTS13, D-dimer, Factors, …)

Cost-effective

4. Focus on analytic errors but not preanalytic errors (60-70% of errors) 4

5. Bayes approach compared to sigma approach still required

6. Bayes approach combining IQC and patient data?

Take Home Messages

– Maité Guldenpfennig

– Célia Devroye

– PharmD Hugues Jacqmin

– Prof Bernard Chatelain

– Prof François Mullier

Acknowledgement

Westgard rules (3)

• Sigma-metric (σ) = (TEa% – Bias%)/CV%

• Examples CHU UCL Namur (STAR-Max²):

– Fibrinogen (clauss): TEa = 20%, bias = 0.1% and CV = 4.0% Sigma-metric = 5 (3 rules)

– Current strategy in CHU UCL Namur: 13s, 22s, R4s, 10x Two levels of QC three times per day

– Advantage? Less rules, less QC per day cost ↓

– Limitation: not widely studied in hemostasis laboratories

2. Westgard rules

Sigma metric Category Proposed QC rule Proposed QC frequency

4-6 Tests that are suited for

purpose 12.5s Two levels of QC per day

Cooper G. Clin Chem Lab Med (2011) 49:5, 793-802