Clinical medicine has talked about thresholds starting with Pauker and Kassirer (1975)
on treatment thresholds (the probability that this patient has the disease, above which you would
treat the disease, below which you would not) and Pauker and Kassirer (1980) on test thresholds
(a low probability, below which you would neither test nor treat; a high probability, above which
you would treat; and do the test in the middle range and treat according to the test result).
Signal detection theory, in engineering and psychology, has talked about thresholds
for calling the situation a signal, famously with World War II analysis of sonar signals and explicitly
in psychology with Green and Swets as a reference milestone.
The thresholds in clinical medicine depend only on the utilities of the two errors,
false positives and false negatives. The thresholds in SDT depend on those utilities as well as
the prior probabilities. The purpose of this calculator is to illustrate why these two thresholds differ.
The approach will be to produce three graphs: a Strength of Evidence Graph (from
signal detection theory), a Pauker and Kassirer threshold graph (from clinical medicine), and
a Bayes' graph (Bayes' football; from clinical medicine) and illustrate how they integrate. All will
The Strength of Evidence Graph has signal strength or evidence strength on the x axis,
and the y axis shows the probability density curves for each level of strength of evidence,
for the hypothesis and for the non hypothesis. Two separate density curves. We define each
density curve as summing to 1. We pick the point of maximum separation, ignoring base rate and utility of errors.
We leave out of the Strength of Evidence Graph considerations of the base rate
as well as considerations of error utility. That is because we consider utility in the Threshold Graph (and pass
it through to the Bayes' Graph), and the Bayes' Graph allows you to apply the analysis to a patient
with any base rate, any prior probability. If we also considered those here, then in effect the same
considerations would be considered twice, and things conceptually would be a confused mess.
There is another way the evidence, under the two hypotheses, can be graphed:
the total data could recognize base rate, where the data in each curve occur as they would
in the environment. This could either be by dividing a total probability of 1 into two parts,
according to the base rate of H and not-H, or it could be a count of observations with normalization
coming later. (The choice to ignore base rate has implications for whether the base rate is
represented in this graph, or not. Pure likelyhood ratio ignores the base rate. Or it can represent the
total impact of evidence including the base rate.)
The Threshold Graph represents the utilities that would follow from the actions of treating
or not treating in two basic conditions: if the patient has the disease (hit and miss), at the right end of the x-axis
and if the patient does not have the disease (false alarm and correct rejection), at the left end of the
x-axis. The actions (treat and not treat) are connected by two straight lines, which cross. The lines
indicate the expected utility of that action in the case of the action, which is a probability mix between
having the disease and not having the disease. The point of indifference is the treatment threshold probability.
This threshold is dependent completely on the utilities of the actions. It is not in any way dependent on
the patient's actual probability, nor on the prevalence.
The Bayes Graph shows the effects of positive or negative observations on one test or
finding. The x-axis is the pretest probability. The Y axis is the post test probability. The curve showing the
post test probability following a positive observation, for every possible pre-test probability, is above
the ascending diagonal. The curve of the post test probability after a negative observation is below
the diagonal. The threshold (from the threshold graph) is a point on the axis with the most up to
date probability, the y-axis. Draw a straight horizontal line across the graph from that point.
Where it intersects the two post-test curves represents the no-test/test threshold (the intersection
with the positive test curve) and the test/treat threshold (the intersection with the negative test curve).
Project these down to the x-axis to read the probabilities that have those meanings. This is relevant
for the pre-test probability because you're considering whether to do a test, i.e., you have not done it.
The test characteristics which determine the positive test and negative test Bayes
curves come from the Strength of Evidence graph. At whatever cutoff point on the strength of evidence
continuum that you decide to call the evidence 'positive' (or, for a 'bin'), the likelihood ratio defines the
test characteristic to be used in interpreting the results. This may be a simple rule or a complex set of
rules, for example if multiple bins are used, or if we are considering the evidence from multiple tests
(conceptually a pair of distributions still).
Concept and Prototype by Rob Hamm
Robert M. Hamm, PhD; Clinical Decision Making Program; Department of Family Medicine [www.oumedicine.com/familymedicine]; University of Oklahoma Health Sciences Center; Oklahoma City OK 73190; 405/271-8000 ext 32306; Fax: 405/271-4125; email: email@example.com
Shiny Implementation by Will Beasley
William Howard Beasley, PhD; Biomedical and Behavioral Methodology Core [http://ouhsc.edu/bbmc/]; Department of Pediatrics; University of Oklahoma Health Sciences Center; email: firstname.lastname@example.org