Essay Essays are opinion pieces on a topic of broad interest to a general medical audience.

See all article types »

Is Scale-Up Worth It? Challenges in Economic Analysis of Diagnostic Tests for Tuberculosis

  • David W. Dowdy mail,

    Affiliation: Department of Medicine, University of California, San Francisco, California, United States of America

    Current address: Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United States of America

  • Adithya Cattamanchi,

    Affiliation: Division of Pulmonary and Critical Care Medicine, San Francisco General Hospital, San Francisco, California, United States of America

  • Karen R. Steingart,

    Affiliation: Curry International Tuberculosis Center, San Francisco, California, United States of America

  • Madhukar Pai

    Affiliation: Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Canada

  • Published: July 26, 2011
  • DOI: 10.1371/journal.pmed.1001063

Reader Comments (1)

Post a new comment on this article

False positives in tuberculosis.

Posted by Jef_Van_den_Ende on 29 Aug 2011 at 13:44 GMT

To the editor,

Generally speaking, we agree with the messages Dowdy et al. wrote in the 28 of July issue of Plos Medicine, in their article “Is Scale-Up Worth It? Challenges in Economic Analysis of Diagnostic Tests for Tuberculosis”. But we have some serious concerns about their chapter on false positive diagnoses.

The authors fear induction of drug resistance by treating false positives. The question arises how resistance can be created in patients who do not have tuberculosis (TB) disease? Full ongoing treatment will even eradicate the few bacilli of possible latent TB, and will rapidly kill occasional bacilli of a new infection. This is completely different from the selection of resistant strains in a patient with undiscovered active TB who is treated with e.g., fluoroquinolones for another infection.

Further they cite our work in Rwanda with “… it has been argued that treating 29 false-positives for every additional case of active TB would be cost-effective”, and state “… it is unlikely that patients or physicians would accept a diagnosis that is wrong 29 times out of 30”. Our article analyzed the minimum disease probability that would justify a treatment taking into account the relative expected harm done to true and false positives and to false negatives. It was not on cost-effectiveness, but about the Pauker&Kassirer threshold concept, an exercise in expected utility, using a cost benefit analysis in a hypothetical cohort of patients with a marginally low post test probability.(1) Thresholds concern the required probability for marginal cases, rather than the mean disease probability of the cohort, which is represented by the Blackstone error.(2) In two publications about thresholds in TB,(3, 4) we show that the distribution of disease probability has a U-shape: most patients have either a high or a low probability, reflecting their having or not the disease. If we would accept to treat any patient over a threshold of 0.10, it does not mean that only 10 treatments out of 100 will be administered to patients with the disease, as the 100 will include patients with confirmed TB and patients with a high probablility of disease, as well as some patients with a low probability.

Finally, the authors cite our work on local preferences. The message of our research was that we should not rely on intuitive estimations of weight of false positives vs. false negatives. We show that calculation of these weights based on the data clinicians know or estimate themselves, proves these intuitive weights to be completely erroneous. In stead of the intuitive 2 to 1 ratio, calculated weights give a ratio of 3.7 vs. 00.5. Clinicians were accurate in estimating probabilities but failed to incorporate them into therapeutic decisions.

Jef Van den Ende, Institute of Tropical Medicine, Antwerp, Belgium
Zeno Bisoffi, Centre for Tropical Diseases, S. Cuore Hospital, Negrar (Verona), Italy
Paulin Basinga, School of Public Health, National University of Rwanda, Kigali, Rwanda.

(1) Pauker SG, Kassirer JP. Therapeutic decision making: a cost-benefit analysis. N Engl J Med 1975;293(5):229-34.
(2) DeKay ML. The difference between Blackstone-Like error ratios and the probabilistics standards of proof. Law and Social Inquiry 1996;21:95-132.
(3) Van den Ende J, Mugabekazi J, Moreira J, et al. Effect of applying a treatment threshold in a population. An example of pulmonary tuberculosis in Rwanda. J Eval Clin Pract 2010;16(3):499-508.
(4) Tuyisenge L, Ndimubanzi CP, Ndayisaba G, et al. Evaluation of latent class analysis and decision thresholds to guide the diagnosis of pediatric tuberculosis in a Rwandan reference hospital. Pediatr Infect Dis J 2010;29(2):e11-e18.

No competing interests declared.

RE: False positives in tuberculosis.

ddowdy replied to Jef_Van_den_Ende on 06 Sep 2011 at 04:54 GMT

We appreciate the thoughtful comments of Drs. Van den Ende, Bisoffi, and Basinga, and we thank them for their general support of our manuscript’s ideas. The authors point out that treatment of individuals without TB disease does not induce resistance to anti-TB medications. At the individual level, this is undoubtedly true. However, a key concern – not discussed at length in our manuscript – is that large-scale treatment of false-positives at the population level greatly increases the distribution of TB drugs, with potential adverse consequences. These include stock-outs or “stretching” (1) of drugs for people with active TB, and sharing of medicine with symptomatic friends or family members (2) where directly-observed therapy is not strict. Thus, we would argue that increased circulation of anti-TB drugs from false-positive diagnosis increases the population-level risk of drug resistance, even when the individual being treated is not affected.

We agree with the comments regarding the importance of distinguishing between marginal and mean cases to avoid Blackstone-type errors. Thus, the authors make a key point that a threshold probability of 1/30 does not imply that 29 of 30 diagnoses are incorrect in the aggregate. Nevertheless, it remains true that a marginal diagnosis on an individual patient with a post-test probability of 1/30 would be wrong 29 times out of 30, and that the expected-utility framework discussed would advocate for treatment of such patients. The authors make an important distinction between expected-utility thresholds and cost-effectiveness analysis; however, given that this particular threshold analysis (3) included costs and effectiveness measures, it can be shown that a corresponding cost-effectiveness analysis should reach the same conclusion. Thus, in the end, economists working under either framework would recommend that patients or physicians accept a (marginal) diagnosis that is expected to be wrong 29 times out of 30. This recommendation is appropriate from a classical economic perspective, but is unlikely to find acceptance among physicians.

Finally, Dr. Van den Ende and colleagues point out that physicians’ intuitive weights differ dramatically from calculated weights based on their accurate knowledge of TB biology. We would argue that neither the calculated nor intuitive weights are erroneous; rather, the authors’ analysis (3) elegantly shows that physicians are not rational in their decision-making. Nevertheless, physicians’ intuitive weights must be considered, as these weights – not calculated weights – ultimately guide their decisions. As we argue, “[w]hen local preferences seem inappropriate to policy-makers, educational efforts or recommendations for empiric therapy should be prioritized over scale-up of novel diagnostics.” Ultimately, if new diagnostics do not live up to physicians’ intuitive expectations, they are unlikely to be used, regardless of how irrational those intuitive weights may be.

David Dowdy, Johns Hopkins Bloomberg School of Public Health, Baltimore, USA

Adithya Cattamanchi, San Francisco General Hospital, San Francisco, USA

Karen Steingart, Curry International Tuberculosis Center, San Francisco, USA

1. Rookkapan K, Chongsuvivatwong V, Kasiwong S, Pariyawatee S, Kasetcharoen Y, et al. (2005) Deteriorated tuberculosis drugs and management system problems in lower southern thailand. The International Journal of Tuberculosis and Lung Disease 9(6): 654-660.

2. Goldsworthy RC, Schwartz NC, Mayhorn CB. (2008) Beyond abuse and exposure: Framing the impact of prescription-medication sharing. Am J Public Health 98(6): 1115-1121.

3. Basinga P, Moreira J, Bisoffi Z, Bisig B, Van den Ende J. (2007) Why are clinicians reluctant to treat smear-negative tuberculosis? an inquiry about treatment thresholds in rwanda. Medical Decision Making 27(1): 53-60.

No competing interests declared.