Ebola Tests: Sensitivity and Specificity

Yesterday's headlines on USA Today included one that stated "Nebraska Patient Cases Raises Questions about Ebola Test."  The reason for this is the fact that the Nebraska patient was a physician treating Ebola patients who came down with symptoms and was tested and found not to be Ebola positive before other clinicians eventually realized (and found in a later test) that the man in question did have Ebola.  If the physician had been treated earlier, it is possible that his outcome--being brought back to the US and treated--may have been different from the unfortunate death he suffered.

I'd like to comment on this because it raises some really important issues about diagnostic tests.  I would have expected to find some previous writing on this, but I do not.  So here it goes.

People either have a condition or do not.  We refer to those who have the condition as positive and those who dont' as negative.  Sometimes it is good to be "positive."  With the Ebola virus, it is obvious that being positive is a very bad thing.

Test results are also positive or negative.  (Some tests are uncertain, but for the time being we will ignore that.)  So, the result at the end of a diagnostic test suggests that a person either has or does not have a condition.

What makes this most interesting is the fact that there are some people who are truly positive and for which the test identifies the person as positive.  Similarly, there are some people who are negative and whom the test identifies as negative.  These are both desirable outcomes.

However, there are also two other groups.  There are some people who are positive whom the test incorrectly identifies as negative.  And there are some people who are negative whom the test incorrectly identifies as positive.  

A test's ability to correctly identify those who are positive as positive is called sensitivity.  A test's ability to correctly identify those who are negative as negative is called specificity.

In different situations, clinicians and policy makes focus on either sensitivity or specificity.  Sometimes it is important not to over identify cases.  If the consequences of being positive are fairly small, there will be future attempts to diagnose, and further identification and treatment is costly, we can focus on making sure to identify negatives properly--specificity.  Or if there are strong negative connotations and stigmas associated with being positive it can be important to correctly identify negatives.

However there are other cases in which failing to identify a positive case can be deadly.  This is true with Ebola.  This is why in other cases sensitivity is critical.  This does not mean that there should not be concerns about specificity.  If we had a lot of people mis-identified as positive this would require an enormous amount of resources and there may be side effects of the treatment.

The main "question" I think we should asking about the Ebola test is whether there is a way to develop a test that is more sensitive.  And quickly.  And if the government can support that and someone can find it, there may be a profit to be made from an innovation for humanity.  

What is Difficult to Include in Cost-Benefit Analysis?

A couple weeks back, I discussed what size and type of investment might be appropriate in response to the current Ebola outbreak.  I also discussed how the answer might vary between the United States and other locations around the world.  And I tried to place the discussion in the context of just how big a problem Ebola is (or is not) in comparison with other public health problems.  

There are two interesting thoughts that follow this coming out of a recent Wall Street Journal piece. The first, is that it is somewhat ironic that in a part of the world that is home to the natural resource that produces rubber, a number of health care providers have had incident cases of Ebola for a lack of rubber gloves.  This is a public health problem.  But it is also a problem that demonstrates just how important the many inter-linkages between different parts of an economic system are.  And how having the natural resources doesn't guarantee that without a production facility, a supply chain, or an export (and re-import) opportunity anything will be done with the resource to help to solve the public health problem.  For this, a cost-benefit analysis could be conducted, analyzing the cost of improving the system of moving the natural resource into production, the implications of any threats to the permanent viability of the natural resource (anyone who has ever read The Lorax knows what can happen with overuse of a resource), and the benefit of reducing the Ebola transmission.  There are markets.  We can infer values.  We can conduct the analysis.

But even this analysis would not capture the other aspect that was brought out in the WSJ piece.  What is that?  The article talks about health care workers going on strike to demand improved protection, higher "risk-based" pay, and life insurance.  Then the article states

"In the meantime, because doctors aren't at work, other diseases besides Ebola are going untreated. As a result, those ailments—chiefly typhoid and dysentery—may be killing more West Africans than Ebola, according to the United Nations Children's Fund."

This, is something that most cost-benefit analyses do not include and would honestly have a difficult time predicting.  This, is where the cost of an uncontrolled Ebola outbreak really begins to grow--and potentially exponentially.  

To some degree it is the difference between a partial equilibrium analysis (looking just at Ebola treatment) and a general equilibrium analysis (asking how treating Ebola will impact the rest of the health care system and ultimately the rest of the economy).  

What it demonstrates is how hard it can be to make a complete analysis using the tools we have when there are unpredictable outcomes.  And how policy makers and business leaders must still make decisions with the best information available.  And reassess the information and update the decisions frequently.

This is one case in which the economic argument for incremental spending to control the disease is much clearer in light of the information about the breakdown of usual work and social roles.  

Mixed Messages About Public Health Surveillance

An opinion piece in the Wall Street Journal today left me wondering after I read it.  The piece focuses on how concerned we should be about Ebola.  The writer's opinion is that we should not worry too much.  While this outbreak has been bigger than any in the past and killed more than any in the past, the spread can be contained.  And while someone may emerge as being ill after reaching a more developed part of the world, it would be noticed, contained, and not spread widely.  

The writer mentions his own experience with respect to SARS (he was suspected of having it but did not, fortunately) and concludes the piece talking about international commitments to prevent the spread of disease throughout the world and the reason that the US should invest in science and public health surveillance at home and abroad.  

Many readers might be puzzled by that conclusion.

For control abroad (paritcularly for something like Ebola which is only spread through contact with bodily fluids) the steps to control can be fairly straightforward.   As the writer points out, cultural traditions of family caring for sick individuals at home and touching dead bodies in preparation for burial make avoiding all bodily fluids difficult.  (I do find it interesting that most accounts I have seen don't mention sweat as a bodily fluid by which the disease can be transmitted but it is clearly listed on a WHO website. That may be more of a cause for alarm in countries in which individuals are on public transportation in very close quarters.  Or perhaps for those of us who sweat like crazy at the gymnasium.)  However, the steps toward control are well known and may have as much to do with culture and the availability of personnel than anything else.

Would science help?  For preventing transmission and a cure, of course.  Would surveillance help?  For making sure that a minimal number of people are exposed, of course.  Would education and personnel help?  From what I have read, perhaps this would help more than anything else.  And while education often takes a long time, none of these will help very quickly.

For control at home in the US, it seems like we generally have things under control.  At least the writer does not suggest that there is a need to worry given current circumstances.  So, while a failure to act at all would be a disaster, our current infrastructure supports quite a bit of activity already.  Do we need that much beyond what we are already doing?  How much extra would it cost to go beyond what we are doing?  And if we are not likely to end up with a whole lot of morbidity or mortality with current activities in place, how much more would be prevented?  

I am in favor of wise icreases in investments in multiple aspects of public health and science.  But for an opinion piece to make the argument that there is not much to worry about while saying that investment is the right thing to do--unquestionably--is confusing.  Readers will wonder--appropriately, in my opinion--about how much investment will be needed to make a difference and how much of a difference will be made.  Clearly, in a world of limited resources, we cannot make every possibly investment in every mechanism to control all threats to public health.  This is one that appears to need more justification in light of what is already being done.  Using a potentially questionable line of logic to call for greater investment may not help when there is a clear need to make more sustainbed and larger investments later on.  

I, of course, am not saying that I have zero sympathy for the 1400+ infected so far.  I am saying that we have to live with some public health risks and this may just be one to live with or to leave to the market to solve.