Hand's paradox

In statistics, Hand's paradox arises from ambiguity when comparing two treatments. It shows that a comparison of the effects of the treatments applied to two independent groups that can contradict a comparison between the effects of both treatments applied to a single group.

Paradox

Comparisons of two treatments often involve comparing the responses of a random sample of patients receiving one treatment with an independent random sample receiving the other. One commonly used measure of the difference is then the probability that a randomly chosen member of one group will have a higher score than a randomly chosen member of the other group. However, in many situations, interest really lies on which of the two treatments will give a randomly chosen patient the greater probability of doing better. These two measures, a comparison between two randomly chosen patients, one from each group, and a comparison of treatment effects on a randomly chosen patient, can lead to different conclusions.

This has been called Hand's paradox, and appears to have first been described by David J. Hand.

Examples

Example 1

Label the two treatments A and B and suppose that:

Patient 1 would have responded values 2 and 3 to A and B respectively. Patient 2 would have responded values 4 and 5 to A and B respectively. Patient 3 would have responded values 6 and 1 to A and B respectively.

Then the probability that the response to A of a randomly chosen patient is greater than the response to B of a randomly chosen patient is 6/9 = 2/3. But the probability that a randomly chosen patient will have a greater response to A than B is 1/3. Thus a simple comparison of two independent groups may suggest that patients have a higher probability of doing better under A, whereas in fact patients have a higher probability of doing better under B.

Example 2

Suppose we have two random variables, $x_{A}\sim N(1,1)$ and $x_{B}\sim N(0,1)$, corresponding to the effects of two treatments. If we assume that $x_{A}$ and $x_{B}$ are independent, then $\Pr(x_{A}>x_{B})=0.76$, suggesting that A is more likely to benefit a patient than B. In contrast, the joint distribution which minimizes $Pr(x_{A}>x_{B})$ leads to $Pr(x_{A}>x_{B})\geq 0.38$. This means that it is possible that in up to 62% of cases treatment B is better than treatment A.