Win Odds, Win Ratio, and Net Benefit

Setup

Load the package hce and check the version:

library(hce)
packageVersion("hce")
#> [1] '0.6.5'

For citing the package, run citation("hce") (Samvel B. Gasparyan 2024).

Terminology

The concept of win probability for binary and continuous outcomes has been described by Buyse (2010) as the “proportion in favor of treatment” (see also Rauch et al. (2014)), while in Verbeeck et al. (2021) refers to it as the “probabilistic index”.

The concept of “win ratio” was introduced by Pocock et al. (2012). Unlike the win odds, the win ratio does not account for ties. The win odds, however, is the odds of winning, as described by G. Dong et al. (2020) (see also Peng (2020); Brunner, Vandemeulebroecke, and Mütze (2021); Samvel B. Gasparyan, Kowalewski, et al. (2021)). The same statistic was named as the Mann-Whitney odds by O’Brien and Castelloe (2006). In Samvel B. Gasparyan, Folkvaljon, et al. (2021), the “win ratio” was used as a general term and included ties in the definition. Gaohong Dong et al. (2022) suggested considering win ratio, win odds, and net benefit together as win statistics.

The concept of winning for the active group is the same as concordance for the active group. For two variables, \(X\) and \(Y,\) a pair is concordant if the observation with the larger value of \(X\) also has the better value of \(Y\) (Agresti 2013). If \(X\) indicates the treatment group with the value 1 for the active group and 0 for the control group, and \(Y\) is the ordinal value for the analysis, then concordance means that a patient in the active treatment group has a better value than a patient in the control group. Discordance means the patient in the active group has a worse value than the control patient. Therefore, the win ratio is the total number of concordances divided by the total number of discordances. The win ratio can be obtained from the Goodman-Kruskal gamma (Kruskal and Goodman 1954), \(G\), as follows:

\[WR=(1+G)/(1-G).\]

The net benefit is Somers’ D C/R (Somers 1962), while the win odds is the Mann-Whitney odds (Mann and Whitney 1947). Estimation of win statistics in the absence of censoring can be done using the theory of U-statistics (Hoeffding 1948).

Definitions

Two treatment groups are compared using an ordinal endpoint, and each comparison results in a win, loss, or tie for the patient in the active group compared to a patient in the placebo group. All possible (overall) combinations are denoted by \(O\), with \(W\) denoting the total wins for the active group, \(L\) the total losses, and \(T\) the total ties, so that \(O=W+L+T.\) The following quantities are called win statistics:

  • Win Probability defined as \(WP=\frac{W+0.5T}{O}\), which is the total number of wins, adding half of the total number of ties, divided by the overall number of comparisons.

  • Number Needed to Treat defined as \(NNT=\frac{1}{2WP-1}=\frac{O}{W-L}\) (rounded up to the nearest natural number for interpretation).

  • Win Ratio defined as \(WR=\frac{W}{L}\).

  • Win Odds defined as \(WO=\frac{W+0.5T}{L+0.5T} = \frac{WP}{1-WP}\).

  • Net Benefit defined as \(NB=\frac{W - L}{O}=2WP-1=\frac{1}{NNT}\).

Given the overall number of comparisons \(O,\) the win proportion \(WP\), and the win ratio \(WR\), it is possible to find the total number of wins and losses: \[\begin{align*} &L = O*\frac{2WP-1}{WR-1},\nonumber\\ &W = WR*L = WR*O*\frac{2WP-1}{WR-1},\nonumber\\ &T=O-W-L = O*\left[1 - (WR+1)\frac{2WP-1}{WR-1}\right]. \end{align*}\]

The function propWINS() implements the formula above:

args("propWINS")
#> function (WO, WR, Overall = 1, alpha = NULL, N = NULL) 
#> NULL
propWINS(WO = 1.5, WR = 2)
#>   WIN LOSS TIE TOTAL WR  WO
#> 1   0    0   1     1  2 1.5

Suppose there are \(n_1=120\) patients in the placebo group and \(n_2=150\) in the active group. If the win ratio is 1.5 and the win odds is 1.25, then the number of wins and losses for the active group can be calculated using the Overall argument, which represents all possible comparisons.

propWINS(WO = 1.25, WR = 1.5, Overall = 120*150)
#>    WIN LOSS  TIE TOTAL  WR   WO
#> 1 6000 4000 8000 18000 1.5 1.25

This function call will provide the number of wins, losses, and ties for the active group based on the specified win odds and win ratio.

References

Agresti, Alan. 2013. Categorical Data Analysis. John Wiley & Sons. https://www.wiley.com/en-gb/Categorical+Data+Analysis%2C+3rd+Edition-p-9780470463635.
Brunner, Edgar, Marc Vandemeulebroecke, and Tobias Mütze. 2021. “Win Odds: An Adaptation of the Win Ratio to Include Ties.” Statistics in Medicine. https://doi.org/10.1002/sim.8967.
Buyse, Marc. 2010. “Generalized Pairwise Comparisons of Prioritized Outcomes in the Two-Sample Problem.” Statistics in Medicine 29 (30): 3245–57. https://doi.org/10.1002/sim.3923.
Dong, Gaohong, Bo Huang, Johan Verbeeck, Ying Cui, James Song, Margaret Gamalo-Siebers, Duolao Wang, et al. 2022. “Win Statistics (Win Ratio, Win Odds, and Net Benefit) Can Complement One Another to Show the Strength of the Treatment Effect on Time-to-Event Outcomes.” Pharmaceutical Statistics. https://doi.org/10.1002/pst.2251.
Dong, G, DC Hoaglin, J Qiu, RA Matsouaka YW Chang, J Wang, and M Vandemeulebroecke. 2020. “The Win Ratio: On Interpretation and Handling of Ties.” Statistics in Biopharmaceutical Research 12 (1): 99–106. https://doi.org/10.1080/19466315.2019.1575279.
Gasparyan, Samvel B. 2024. hce: Design and Analysis of Hierarchical Composite Endpoints. CRAN: The Comprehensive R Archive Network, R Package, Version 0.6.5. https://CRAN.R-project.org/package=hce.
Gasparyan, Samvel B, Folke Folkvaljon, Olof Bengtsson, Joan Buenconsejo, and Gary G Koch. 2021. “Adjusted Win Ratio with Stratification: Calculation Methods and Interpretation.” Statistical Methods in Medical Research 30 (2): 580–611. https://doi.org/10.1177/0962280220942558.
Gasparyan, Samvel B, Elaine K Kowalewski, Folke Folkvaljon, Olof Bengtsson, Joan Buenconsejo, John Adler, and Gary G Koch. 2021. “Power and Sample Size Calculation for the Win Odds Test: Application to an Ordinal Endpoint in COVID-19 Trials.” Journal of Biopharmaceutical Statistics 31 (6): 765–87. https://doi.org/10.1080/10543406.2021.1968893.
Hoeffding, W. 1948. “A Class of Statistics with Asymptotically Normal Distribution.” The Annals of Mathematical Statistics 19 (3): 293–325. https://doi.org/10.1214/aoms/1177730196.
Kruskal, William H, and Leo Goodman. 1954. “Measures of Association for Cross Classifications.” Journal of the American Statistical Association 49 (268): 732–64. https://doi.org/10.2307/2281536.
Mann, Henry B, and Donald R Whitney. 1947. “On a Test of Whether One of Two Random Variables Is Stochastically Larger Than the Other.” The Annals of Mathematical Statistics, 50–60.
O’Brien, RG, and JM Castelloe. 2006. Exploiting the Link Between the Wilcoxon–Mann–Whitney Test and a Simple Odds Statistic. Cary, NC: SAS Institute Inc. https://support.sas.com/resources/papers/proceedings/proceedings/sugi31/209-31.pdf.
Peng, Lei. 2020. “The Use of the Win Odds in the Design of Non-Inferiority Clinical Trials.” Journal of Biopharmaceutical Statistics 30 (5): 941–46. https://doi.org/10.1080/10543406.2020.1757690.
Pocock, SJ, CA Ariti, TJ Collier, and D Wang. 2012. “The Win Ratio: A New Approach to the Analysis of Composite Endpoints in Clinical Trials Based on Clinical Priorities.” European Heart Journal 33 (2): 176–82. https://doi.org/10.1093/eurheartj/ehr352.
Rauch, Geraldine, Antje Jahn-Eimermacher, Werner Brannath, and Meinhard Kieser. 2014. “Opportunities and Challenges of Combined Effect Measures Based on Prioritized Outcomes.” Statistics in Medicine 33 (7): 1104–20. https://doi.org/10.1002/sim.3923.
Somers, Robert H. 1962. “A New Asymmetric Measure of Association for Ordinal Variables.” American Sociological Review, 799–811. https://doi.org/10.2307/2090408.
Verbeeck, Johan, Vaiva Deltuvaite-Thomas, Ben Berckmoes, Tomasz Burzykowski, Marc Aerts, Olivier Thas, Marc Buyse, and Geert Molenberghs. 2021. “Unbiasedness and Efficiency of Non-Parametric and UMVUE Estimators of the Probabilistic Index and Related Statistics.” Statistical Methods in Medical Research 30 (3): 747–68. https://doi.org/10.1177/0962280220966629.