🎓📑✨ Preprint ( #AIstats ): survival analysis and competing risks with scalable stochastic solvers that recovers well outcome probabilities.
TL;DR: using stochastic optimization, our approach gives survival models both faster and predicting better event probabilities
🧵
1/7
TL;DR: using stochastic optimization, our approach gives survival models both faster and predicting better event probabilities
🧵
1/7
Comments
We proposed something very similar a while ago ( https://jmlr.org/papers/volume23/19-450/19-450.pdf ).
I toyed at the time with the idea of jointly estimating the weight and the survival estimator in an iterative fashion but it wasn't stable.
Did you have any divergence problems ?
No, we did not have any divergence problems. The solver is actually very stable.
Try it out! The code is linked in the manuscript.
For some "censored" individuals, the window was too short to observe the outcome (common in marketing, insurance, medicine)
2/
Censoring must be compensated: dropping individuals with unobserved outcome leads to bias (neglecting cases with long times).
Solutions are typically non-scalable, and practitioners often use linear models.
3/7
It is a sum over individuals, and can be optimized by stochastic methods.
It does not need time-wise gradients, and can thus use tree-based models.
4/7
This loss can be plugged into any stochastic solver, eg neural networks or Gradient-Boosted Trees, with an alternate optimization of censoring
5/7
On simulated data, 4 real-life datasets, and a variety of metrics, SurvivalBoost is markedly better predictor and faster than alternatives, sometimes much much faster.
It really enables easy survival analysis and competing risk on large data, such as electronic health records.
Read it here:
https://arxiv.org/abs/2410.16765
6/7
Are these 2 separate boosting trees, or can they share parameters?