Can I use survival analysis on a regression problem (not survival) with censored outcome

[ogencoglu]

I have a machine learning problem of regression (not survival) where I have certain numerical and categorical features and am trying to predict a numerical outcome. I know for a fact that the phenomena is highly non-linear. The problem has nothing to do with survival in that sense.

But for some of the outcomes (target values), I just know that they are below certain value. I don't know the exact value. Therefore I can not use standard scikit-learn regression models here as I can not calculate the loss accurately. Is survival analysis suitable for this use case?

And if so, what kind of methods would be suitable for such a regression problem?

[CamDavidsonPilon]

Hi @ogencoglu,

Yes, survival analysis generally models any censoring of outcomes. Survival is associated with right-censoring, but there is also left and interval censoring. In your case, you have left-censoring. Another common problem that has left-censoring is instrument readings, where some readings are below a detection limit. You can read more about left-censoring here.

Anyways, most of the models in lifelines expose a fit_left_censoring method that can be used to infer parameters and perform predictions for your dataset.

[ogencoglu]

Thanks for the swift reply @CamDavidsonPilon !

I have several follow-up questions:

1. Is there any fundamental mathematical or algorithmic difference between left-censoring and right-censoring in this context? For example, can I multiply my targets by -1 which will turn <20 to >-20 and switch from left-censoring to right censoring and the results will be the same?

2. Regarding the first question, does it support negative target values just like regular regression? Or does survival analysis assume some physical meaning of "survival" and therefore negative target values does not make sense or anything like that?

I am really trying to understand how generalizable survival analysis is to arbitrary regression problems where there is some censoring in the targets.

[pzivich]

1. Yes, there is a difference. Right censoring is unbounded (it goes from observed value to infinity) whereas left censoring is bounded (it goes from zero to the observed value).
2. Survival analysis assumes the times are all strictly positive, so negative values are incompatible.

Regarding your final comment, you might be able to transform your problem into one compatible with survival analysis methods but the 'times' do need to be positive. An example of survival analysis methods applied to non-survival data is in the environmental literature where they address left censoring due to limit of detection of chemicals https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8224532/

[ogencoglu]

I see. For my context, both are unbounded and the targets can be both negative or positive. I have left-censoring because sensors can not read below certain value.

I can not assume any linear or parametric underlying model and estimate parameters. What I need is the ability to use machine learning with this sort of data (random forests, gradient boosting machines etc.). I think I might need to implement these myself. Thanks.