“Overview of methods for handling missing values”

Abstract

The problem of missing values has existed since the earliest attempts to exploit data as a source of knowledge, as it is intrinsically linked to the process of collecting, recording, and preparing the data itself. It is all the more unavoidable given that vast amounts of data are currently collected from different sources: "One of the ironies of Big Data is that missing data plays an increasingly important role." There is a vast literature on this topic, and a recent survey even identified more than 150 different implementations.

In this presentation, I will share my experience on the topic. I will start with the inferential framework and then show how missing values create additional challenges for supervised learning, as traditional machine learning algorithms cannot handle incomplete data. Finally, I will illustrate the impact of methods developed in the field of causal inference to estimate treatment effects from clinical data.