Information divergence is relevant to many applications ranging from information theory, machine learning, and signal processing. There are various approaches to derive information divergences, approximately but exploring accurate and fast estimation has been often a challenge, especially for high dimensional data when there is no parametric model for the data. Previous works have shown that by applying plug-in approaches we can empirically estimate divergences when the distributions are unknown but random samples from these distributions are available. The drawback with the plug-in estimates is that these methods are not accurate near support boundaries and are computationally intensive. 

In this talk, I will aim to introduce approaches where we can efficiently estimate information divergences directly using the Friedman-Rafstky (FR) statistic, Nearest Neighbour Ratio (NNR), and Cross-Match statistic based on geometric graphs such as minimal spanning trees (MST), K-nearest neighbours graph (K-NNG), and optimal weighted matching (OWM), respectively. Unlike the FR statistic approach, we argue that the cross-match statistic is dimension-independent and this causes the cross-match statistic to perform more accurately in higher dimensions. Lastly, we show that our new direct estimators outperform previous plug-in methods such as Kernel density estimation (KDE) in addition to maintaining a computational advantage.