Info Geometric

Information Geometric Characterizations and Applications of Machine Learning Algorithms

With the information geometric method from information theory, we build a theoretical framework to analyze features in machine learning algorithms. This framework has revealed the essential connections between information theory and machine learning. In particular, our framework has revealed that the feature extracted in deep neural networks is the most informative feature from the perspective of information theory, which provides a theoretic interpretation of deep neural networks. In addition to the theoretic interpretations, our framework also suggests practical approaches for extracting the common information between multi-sourced data, which can then be used for multi-modal data processing and privacy preserving.