Melanie Swan is discussing deep learning as a statistical method in data science and its philosophical implications. Statistics is a mode of viewing the world. We think that reality is composed of patterns, which can become clear through statistical and data science methods. In statistical data science methods, hypotheses are formulated in terms of probability distributions over possible datasets of empirical facts. In the application of statistical methods, learning is increasingly becoming a component. Statistical learning is applying automated learning algorithms to identify the features of systems and recognize patterns in very-large datasets.
Deep learning is an advanced statistical method that has come into use in recent years. In deep learning, algorithms learn from and make predictions about data, particularly by modeling high-level abstractions in datasets and assigning probability-weighting to nodes as they move through the system modeling. An important challenge in deep learning is that these methods seem to work, but we do not have a good explanation for why, which prevents a wider application to problems beyond the domains of their current success in image recognition (with convolutional nets) and text/speech recognition (with recurrent nets). This talk looks at what happens in deep learning, where there are multiple layers of increasingly abstract representation and non-linear operation used by algorithms to determine an answer. Deep learning is essentially a complexity optimization problem, where some concepts that may explain how deep learning systems operate include backpropagation, supervised-unsupervised learning, attractors, non-convexity, saddle-point local minima-maxima, and energy-based models.
The philosophy of statistical deep learning involves the meaning, justification, use, and interpretation of these methods to arrive at empirical scientific knowledge. How are we to understand and trust these methods as an explanatory mechanism for patterns in datasets? Since statistical data science methods are crucial to any field of scientific inquiry as an interpretation mechanism for empirical data, a better understanding of the theory underlying current topics such as deep learning may help to improve our overall understanding.
