Deep learning in kernel methods Afzal A L

The attempt to build algorithms to solve cognitive tasks such as visual object or
pattern recognition, speech perception, language understanding etc. have attracted the
attention of many machine learning researchers in the recent past. The theoretical and
biological arguments in this context strongly suggest that building such systems requires
deep learning architectures that involve many layers of nonlinear information processing.
Deep learning approach has originally emerged and been widely used in the area
of neural networks. The techniques developed from deep learning research have already
been impacting a wide range of signal and information processing applications. In the
recent past, excited by the startling performance that deep learning approaches have to
offer, there are many attempts to embrace deep learning techniques in other machine
learning paradigms, particularly in kernel machines. Convex optimization, structural
risk minimization, margin maximization, etc. are the some of the elegant features that
makes kernel machines popular among the researchers. With the advent of recently
developed multi layered kernel called arc-cosine kernel, the multilayer computations is
made possible in kernel machines. The multi-layered feature learning perceptiveness of
deep learning architecture have been re-created in kernel machines through the model
called Multilayer Kernel Machines(MKMs). Support vector machines were often used
as the classifier in these models. These deep models have been widely used in many
applications that involves small-size datasets. However the scalability,
multilayer multiple kernel learning, unsupervised feature learning etc. were untouched in the context
of kernel machines. This research explored above problems and developed three deep
kernel learning models viz; (i) Deep kernel learning in core vector machine that analyze the
behavior of arc-cosine kernel and modeled a scalable deep kernel machine by
incorporating arc-cosine kernel in core vector machines. (ii) Deep multiple multilayer
kernel learning in core vector machines modeled a scalable deep learning architecture
with unsupervised feature extraction. Each feature extraction layer in this model exploit
multiple kernel learning framework that involves both single layer and multilayer kernel
computations. (iii) Deep kernel based extreme learning machine combines the multilayer
kernel computation of arc-cosine kernel and fast, non-iterative learning mechanism
of Extreme Learning Machines. The theoretical and empirical analysis of the proposed
methods show promising results.

PhD Thesis September 2018 INT Dr Asharaf S
IIITMK