online read us now
Paper details
Number 3 - September 2019
Volume 29 - 2019
A fast neural network learning algorithm with approximate singular value decomposition
Norbert Jankowski, Rafał Linowiecki
Abstract
The learning of neural networks is becoming more and more important. Researchers have constructed dozens of learning
algorithms, but it is still necessary to develop faster, more flexible, or more accurate learning algorithms. With fast learning
we can examine more learning scenarios for a given problem, especially in the case of meta-learning. In this article we focus
on the construction of a much faster learning algorithm and its modifications, especially for nonlinear versions of neural
networks. The main idea of this algorithm lies in the usage of fast approximation of the Moore–Penrose pseudo-inverse
matrix. The complexity of the original singular value decomposition algorithm is O(mn2). We consider algorithms with a complexity of O(mnl), where l < n and l is often significantly smaller than n. Such learning algorithms can be applied to the learning of radial basis function networks, extreme learning machines or deep ELMs, principal component analysis or even missing data imputation.
Keywords
Moore–Penrose pseudo-inverse learning, radial basis function network, extreme learning machines, kernel methods, machine learning, singular value decomposition, deep extreme learning, principal component analysis