A Simulated Annealing-based Learning Algorithm for Block-Diagonal Recurrent Neural Networks

P.A. Mastorocostas, D.N. Varsamis, and C.A. Mastorocostas (Greece)


In this context, this work proposes a modification of the standard SARPROP method, entitled Modified Simulated Annealing Resilient Back-propagation (M-SARPROP), which can be applied to a BDRNN, by taking into consideration the temporal relations existing in such a system. The rest of this paper is organized as follows: In Section 2 the structure and characteristics of the BDRNN are illustrated. The learning algorithm is developed in Section 3. In the next section a simulation example is


The RPROP algorithm was originally developed in [5] for static networks and constitutes one of the best performing first order learning methods for neural networks [6]. However, in RPROP the problem of poor convergence to local minima, faced by all gradient descent-based methods, is not fully eliminated. Hence, in an attempt to alleviate this drawback, a combination of RPROP with the global search technique of Simulated Annealing (SA) was introduced in [7]. The resulted algorithm, named SARPROP, was proved to be an efficient learning method for static neural networks. A fast and efficient training method for block-diagonal recurrent fuzzy neural networks is proposed. The method modifies the Simulated Annealing RPROP algorithm, originally developed for static models, in order to be applied to dynamic systems. A comparative analysis with a series of algorithms and recurrent models is given, indicating the effectiveness of the proposed learning approach.

Important Links:

Go Back