PolyU IR
 

PolyU Institutional Repository >
Electronic and Information Engineering >
EIE Journal/Magazine Articles >

Please use this identifier to cite or link to this item: http://hdl.handle.net/10397/189

Title: An improved algorithm for learning long-term dependency problems in adaptive processing of data structures
Authors: Cho, Siu-Yeung
Chi, Zheru George
Siu, Wan-chi
Tsoi, Ah Chung
Subjects: Adaptive processing of data structures
Back-propagation through structure (BPTS)
Least-squares method
Long-term dependency
Issue Date: Jul-2003
Publisher: IEEE
Citation: IEEE Transactions on neural networks, July 2003, v. 14, no. 4, p. 781-793.
Abstract: For the past decade, many researchers have explored the use of neural-network representations for the adaptive processing of data structures. One of the most popular learning formulations of data structure processing is backpropagation through structure (BPTS). The BPTS algorithm has been successful applied to a number of learning tasks that involve structural patterns such as logo and natural scene classification. The main limitations of the BPTS algorithm are attributed to slow convergence speed and the long-term dependency problem for the adaptive processing of data structures. In this paper, an improved algorithm is proposed to solve these problems. The idea of this algorithm is to optimize the free learning parameters of the neural network in the node representation by using least-squares-based optimization methods in a layer-by-layer fashion. Not only can fast convergence speed be achieved, but the long-term dependency problem can also be overcome since the vanishing of gradient information is avoided when our approach is applied to very deep tree structures.
Rights: © 2003 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.
Type: Journal/Magazine Article
URI: http://hdl.handle.net/10397/189
ISSN: 10459227
Appears in Collections:EIE Journal/Magazine Articles

Files in This Item:

File Description SizeFormat
adaptive-processing_03.pdf891.03 kBAdobe PDFView/Open



Facebook Facebook del.icio.us del.icio.us LinkedIn LinkedIn


All items in the PolyU Institutional Repository are protected by copyright, with all rights reserved, unless otherwise indicated.
No item in the PolyU IR may be reproduced for commercial or resale purposes.

 

© Pao Yue-kong Library, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong
Powered by DSpace (Version 1.5.2)  © MIT and HP
Feedback | Privacy Policy Statement | Copyright & Restrictions - Feedback