HBP: A NOVEL TECHNIQUE FOR DYNAMIC OPTIMIZATION OF THE FEED-FORWARD NEURAL NETWORK CONFIGURATION

Allan K. Y. Wong, Wilfred W.K. Lin, Tharam S. Dillon

2004

Abstract

The novel Hessian-based pruning (HBP) technique to optimize the feed-forward (FF) neural network (NN) configuration in a dynamic manner is proposed. It is then used to optimize the extant NNC (Neural Network Controller) as the verification exercise. The NNC is designed for dynamic buffer tuning to eliminate buffer overflow at the user/server level. The HBP optimization process is also dynamic and operates as a renewal process within the service life expectancy of the target FF neural network. Every optimization renewal cycle works with the original NN configuration. In the cycle all the insignificant NN connections are marked and then skipped in the subsequent operation before the next optimization cycle starts. The marking and skipping operations together characterize the dynamic virtual pruning nature of the HBP. The interim optimized NN configuration produced by every HBP cycle is different, as the response to the current system dynamics. The verification results with the NNC indicate that the HBP technique is indeed effective because all the interim optimized/pruned NNC versions incessantly and consistently yield the same convergence precision to the original NNC predecessor, and with a shorter control cycle time.

References

  1. B. Braden et al., Recommendation on Queue Management and Congestion Avoidance in the Internet, RFC2309, April 1998
  2. Allan K.Y. Wong, Wilfred W.K. Lin, May T.W. Ip, and Tharam S. Dillon, Genetic Algorithm and PID Control Together for Dynamic Anticipative Marginal Buffer Management: An Effective Approach to Enhance Dependability and Performance for Distributed Mobile Object-Based Real-time Computing over the Internet, Journal of Parallel and Distributed Computing (JPDC), vol. 62, Sept. 2002
  3. A. R. Gallant and H. White, On Learning the Derivatives of an Unknown Mapping and Its Derivatives Using Multiplayer Feedforward Networks, Neural Networks, vol. 5, 1992
  4. M. Hagan et al, Neural Network Design, PWS Publishing Company, 1996
  5. T. Karagiannis, M. Faloutsos, M. Molle, A User-friendly Self-similarity Analysis Tool, ACM SIGCOMM Computer Communication Review, 33(3), July 2003, 81-93
  6. Wilfred W.K. Lin, An Adaptive IEPM (Internet End-toEnd Performance Measurement) Based Approach to Enhance Fault Tolerance and Performance in Object Based Distributed Computing over a Sizeable Network (Exemplified by the Internet), MPhil Thesis, Department of Computing, Hong Kong PolyU, 2002
  7. Wilfred W.K. Lin, Allan K.Y. Wong and Tharam S. Dillon, HBM: A Suitable Neural Network Pruning Technique to Optimize the Execution Time of the Novel Neural Network Controller (NNC) that Eliminates Buffer Overflow, Proc. of Parallel and Distributed Processing Techniques and Applications (PDPTA), vol. 2, Las Vegas USA, June 2003
  8. May T.W. Ip, Wilfred W.K. Lin, Allan K.Y. Wong, Tharam S. Dillon and Dian Hui Wang, An Adaptive Buffer Management Algorithm for Enhancing Dependability and Performance in Mobile-ObjectBased Real-time Computing, Proc. of the IEEE ISORC'2001, Magdenburg, Germany, May 2001
Download


Paper Citation


in Harvard Style

K. Y. Wong A., W.K. Lin W. and S. Dillon T. (2004). HBP: A NOVEL TECHNIQUE FOR DYNAMIC OPTIMIZATION OF THE FEED-FORWARD NEURAL NETWORK CONFIGURATION . In Proceedings of the First International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO, ISBN 972-8865-12-0, pages 346-349. DOI: 10.5220/0001126103460349


in Bibtex Style

@conference{icinco04,
author={Allan K. Y. Wong and Wilfred W.K. Lin and Tharam S. Dillon},
title={HBP: A NOVEL TECHNIQUE FOR DYNAMIC OPTIMIZATION OF THE FEED-FORWARD NEURAL NETWORK CONFIGURATION},
booktitle={Proceedings of the First International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,},
year={2004},
pages={346-349},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001126103460349},
isbn={972-8865-12-0},
}


in EndNote Style

TY - CONF
JO - Proceedings of the First International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,
TI - HBP: A NOVEL TECHNIQUE FOR DYNAMIC OPTIMIZATION OF THE FEED-FORWARD NEURAL NETWORK CONFIGURATION
SN - 972-8865-12-0
AU - K. Y. Wong A.
AU - W.K. Lin W.
AU - S. Dillon T.
PY - 2004
SP - 346
EP - 349
DO - 10.5220/0001126103460349