# REVERSE ENGINEERING AND SYMBOLIC KNOWLEDGE EXTRACTION ON ŁUKASIEWICZ LOGICS USING NEURAL NETWORKS

### Carlos Leandro

#### 2009

#### Abstract

This work describes a methodology that combines logic-based systems and connectionist systems. Our approach uses finite truth-valued Łukasiewicz logic, where we take advantage of fact, presented in (Castro and Trillas, 1998), wherein every connective can be defined by a neuron in an artificial network having, by activation function, the identity truncated to zero and one. This allowed the injection of formulas into a network architecture, and also simplified symbolic rule extraction. Neural networks are trained using the Levenderg-Marquardt algorithm, where we restricted the knowledge dissemination in the network structure, and the generated network is simplified applying the ”Optimal Brain Surgeon” algorithm proposed by B. Hassibi, D. G. Stork and G.J. Wolf. This procedure reduces neural network plasticity without drastically damaging the learning performance, thus making the descriptive power of produced neural networks similar to the descriptive power of Łukasiewicz logic language and simplifying the translation between symbolic and connectionist structures. We used this method in the reverse engineering problem of finding the formula used on the generation of a given truth table. For real data sets the method is particularly useful for attribute selection, on binary classification problems defined using nominal attributes, where each instance has a level of uncertainty associated with it.

#### References

- Doktor der Naturwissenschaften, 2007.
- Andersen, T. and Wilamowski, B. (1995). A modified reington DC, USA, Vol. 1 no. 4, CA, (1995)687-690.
- Castro, J. and Trillas, E. (1998). The logic of neural networks. Mathware and Soft Computing, vol. 5, (1998)23-27.
- Charalambous, C. (1992). Conjugate gradient algorithm for efficient training of artificial neural networks. IEEE Proceedings, Vol. 139 no. 3, (1992)301-310.
- d'Avila Garcez, A. S. (2007). Advances in neural-symbolic learning systems: Modal and temporal reasoning. In B. Hammer and P. Hitzler (ed.), Perspectives of Neural-Symbolic Integration, Studies in Computational Intelligence, Volume 77, Springer, 2007.
- d'Avila Garcez, A. S., Lamb, L. C., and Gabbay, D. M. (2008). Neural-simbolic Cognitive Reasoning. Cognitive Technologies, Springer.
- Dubois, D. and Prade, H. (2000). Fundamentals of fuzzy sets. Kluwer, 2000.
- Eklund, P. and Klawonn, F. (1992). Neural fuzzy logic programming. IEEE translations on neural networks, Vol. 3, No. 5, 1992.
- Fiadeiro, J. and Lopes, A. (1997). Semantics of architectural connectors. TAPSOFT'97 LNCS, v.1214, p.505- 519, Springer-Verlag, 1997.
- Frank, M. (1979). On the simultaneous associativity of f (x; y) and x + y f (x; y). Aequations Math., vol. 19, (1979)194-226.
- Fu, L. (1993). Knowledge-based connectionism from revising domain theories. IEEE Trans. Syst. Man. Cybern, Vol. 23 ,(1993)173-182.
- Gallant, S. (1988). Connectionist expert systems. Commun. ACM, Vol. 31 ,(1988)152-169.
- Gallant, S. (1994). Neural Network Learning and Expert Systems. Cambridge, MA, MIT Press.
- Gerla, B. (2000). Functional representation of many-valued logics based on continuous t-norms. PhD thesis, University of Milano, 2000.
- Hagan, M., Demuth, H., and Beal, M. (1996). Neural Network Design. PWS Publishing Company, Boston.
- Hagan, M. and Menhaj, M. (1999). Training feedforward networks with marquardt algorithm. IEEE Transaction on Neural Networks, vol. 5 no. 6, (1999)989-993.
- Hájek, P. (1995). Fuzzy logic from the logical point of view. In Proceedings SOFSEM'95, LNCS, Springer-Verlag, 1995.
- Hassibi, B., Stork, D., and Wolf, G. (1993). Optimal brain surgeon and general network pruning. IEEE International Conference on Neural Network, vol. 4 no. 5, (2003)740-747.
- Hitzler, P., Hö lldobler, S., and Seda, A. (2004). Logic programs and connectionist networks. Journal of Applied Logic, 2, (2004)245-272.
- Hö lldobler, S. (2000). Challenge problems for the integration of logic and connectionist systems. in: F. Bry, U.Geske and D. Seipel, editors, Proceedings 14. Workshop Logische Programmierung, GMD Report 90, (2000)161-171.
- Hö lldobler, S. and Kalinke, Y. (1994). Towards a new massively parallel computational model for logic programming. in: Proceedings ECAI94 Workshop on Combining symbolic and Connectionist Processing, (1994)68-77.
- Hö lldobler, S., Kalinke, Y., and Stö rr, H. (1999). Approximating the semantics of logic programs by recurrent neural networks. Applied Intelligence 11, (1999)45- 58.
- Jacobs, R. (1988). Increased rates of convergence through learning rate adaptation. Neural Networks, Vol. 1 no. 4, CA, (1988)295-308.
- Komendantskaya, E., Lane, M., and Seda, A. K. (2007). Connectionistic representation of multi-valued logic programs. In B. Hammer and P. Hitzler (ed.), Perspectives of Neural-Symbolic Integration, Studies in Computational Intelligence, Volume 77, Springer, 2007.
- Mehrotra, K., Mohan, C., and Ranka, S. (1997). Elements of Artificial Neural Networks. The MIT Press.
- Miniani, A. and Williams, R. (1990). Acceleration of back-propagation through learning rate and momentum adaptation. Proceedings of International Joint Conference on Neural Networks, San Diego, CA, (1990)676-679.
- Samad, T. (1990). Back-propagation improvements based on heuristic arguments. Proceedings of International Joint Conference on Neural Networks, Washington (1990)565-568.
- Solla, S., Levin, E., and Fleisher, M. (1988). Accelerated learning in layered neural networks. Complex Sustems, 2, (1988)625-639.
- Towell, G. and Shavlik, J. (1993). Extracting refined rules from knowledge-based neural networks. Mach. Learn., Vol. 13 ,(1993)71-101.
- Towell, G. and Shavlik, J. (1994). Knowledge-based artificial neural networks. Artif. Intell., Vol. 70 ,(1994)119- 165.

#### Paper Citation

#### in Harvard Style

Leandro C. (2009). **REVERSE ENGINEERING AND SYMBOLIC KNOWLEDGE EXTRACTION ON ŁUKASIEWICZ LOGICS USING NEURAL NETWORKS** . In *Proceedings of the International Joint Conference on Computational Intelligence - Volume 1: ICFC, (IJCCI 2009)* ISBN 978-989-674-014-6, pages 5-16. DOI: 10.5220/0002283900050016

#### in Bibtex Style

@conference{icfc09,

author={Carlos Leandro},

title={REVERSE ENGINEERING AND SYMBOLIC KNOWLEDGE EXTRACTION ON ŁUKASIEWICZ LOGICS USING NEURAL NETWORKS},

booktitle={Proceedings of the International Joint Conference on Computational Intelligence - Volume 1: ICFC, (IJCCI 2009)},

year={2009},

pages={5-16},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0002283900050016},

isbn={978-989-674-014-6},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the International Joint Conference on Computational Intelligence - Volume 1: ICFC, (IJCCI 2009)

TI - REVERSE ENGINEERING AND SYMBOLIC KNOWLEDGE EXTRACTION ON ŁUKASIEWICZ LOGICS USING NEURAL NETWORKS

SN - 978-989-674-014-6

AU - Leandro C.

PY - 2009

SP - 5

EP - 16

DO - 10.5220/0002283900050016