There are several well-known problems with conventional artificial neural networks (ANN), such as difficulties with selecting the structure of the network, and problems with forgetting previously-learned knowledge after further training. Constructive neural network algorithms attempt to solve these problems, but in turn have problems of their own. The Evolving Connectionist System (ECoS) is a class of open architecture artificial neural networks that are similar in the way in which neurons are added to their structures, and in the way in which their connection weights are modified. The ECoS algorithm is intended to address the problems with constructive neural networks.
Several problems with ECoS are identified and discussed in this thesis. These problems are: the lack of comparison of ECoS with constructive neural networks; the excessive complexity of the Evolving Fuzzy Neural Network (EFuNN), which is the seminal ECoS network: the lack of a testable formalisation of ECoS; the dependence on fuzzy logic elements embedded within the network for fuzzy rule extraction; and the lack of methods for optimising ECoS networks. The research in this thesis addresses these problems.
The overall theme of the research can be summarised as the characterisation, simplification, formalisation, explanation and optimisation of ECoS. Characterisation in this thesis means the comparison of ECoS with existing constructive ANN. Simplification means reducing the network to a minimalist implementation. Formalisation means the creation of a testable predictive model of ECoS training. Explanation means explaining ECoS networks via the extraction of fuzzy rules. Finally, optimisation means creating ECoS networks that have a minimum number of neurons with maximum accuracy.
Each of these themes is approached in ways that build upon, and are complementary to, the basic ECoS network and ECoS training algorithm. The basic ECoS structure and algorithm is left unchanged, and the problems are addressed by extending that structure, rather than altering it as has been done in other work on EcoS. The principal contributions of this thesis are: a qualitative comparison of ECoS to constructive neural network algorithms; a proposed simplified version of EFuNN called SECoS; an experimentally tested formalisation of ECoS: novel algorithms for explicating SECoS via the extraction of fuzzy rules; and several novel algorithms for the optimisation of ECoS networks.
The formalisation of ECoS and the proposed algorithms are evaluated on data from a set of standard benchmarking problems. Further experiments are performed with a data set with real-world applications, namely the recognition of isolated New Zealand English phonemes. The analyses of the experimental results show that the proposed algorithms are effective across both the benchmark data sets and the case study data set.
