Autonomous Learning Systems is the result of over a decade of focused research and studies in this emerging area which spans a number of well-known and well-established disciplines that include machine learning, system identification, data mining, fuzzy logic, neural networks, neuro-fuzzy systems, control theory and pattern recognition. The evolution of these systems has been both industry-driven with an increasing demand from sectors such as defence and security, aerospace and advanced process industries, bio-medicine and intelligent transportation, as well as research-driven – there is a strong trend of innovation of all of the above well-established research disciplines that is linked to their on-line and real-time application; their adaptability and flexibility. Providing an introduction to the key technologies, detailed technical explanations of the methodology, and an illustration of the practical relevance of the approach with a wide range of applications, this book addresses the challenges of autonomous learning systems with a systematic approach that lays the foundations for a fast growing area of research that will underpin a range of technological applications vital to both industry and society.  Key features:  Presents the subject systematically from explaining the fundamentals to illustrating the proposed approach with numerous applications. Covers a wide range of applications in fields including unmanned vehicles/robotics, oil refineries, chemical industry, evolving user behaviour and activity recognition. Reviews traditional fields including clustering, classification, control, fault detection and anomaly detection, filtering and estimation through the prism of evolving and autonomously learning mechanisms. Accompanied by a website hosting additional material, including the software toolbox and lecture notes. Autonomous Learning Systems provides a ‘one-stop shop’ on the subject for academics, students, researchers and practicing engineers. It is also a valuable reference for Government agencies and software developers.

Table of Contents:
Forewords xi Preface xix About the Author xxiii 1 Introduction 1 1.1 Autonomous Systems 3 1.2 The Role of Machine Learning in Autonomous Systems 4 1.3 System Identification – an Abstract Model of the Real World 6 1.4 Online versus Offline Identification 9 1.5 Adaptive and Evolving Systems 10 1.6 Evolving or Evolutionary Systems 11 1.7 Supervised versus Unsupervised Learning 13 1.8 Structure of the Book 14 PART I FUNDAMENTALS 2 Fundamentals of Probability Theory 19 2.1 Randomness and Determinism 20 2.2 Frequentistic versus Belief-Based Approach 22 2.3 Probability Densities and Moments 23 2.4 Density Estimation – Kernel-Based Approach 26 2.5 Recursive Density Estimation (RDE) 28 2.6 Detecting Novelties/Anomalies/Outliers using RDE 32 2.7 Conclusions 36 3 Fundamentals of Machine Learning and Pattern Recognition 37 3.1 Preprocessing 37 3.2 Clustering 42 3.3 Classification 56 3.4 Conclusions 58 4 Fundamentals of Fuzzy Systems Theory 61 4.1 Fuzzy Sets 61 4.2 Fuzzy Systems, Fuzzy Rules 64 4.3 Fuzzy Systems with Nonparametric Antecedents (AnYa) 69 4.4 FRB (Offline) Classifiers 73 4.5 Neurofuzzy Systems 75 4.6 State Space Perspective 79 4.7 Conclusions 81 PART II METHODOLOGY OF AUTONOMOUS LEARNING SYSTEMS 5 Evolving System Structure from Streaming Data 85 5.1 Defining System Structure Based on Prior Knowledge 85 5.2 Data Space Partitioning 86 5.3 Normalisation and Standardisation of Streaming Data in an Evolving Environment 96 5.4 Autonomous Monitoring of the Structure Quality 98 5.5 Short- and Long-Term Focal Points and Submodels 104 5.6 Simplification and Interpretability Issues 105 5.7 Conclusions 107 6 Autonomous Learning Parameters of the Local Submodels 109 6.1 Learning Parameters of Local Submodels 110 6.2 Global versus Local Learning 111 6.3 Evolving Systems Structure Recursively 113 6.4 Learning Modes 116 6.5 Robustness to Outliers in Autonomous Learning 118 6.6 Conclusions 118 7 Autonomous Predictors, Estimators, Filters, Inferential Sensors 121 7.1 Predictors, Estimators, Filters – Problem Formulation 121 7.2 Nonlinear Regression 123 7.3 Time Series 124 7.4 Autonomous Learning Sensors 125 7.5 Conclusions 131 8 Autonomous Learning Classifiers 133 8.1 Classifying Data Streams 133 8.2 Why Adapt the Classifier Structure? 134 8.3 Architecture of Autonomous Classifiers of the Family AutoClassify 135 8.4 Learning AutoClassify from Streaming Data 139 8.5 Analysis of AutoClassify 140 8.6 Conclusions 140 9 Autonomous Learning Controllers 143 9.1 Indirect Adaptive Control Scheme 144 9.2 Evolving Inverse Plant Model from Online Streaming Data 145 9.3 Evolving Fuzzy Controller Structure from Online Streaming Data 147 9.4 Examples of Using AutoControl 148 9.5 Conclusions 153 10 Collaborative Autonomous Learning Systems 155 10.1 Distributed Intelligence Scenarios 155 10.2 Autonomous Collaborative Learning 157 10.3 Collaborative Autonomous Clustering, AutoCluster by a Team of ALSs 158 10.4 Collaborative Autonomous Predictors, Estimators, Filters and AutoSense by a Team of ALSs 159 10.5 Collaborative Autonomous Classifiers AutoClassify by a Team of ALSs 160 10.6 Superposition of Local Submodels 161 10.7 Conclusions 161 PART III APPLICATIONS OF ALS 11 Autonomous Learning Sensors for Chemical and Petrochemical Industries 165 11.1 Case Study 1: Quality of the Products in an Oil Refinery 165 11.2 Case Study 2: Polypropylene Manufacturing 172 11.3 Conclusions 178 12 Autonomous Learning Systems in Mobile Robotics 179 12.1 The Mobile Robot Pioneer 3DX 179 12.2 Autonomous Classifier for Landmark Recognition 180 12.3 Autonomous Leader Follower 193 12.4 Results Analysis 196 13 Autonomous Novelty Detection and Object Tracking in Video Streams 197 13.1 Problem Definition 197 13.2 Background Subtraction and KDE for Detecting Visual Novelties 198 13.3 Detecting Visual Novelties with the RDE Method 203 13.4 Object Identification in Image Frames Using RDE 204 13.5 Real-time Tracking in Video Streams Using ALS 206 13.6 Conclusions 209 14 Modelling Evolving User Behaviour with ALS 211 14.1 User Behaviour as an Evolving Phenomenon 211 14.2 Designing the User Behaviour Profile 212 14.3 Applying AutoClassify0 for Modelling Evolving User Behaviour 215 14.4 Case Studies 216 14.5 Conclusions 221 15 Epilogue 223 15.1 Conclusions 223 15.2 Open Problems 227 15.3 Future Directions 227 APPENDICES Appendix A Mathematical Foundations 231 Appendix B Pseudocode of the Basic Algorithms 235 References 245 Glossary 259 Index 263