Comprehensive, accessible introduction to deep learning for engineering tasks through Python programming, low-cost hardware, and freely available software Deep Learning on Embedded Systems is a comprehensive guide to the practical implementation of deep learning for engineering tasks through computers and embedded hardware such as Raspberry Pi and Nvidia Jetson Nano. After an introduction to the field, the book provides fundamental knowledge on deep learning, convolutional and recurrent neural networks, computer vision, and basics of Linux terminal and docker engines. This book shows detailed setup steps of Jetson Nano and Raspberry Pi for utilizing essential frameworks such as PyTorch and OpenCV. GPU configuration and dependency installation procedure for using PyTorch is also discussed allowing newcomers to seamlessly navigate the learning curve. A key challenge of utilizing deep learning on embedded systems is managing limited GPU and memory resources. This book outlines a strategy of training complex models on a desktop computer and transferring them to embedded systems for inference. Also, students and researchers often face difficulties with the varying probabilistic theories and notations found in data science literature. To simplify this, the book mainly focuses on the practical implementation part of deep learning using Python programming, low-cost hardware, and freely available software such as Anaconda and Visual Studio Code. To aid in reader learning, questions and answers are included at the end of most chapters. Written by a highly qualified author, Deep Learning on Embedded Systems includes discussion on: Fundamentals of deep learning, including neurons and layers, activation functions, network architectures, hyperparameter tuning, and convolutional and recurrent neural networks (CNNs & RNNs) PyTorch, OpenCV, and other essential framework setups for deep transfer learning, along with Linux terminal operations, docker engine, docker images, and virtual environments in embedded devices Training models for image classification and object detection with classification, then converting trained PyTorch models to ONNX format for efficient deployment on Jetson Nano and Raspberry Pi Deep Learning on Embedded Systems serves as an excellent introduction to the field for undergraduate engineering students seeking to learn deep learning implementations for their senior capstone or class projects and graduate researchers and educators who wish to implement deep learning in their research.

Table of Contents:
Preface x Acknowledgment xi Biography xii About the Companion Websitexiii 1 Introduction 1 1.1 Machine Learning to Deep Learning 1 1.2 Modern Embedded Systems 2 1.3 Deep Transfer Learning in Embedded System 4 1.4 Deep Learning Frameworks: An Overview 5 1.4.1 PyTorch 5 1.4.2 TensorFlow 6 1.4.3 Other Major Frameworks 6 1.5 Deep Learning and AI: Big Data and the Road Ahead 7 1.5.1 Advances in Computational Hardware 7 1.5.2 Adoptions in Industry and Business 8 1.5.3 Ethical, Legal, and Privacy Concerns 8 1.5.4 Future Challenges 9 2 Fundamentals of Deep Learning 15 2.1 Neural Networks Overview 15 2.2 Basic Concepts and Terminologies 15 2.2.1 Neurons and Layers 15 2.2.2 Role of Activation Functions 17 2.2.3 Learning Types 18 2.3 Training on a Network 18 2.3.1 Forward and Backward Propagation 18 2.3.2 Loss Functions 19 2.4 Gradient Descent Algorithm 19 2.5 Weight Initialization and Regularization 21 2.6 Hyperparameter Tuning 22 2.6.1 Architecture-based Hyperparameters 23 2.6.2 Training-based Hyperparameters 23 2.7 Overview of Common Hyperparameters 23 2.7.1 Kernel Size 23 2.7.2 Learning Rate 23 2.7.3 Weight Decay 24 2.7.4 Dropout 24 2.7.5 Batch, Iteration, and Epochs 24 2.8 Challenges and Best Practices for Efficient Tuning 25 3 Convolutional and Recurrent Neural Network 27 3.1 Introduction 27 3.2 Historical Background 27 3.3 Convolutional Neural Networks 28 3.3.1 CNN Framework 28 3.3.2 Convolutional Operations 28 3.3.3 Padding and Stride 30 3.3.4 Pooling Layers 31 3.3.5 Fully Connected Layers 32 3.3.6 Output Layers 32 3.3.7 Overall Architecture 32 3.4 Recurrent Neural Networks 33 3.4.1 RNN Framework 33 3.4.2 Basic Architectures 33 3.4.3 Backpropagation Through Time 34 3.4.4 Long Short-term Memory Networks 35 3.4.5 Gated Recurrent Units 35 3.4.6 Bidirectional RNNs 35 3.5 Applications 36 3.6 Conclusion 37 4 Deep Learning Using PyTorch 41 4.1 Introduction to PyTorch 41 4.2 Anaconda and PyTorch for Windows System 41 4.2.1 Anaconda Installation and Environment Setup 42 4.2.2 Setting up the PyTorch Framework 44 4.2.2.1 Visual Studio Professional 47 4.2.2.2 CUDA and cuDNN Installation 48 4.3 Other Essential Packages for Deep Learning 50 4.4 Introduction to Tensor 52 4.4.1 Defining a Tensor 52 4.4.2 Tensors in PyTorch 53 4.5 Basic Torch Operations in PyTorch 53 4.6 Gradient Calculation in PyTorch 55 4.7 Exercise Problems 57 5 Introduction to Jetson Nano and Setup 59 5.1 Introduction to Jetson Embedded Devices 59 5.1.1 Jetson Nano 60 5.1.2 Hardware and Power Requirements 61 5.2 Jetpack Installation 62 5.3 Direct Setup 65 5.3.1 Increase Root Partition Size 65 5.3.2 Other Settings 68 5.3.3 Wi-Fi Driver 70 5.4 Configure Visual Studio Code on Jetson 71 5.5 OpenCV and PyTorch in Jetson 75 5.6 Setting up Jetson Inference 76 5.7 OpenCV Library and Test Video Capture Functionality 79 5.8 Conclusion 81 5.9 Exercise Problems 81 6 Linux Terminal Overview 85 6.1 Introduction 85 6.2 Basic Terminal Commands and Syntax 86 6.3 Overview of File System 87 6.4 Navigating Files and Directories 87 6.5 Create, Edit, and Delete 90 6.5.1 Copy and Move Operations 92 6.6 Create and Execute Python Code from Terminal 93 6.7 Common Wildcard Characters 95 6.8 Find, View, and Get Information 96 6.9 Permission and Ownership 99 6.9.1 Permissions Using Octal and Symbolic Notations 100 6.9.2 Update Permission Using Operators 101 6.10 Install and Uninstall Packages Using “sudo” 104 6.11 Conclusion 105 7 Docker Engine Setup 107 7.1 Introduction to Docker Engine 107 7.2 Docker in Embedded Devices 107 7.3 Jetson Inference Docker 108 7.3.1 Download and Test Pre-trained Models 109 7.4 Using Host Files in Docker Environment 110 7.4.1 Create Executable Python 111 7.4.2 Attach Host Directory to Root System and Testing 113 7.5 Building a Docker Image 115 7.5.1 Runtime Image for MediaPipe Ecosystem 115 7.5.2 Install Docker Extension 116 7.5.3 Create Docker File and Build Image 116 7.6 Run Python Through Docker Container 118 7.7 Exercise Problems 119 8 Dataset Development 121 8.1 Introduction and Requirements 121 8.2 Types of Datasets 121 8.3 Manual Dataset Creation 122 8.3.1 Classification and Detection Dataset 122 8.3.2 LabelImg Setup 123 8.4 Automatic Image Collection Using Embedded Device 127 8.5 Automatic Data Labeling 128 8.6 Data Preprocessing and Cleaning 130 8.7 Exercise Problem 131 9 Training Model for Image Classification 133 9.1 Problem Statement 133 9.2 Default Configurations and Libraries 134 9.3 Setup Data Frame Using Annotations 134 9.4 Dataset Class and Methods 136 9.5 Data Loader and Model Configuration 137 9.6 Model Training 140 9.7 Testing and Inference 141 9.8 Fine-tuning 142 9.8.1 Using a Different Model 144 9.8.2 Using Different Optimizers and Schedulers 145 9.9 Application in Embedded System 146 9.10 Exercise Problems 147 10 Object Detection with Classification 149 10.1 Introduction 149 10.2 Import Modules and Libraries 149 10.3 Default Configurations and Random Seeds 151 10.4 Create Data Frame and Process Labels 151 10.5 Training and Validation of Transformers 152 10.6 Dataset Class and Methods 153 10.7 Data Loader and Classification Backbone 153 10.8 Training and Validation Approach 155 10.9 Run Multiple Epochs and Save the Best 156 10.10 Model Inference 157 10.10.1 Preprocessing Functions 158 10.10.2 Inference on Test Images 160 10.11 Multiple Object Detection with Classification 161 10.12 Model Inference for Multiple Object Detection 163 10.13 Conclusion 165 10.14 Exercise Problems 167 11 Deploy Deep Learning Models on Jetson Nano 169 11.1 Introduction 169 11.2 Pre-trained Models 169 11.3 Inference on an Image File 170 11.4 ONNX Model 172 11.4.1 Convert PyTorch Model to ONNX 172 11.5 Inference on Live Video Stream 173 11.6 Conclusion 175 11.7 Exercise Problem 176 12 Trained PyTorch Model: From Desktop PC to Jetson Nano 177 12.1 Introduction 177 12.2 Model Training on a PC 177 12.3 ONNX Model Inference 177 12.3.1 PyTorch’s.pt to ONNX Conversion 177 12.3.2 Image Classification Using ONNX 179 12.3.3 PyTorch’s.pth to ONNX Conversion 181 12.3.4 Object Detection Using ONNX 181 12.4 Conclusion 186 12.5 Exercise Problems 186 13 Setting up Raspberry Pi 187 13.1 Introduction to Raspberry Pi 187 13.2 Hardware and Power Requirements 188 13.2.1 Direct Setup 190 13.3 Operating System Setup 190 13.3.1 Flashing the OS Image 191 13.3.2 Setting up OS 194 13.4 Create Virtual Environment 197 13.5 PyTorch and OpenCV Installation 198 13.6 Other Essential Packages 201 13.7 Conclusion 206 13.8 Exercise Problem 207 14 Deploy Deep Learning Models on Raspberry Pi 209 14.1 Introduction 209 14.2 Face Detection and Recognition in Video Feeds 209 14.2.1 Face Detection Using HOG 209 14.2.2 Face Recognition Using Transfer Learning 211 14.3 Real-time Object Detection 214 14.3.1 Object Detection Using YoloV3 214 14.4 Real-time Classification 217 14.4.1 Classification Using PyTorch’s Quantized Model 217 14.5 Real-time Segmentation 219 14.5.1 Segmentation Using k-means Clustering 219 14.6 Exercise Problems 221 15 Trained PyTorch Model: From Desktop PC to Raspberry Pi 225 15.1 Introduction 225 15.2 Model Training on a Desktop PC 225 15.3 PyTorch’s.pth to ONNX 225 15.3.1 ONNX Runtime 225 15.3.2 Model Conversion in Raspberry Pi 5 226 15.4 ONNX Model Inference on Raspberry Pi 5 228 15.4.1 Import Libraries and Define Configurations 228 15.4.2 Image Preprocessing 228 15.4.3 Model Inference with Bounding Boxes 229 15.5 Conclusion 231 15.6 Exercise Problems 232 Index 235