The aim was to train and evaluate the MNIST dataset of 60000 labeled handwritten digits. This was an example of a Supervised Machine Learning system. The Anaconda platform is used to setup Keras with a TensorFlow backend, using Jupyter as a frontend to run kernels and notebooks. TensorBoard is used to view the scalars and graphs used to run the job.
Tested on:
- CPU: AMD Ryzen 5 2600
- RAM: 32GB (4x8) Dual Channel at 2400MHz
- GPU: NVIDIA GTX 1060 6GB
- Storage: 4TB (4x1TB in RAID0 LVM) with 256GB bcache
- OS: *nix (Ubuntu/Arch) virtualised on Proxmox VE achieved via GPU passthrough.
A Sequential model (stack layers sequentially) is used, implementing a Convolutional Neural Network (CNN) – LeNet – architecture, using Python and the Keras deep learning package. This has been compiled using a Categorical Cross Entropy loss function, accuracy metric function, and the Adam optimizer (a method for stochastic optimization).
Basic model training has been done on the sample set, without using any rate annealing methods or sample synthesis (data generation). The model was fitted in 15 epochs to an accuracy of 99.45% and loss of 1.61% in under 95 seconds. Better models having 99.76% accuracy have also been achieved. A personal attempt leads to an accuracy of 98.12% which has not been submitted due to impracticality in application.
Machine Learning (ML) is a computer science field focused on implementing algorithms capable of drawing predictive insight from static or dynamic data sources using analytic or probabilistic models and using refinement via training and feedback. It makes use of pattern recognition, artificial intelligence learning methods, and statistical data modeling.
Learning is achieved in two primary ways:
- Supervised machine learning, where the desired outcome is known, and an annotated data set or measured values are used to train/fit a model to accurately predict values or labels outside of the training set. This is basically “regression” for values and “classification” for labels.
- Unsupervised learning uses “unlabeled” data and seeks to find inherent partitions or clusters of characteristics present in the data. ML draws from the fields of computer science, statistics, probability, applied mathematics, optimization, information theory, graph and network theory, biology, and neuroscience.
In layman terms, it is curve fitting but much more powerful than the basics learned in college mathematics.
Machine Learning has been around for a long time, with the mathematical and computation foundations laid in the 80s. The resurgence of ML has taken place because technology has been able to reach the required computation demand. Moreover, huge amounts of data have been generated in the past few years due to social networking and Internet usage by a huge number of people. India, USA, EU are the few major areas where ML has been useful to extrapolate the data generated by these countries.
Machine Learning is an area of focus in applied mathematics. ML draws on many mathematical topics, the major ones being, Probability and Statistics, Linear Algebra, Calculus, and Computer Programming. Also useful is Optimization theory, Information Theory, Graph/Network Theory, Numerical Analysis, etc.
Deep learning has revolutionized artificial intelligence by helping us build machines and systems that were only dreamt of in the past. In true essence, Deep Learning is a sub-sect of ML that uses deep, artificial neural networks to tackle the problems of Machine Learning. Deep here signifies the depth of the layers used in the model (as compared to Shallow, where one or two layers are used). A DNN is just a Neural Network with several layers stacked on top of each other (Sequential model) – greater the number of layers, deeper the network.
TensorFlow is an open-sourced library that’s available on GitHub. It is one of the more famous libraries when it comes to dealing with Deep Neural Networks. The primary reason behind the popularity of TensorFlow is the sheer ease of building and deploying applications using TensorFlow. The sample projects provided in the GitHub repository are not only powerful but also written in a beginner-friendly way.
TensorFlow excels at numerical computing, which is critical for deep learning. It provides APIs in most major languages and environments needed for deep learning projects: Python, C, C++, Rust, Haskell, Go, Java, Android, iOS, Mac OS, Windows, Linux, and Raspberry Pi.
Keras is a high-level library that’s built on top of TensorFlow. It provides a SciKit-learn type API (written in Python) for building Neural Networks. It is easier to handle and build ML models than compared to TensorFlow. It runs smoothly on both CPU and GPU, and supports almost all the models of a neural network – fully connected, convolutional, pooling, recurrent, embedding, etc. Furthermore, these models can be combined to build more complex models. Keras, being modular in nature, is incredibly expressive, flexible, and apt for innovative research. It is a completely Python-based framework, which makes it easy to debug and explore.