A simple single layer convolution neural network on MNIST database

DeepMind’s founder says to build better machine learning brain, we need to learn from neurosciences. “In both biological and artificial systems, successive non-linear computations transform raw visual input into an increasingly complex set of features, permitting object recognition that is invariant to transformations of pose, illumination, or scale.”

MNIST database

MNIST database, (modified national institute of standards of technology database) is a collection of handwritten 0-9 digit images. It contains training, test and validation dataset, and is a commonly used dataset to train and validate varied image processing and machine learning algorithms.

Introduction to TensorFlow

TensorFlow has been a popular tool in machine learning since Google decided to open source the library. The software is using graph based computation. Unlike regular computations in Python and R where analyses are carried out sequentially, TensorFlow first constructs a graph based on placeholders and variables then do the computations in parallel across CPUs or GPUs with data feeds. Google also creates their own processing unit called TPU which is available to on their cloud platform.

Machine learning - code a neural network from scratch

Overview

In this post, we are going to implement a neural network from scratch in Python, and use it to classify a moon dataset.

Machine Learning in Python - Logistic Regression

Logistic regression

Logistic regression can be used to estimate the probability of response based on one or more variables or features. It can be used to predict categorical response with multiple levels, but the post here focuses on binary response which we can call it binary logistic models. It will only take two kinds of responses, such as fail/pass, 0 or 1.

Matplotlib, a Python plotting library

As matplotlib becomes so powerful, Python can pretty much replace R for a lot of data analysis and visualization. In fact, it is easier to do plot with Matplotlib than R.

Scientific computing with Python Numpy

In the last few years, I have been working with Python and Matlab initially, but switched to R due to the workload to do large scale data analysis. During 2007-2009, I re-visited Python with Rpy and Ppy2 packages which is an interface to R running embedded in a Python process. The integration is not so smooth.