---
title: "Neural Networks - MNIST - Densily connected NN"
subtitle: "TMA4268 Statistical Learning V2018"
author: "Thiago G. Martins, Department of Mathematical Sciences, NTNU"
date: "Spring 2018"
output: #3rd letter intentation hierarchy
#  beamer_presentation:
###    incremental: true # or >* for one at a time
#  slidy_presentation:
#    font_adjustment: +1  
   prettydoc::html_pretty:
    theme: cayman
    highlight: github
    toc: true
    toc_depth: 4
   # prettydoc::html_pretty:
   #  theme: architect
   #  highlight: github
#  pdf_document:
  #  toc: true
  #  toc_depth: 2

---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

```{r, include=FALSE}
library(keras)
```

## MNIST dataset

The objective here is to classify the digit contained in a image.

### Preparing the data

We can download the MNIST dataset using the `dataset_mnist` function available in the `keras` package.

```{r, message=FALSE, warning=FALSE}
# Downloading raw data
mnist <- dataset_mnist()
```

We then need to extract the training and test data from the `mnist` object.

```{r, message=FALSE, warning=FALSE}
# Training data
train_images <- mnist$train$x
train_labels <- mnist$train$y

# Test data
test_images <- mnist$test$x
test_labels <- mnist$test$y
```

The code below shows that the `train_images` is a tensor with 3 axis, `(samples, height, width)`. In this case, the channels axis was not necessary because the grey-scale images has only one channel. 

```{r, message=FALSE, warning=FALSE}
str(train_images)
```

We can use slice operations to inspect specific images. Below we can see the content of the first training image.

```{r, message=FALSE, warning=FALSE}
train_images[1,,]
```

A plot can be made using the `as.raster` function.

```{r, message=FALSE, warning=FALSE}
# Plotting a digit
digit <- train_images[5,,]
plot(as.raster(digit, max = 255))
```

The train labels is a vector holding the digit that the respective image is representing, going from zero to nine.

```{r, message=FALSE, warning=FALSE}
table(train_labels)
```

### Defining the model

In this case we are using a `layer_dense` which expects an input tensor of rank equal to two `(sample, features)` where each `sample` should contain `28*28=784` pixels. 

```{r, message=FALSE, warning=FALSE}
network <- keras_model_sequential() %>%
  layer_dense(units = 512, activation = "relu", input_shape = c(28*28)) %>%
  layer_dense(units = 10, activation = "softmax")
```

### Compiling the model

```{r, message=FALSE, warning=FALSE}
network %>% compile(
  optimizer = "rmsprop",
  loss = "categorical_crossentropy",
  metrics = c("accuracy")
)
```

### Data pre-processing

Before training, we’ll preprocess the data by reshaping it into the shape the network expects and scaling it so that all values are in the [0, 1] interval. 

Previously, our training images, for instance, were stored in an array of shape (60000, 28, 28) of type integer with values in the [0, 255] interval. We transform it into a double array of shape (60000, 28 * 28) with values between 0 and 1.

```{r, message=FALSE, warning=FALSE}
# Processing the data
train_images <- array_reshape(train_images, c(60000, 28 * 28))
train_images <- train_images / 255

train_labels <- to_categorical(train_labels)

test_images <- array_reshape(test_images, c(10000, 28 * 28))
test_images <- test_images / 255

test_labels <- to_categorical(test_labels)
```

### Fitting the model

```{r, message=FALSE, warning=FALSE}
network %>% fit(train_images, train_labels, epochs = 5, batch_size = 128)
```

### Recommended exercise 1

Perform model validation (plot loss and accuracy metric) and model assessment (accuracy in the test dataset) for the model defined here.