---
subtitle: "TMA4268 Statistical Learning V2019. Module 1: INTRODUCTION TO STATISTICAL LEARNING"
title: "Visualizations in R"
author: "Mette Langaas and Julia Debik, Department of Mathematical Sciences, NTNU"
date: "week 2, 2019"
output:
  html_document:
    toc: true
    toc_float: true
    toc_depth: 2
  pdf_document:
    toc: true
    toc_depth: 2
    keep_tex: yes
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE,tidy=TRUE,message=FALSE,warning=FALSE,results="hold")
showsol<-FALSE
```

# Introduction

For each of the plots (scatter plot, histogram, boxplot, area chart, heat map, correlogram) _explain what you see (including what is on the x- and y-axis) and try to transform what you see into insight about the data_. All except the correlogram use `ggplot2` for plotting. If you want to read more about the idea behind `ggplot2` (grammar of graphics) Chapter 3 of [R for Data Science]() is a good read.

## Packages needed

```{r,eval=FALSE}
install.packages("car")
install.packages("faraway")
install.packages("ggplot2")
install.packages("GGally")
install.packages("reshape")
install.packages("corrplot")
install.packages("corrgram")
```

## Data sets

Three different data sets are used - read descriptions in R:

* `SLID`: `?car::SLID`
* `mtcars`: `?datasets::mtcars`
* `ozone`: `?faraway::ozone`

# Scatter Plot

```{r scatter, warning=FALSE, message=FALSE}
library(car)
library(ggplot2)
SLID = na.omit(SLID)
ggplot(SLID, aes(education, wages))+geom_point()+labs(title="Scatterplot")+theme_minimal()
```

```{r, echo=FALSE, results='asis', eval=showsol}
cat("**Solution:** 
The scatterplot shows that the people with the largest wages often are the people with the longest education. The plot also indicates that the variance increases as a function of education, i.e the expected wage vary less for a random person with 0-5 years of education compared to a person with 20 years of education.
")
```

```{r scatter2, warning=FALSE, message=FALSE}
ggplot(SLID, aes(education, wages)) + geom_point(aes(color = language)) + 
  scale_x_continuous("Education")+
  scale_y_continuous("Wages")+ 
  theme_bw() + labs(title="Scatterplot") + facet_wrap( ~ language)+theme_minimal()
```

```{r, echo=FALSE, results='asis', eval=showsol}
cat("**Solution:** 
From this plot we see that there are more english speaking people in the dataset. In general, the english speaking people have large education (relatively few people with education < 8 years). Among the people who speak other langauges than french and english, there is a larger amount of people with low education.
")
```


# Histogram
```{r hist}
ggplot(SLID, aes(wages))+geom_histogram(binwidth=2)+labs(title="Histogram")+theme_minimal()
```

```{r, echo=FALSE, results='asis', eval=showsol}
cat("**Solution:** 
Shows the distributon of wages in the dataset.
")
```


# Box-plot
```{r boxplot}
ggplot(SLID, aes(language,wages ))+geom_boxplot(fill="skyblue")+labs(title="Box Plot")+theme_minimal()
```

```{r, echo=FALSE, results='asis', eval=showsol}
cat("**Solution:** 
The median wage is similar for people speaking english, french and other languages. The $25$ and $75 $ percentiles are also similar for the three boxplots. However, there are more outliers among the english speaking people: There are many people with wages that are larger than the upper $95
$ percentile.
")
```

# All pairs and different plots

```{r ggpairs,message=FALSE}
library(GGally)
ggpairs(SLID)+theme_minimal()
```

```{r, echo=FALSE, results='asis', eval=showsol}
cat("**Solution:** 
This plot gives us an overview of the dataset:

* Correlation between different variables, e.g cor(age,wage)=0.36. 
* Distribution of wages in the dataset (upper left), education (row 1, column 2) and age (row 3, column 3).
* Boxplots for different pairs of variables, e.g boxplots for wage as a function of gender (row 1, column 4). We see that males have a median wage that is larger than for the females in the dataset.
* Histograms showing the distribution of the different covariates, i.e row 4, column 4 shows that there are approximately equally many males and females in the dataset.
* Scatterplots indicating correlation between variables, e.g scatterplot between wages and education in row 2, column 1.
")
```


# Area chart
```{r,message=FALSE}
ages = cut(SLID$age, breaks=3)
SLID2 = cbind(SLID, ages)
ggplot(SLID, aes(x=wages, fill=ages))+geom_area(stat="bin")+theme_minimal()
```


```{r, echo=FALSE, results='asis', eval=showsol}
cat("**Solution:** 
Compares the distribution of wages for different age groups. Young people (red) tend to have lower wages than older people between 31.7 and 51.3 years (green).
")
```

# Heat map

```{r heat, message=FALSE, warning=FALSE}
library(reshape)
head(mtcars) 
carsdf = data.frame(scale(mtcars))
carsdf$model = rownames(mtcars)
cars_melt = melt(carsdf, id.vars="model")

ggplot(cars_melt, aes(x =variable, y = model))+geom_raster(aes(fill=value))+labs(title="Heat Map") + scale_fill_continuous(name="Value")+theme_minimal()

```

```{r, echo=FALSE, results='asis', eval=showsol}
cat("**Solution:** 
Visualization of the data. Shows the values of the different covariates (-1 to 3) for the different car models. 
")
```

# Correlogram

The `ozone` data:

```{r corrplot, message=FALSE, warning=FALSE}
library(faraway)
data(ozone)
library(corrplot)
ozonecorr = cor(ozone)
corrplot(ozonecorr)

library(corrgram)
corrgram(ozone, upper.panel=panel.conf)
```

```{r, echo=FALSE, results='asis', eval=showsol}
cat("**Solution:** 
Visualizes the correlation between different variables in the dataset. We can for example observe a large, negative correlation between $ibh$ and $ibt$ and a large, positive correlation between $ibt$ and $O3$.
")
```
