\documentclass[]{article}
\usepackage{lmodern}
\usepackage{amssymb,amsmath}
\usepackage{ifxetex,ifluatex}
\usepackage{fixltx2e} % provides \textsubscript
\ifnum 0\ifxetex 1\fi\ifluatex 1\fi=0 % if pdftex
  \usepackage[T1]{fontenc}
  \usepackage[utf8]{inputenc}
\else % if luatex or xelatex
  \ifxetex
    \usepackage{mathspec}
  \else
    \usepackage{fontspec}
  \fi
  \defaultfontfeatures{Ligatures=TeX,Scale=MatchLowercase}
\fi
% use upquote if available, for straight quotes in verbatim environments
\IfFileExists{upquote.sty}{\usepackage{upquote}}{}
% use microtype if available
\IfFileExists{microtype.sty}{%
\usepackage{microtype}
\UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts
}{}
\usepackage[margin=1in]{geometry}
\usepackage{hyperref}
\hypersetup{unicode=true,
            pdftitle={Compulsory exercise 2},
            pdfauthor={Michail Spitieris, Andreas Strand and Mette Langaas},
            pdfborder={0 0 0},
            breaklinks=true}
\urlstyle{same}  % don't use monospace font for urls
\usepackage{color}
\usepackage{fancyvrb}
\newcommand{\VerbBar}{|}
\newcommand{\VERB}{\Verb[commandchars=\\\{\}]}
\DefineVerbatimEnvironment{Highlighting}{Verbatim}{commandchars=\\\{\}}
% Add ',fontsize=\small' for more characters per line
\usepackage{framed}
\definecolor{shadecolor}{RGB}{248,248,248}
\newenvironment{Shaded}{\begin{snugshade}}{\end{snugshade}}
\newcommand{\KeywordTok}[1]{\textcolor[rgb]{0.13,0.29,0.53}{\textbf{#1}}}
\newcommand{\DataTypeTok}[1]{\textcolor[rgb]{0.13,0.29,0.53}{#1}}
\newcommand{\DecValTok}[1]{\textcolor[rgb]{0.00,0.00,0.81}{#1}}
\newcommand{\BaseNTok}[1]{\textcolor[rgb]{0.00,0.00,0.81}{#1}}
\newcommand{\FloatTok}[1]{\textcolor[rgb]{0.00,0.00,0.81}{#1}}
\newcommand{\ConstantTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}}
\newcommand{\CharTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}}
\newcommand{\SpecialCharTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}}
\newcommand{\StringTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}}
\newcommand{\VerbatimStringTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}}
\newcommand{\SpecialStringTok}[1]{\textcolor[rgb]{0.31,0.60,0.02}{#1}}
\newcommand{\ImportTok}[1]{#1}
\newcommand{\CommentTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textit{#1}}}
\newcommand{\DocumentationTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\AnnotationTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\CommentVarTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\OtherTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{#1}}
\newcommand{\FunctionTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}}
\newcommand{\VariableTok}[1]{\textcolor[rgb]{0.00,0.00,0.00}{#1}}
\newcommand{\ControlFlowTok}[1]{\textcolor[rgb]{0.13,0.29,0.53}{\textbf{#1}}}
\newcommand{\OperatorTok}[1]{\textcolor[rgb]{0.81,0.36,0.00}{\textbf{#1}}}
\newcommand{\BuiltInTok}[1]{#1}
\newcommand{\ExtensionTok}[1]{#1}
\newcommand{\PreprocessorTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textit{#1}}}
\newcommand{\AttributeTok}[1]{\textcolor[rgb]{0.77,0.63,0.00}{#1}}
\newcommand{\RegionMarkerTok}[1]{#1}
\newcommand{\InformationTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\WarningTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{\textbf{\textit{#1}}}}
\newcommand{\AlertTok}[1]{\textcolor[rgb]{0.94,0.16,0.16}{#1}}
\newcommand{\ErrorTok}[1]{\textcolor[rgb]{0.64,0.00,0.00}{\textbf{#1}}}
\newcommand{\NormalTok}[1]{#1}
\usepackage{graphicx,grffile}
\makeatletter
\def\maxwidth{\ifdim\Gin@nat@width>\linewidth\linewidth\else\Gin@nat@width\fi}
\def\maxheight{\ifdim\Gin@nat@height>\textheight\textheight\else\Gin@nat@height\fi}
\makeatother
% Scale images if necessary, so that they will not overflow the page
% margins by default, and it is still possible to overwrite the defaults
% using explicit options in \includegraphics[width, height, ...]{}
\setkeys{Gin}{width=\maxwidth,height=\maxheight,keepaspectratio}
\IfFileExists{parskip.sty}{%
\usepackage{parskip}
}{% else
\setlength{\parindent}{0pt}
\setlength{\parskip}{6pt plus 2pt minus 1pt}
}
\setlength{\emergencystretch}{3em}  % prevent overfull lines
\providecommand{\tightlist}{%
  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
\setcounter{secnumdepth}{0}
% Redefines (sub)paragraphs to behave more like sections
\ifx\paragraph\undefined\else
\let\oldparagraph\paragraph
\renewcommand{\paragraph}[1]{\oldparagraph{#1}\mbox{}}
\fi
\ifx\subparagraph\undefined\else
\let\oldsubparagraph\subparagraph
\renewcommand{\subparagraph}[1]{\oldsubparagraph{#1}\mbox{}}
\fi

%%% Use protect on footnotes to avoid problems with footnotes in titles
\let\rmarkdownfootnote\footnote%
\def\footnote{\protect\rmarkdownfootnote}

%%% Change title format to be more compact
\usepackage{titling}

% Create subtitle command for use in maketitle
\newcommand{\subtitle}[1]{
  \posttitle{
    \begin{center}\large#1\end{center}
    }
}

\setlength{\droptitle}{-2em}

  \title{Compulsory exercise 2}
    \pretitle{\vspace{\droptitle}\centering\huge}
  \posttitle{\par}
  \subtitle{TMA4268 Statistical Learning V2019}
  \author{Michail Spitieris, Andreas Strand and Mette Langaas}
    \preauthor{\centering\large\emph}
  \postauthor{\par}
      \predate{\centering\large\emph}
  \postdate{\par}
    \date{Deadline: April 12 at 16 on Bb}


\begin{document}
\maketitle

{
\setcounter{tocdepth}{2}
\tableofcontents
}
(Last changes: 13.03 first version)

\section{Introduction}\label{introduction}

Maximal score is 15 points. You need a score of 6/15 for the exercise to
be approved. Your score will make up 15\% points of your final grade.

\subsection{Supervision}\label{supervision}

All supervision is in Smia.

\begin{itemize}
\tightlist
\item
  Thursdays 16.15-18, in addition
\item
  Monday April 1 at 8.15-10
\item
  Thursday April 4 at 14.15-16
\end{itemize}

\subsection{Practical issues}\label{practical-issues}

\begin{itemize}
\tightlist
\item
  Maximal group size is 3 - join a group (self enroll) before handing in
  on Bb.
\item
  Remember to write your names and group number on top of your
  submission.
\item
  The exercise should be handed in as one R Markdown file and a
  pdf-compiled version of the R Markdown file (if you are not able to
  produce a pdf-file directly please make an html-file, open it in an
  internet browser and save as pdf - no, not landscape - but portrait
  please). We will read the pdf-file and use the Rmd file in case we
  need to check details in your submission.
\item
  In \emph{all} the R-chunks please use both \texttt{echo=TRUE} and
  \texttt{eval=TRUE} to make it simpler for us to read and grade.
\item
  Please do \emph{not include all the text from this file} (that you are
  reading now) - we want your R code, plots and written solutions - use
  the template:
  \url{https://www.math.ntnu.no/emner/TMA4268/2019v/CompEx2mal.Rmd}. NB:
  remember that we can't write full latex in R Markdown, and using
  \texttt{\$} to start and stop math mode, or \texttt{\$\$} to start and
  end equation mode should be sufficient for what you need.
\item
  Please save us time and NOT submit word or zip - or only Rmd - that
  only result in us contacting you to ask to upload correct file
  formats.
\end{itemize}

\subsection{R packages}\label{r-packages}

You need to install the following packages in R to run the code in this
file.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"knitr"}\NormalTok{) }\CommentTok{#probably already installed}
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"rmarkdown"}\NormalTok{) }\CommentTok{#probably already installed}
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"bestglm"}\NormalTok{)}\CommentTok{# for subset selection with categorical variables}
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"glmnet"}\NormalTok{)}\CommentTok{# for lasso}
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"tree"}\NormalTok{) }\CommentTok{#tree}
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"randomForest"}\NormalTok{) }\CommentTok{#for random forest}
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"ElemStatLearn"}\NormalTok{) }\CommentTok{#dataset in Problem 2}
\NormalTok{BiocManager}\OperatorTok{::}\KeywordTok{install}\NormalTok{(}\KeywordTok{c}\NormalTok{(}\StringTok{"pheatmap"}\NormalTok{)) }\CommentTok{#heatmap in Problem 2}
\end{Highlighting}
\end{Shaded}

The following are packages that might be useful.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"ggplot2"}\NormalTok{)}
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"GGally"}\NormalTok{) }\CommentTok{# for ggpairs}
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"caret"}\NormalTok{) }\CommentTok{#for confusion matrices}
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"pROC"}\NormalTok{) }\CommentTok{#for ROC curves}
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"e1071"}\NormalTok{) }\CommentTok{# for support vector machines}
\KeywordTok{install.packages}\NormalTok{(}\StringTok{"nnet"}\NormalTok{) }\CommentTok{# for feed forward neural networks}
\end{Highlighting}
\end{Shaded}

\section{Problem 1: Regression {[}6
points{]}}\label{problem-1-regression-6-points}

In this problem we will work with understanding and applying different
regression methods, and will use the \texttt{diamonds} data set from the
\texttt{ggplot2} package. The \texttt{diamonds} data set consists of

\begin{itemize}
\tightlist
\item
  \texttt{price} (in US dollars) - which will be our response,
\end{itemize}

and quality information (9 covariates) for around 54000 diamonds.
According to Wickham (2016) (Section 3.10) there are four C's of diamond
quality:

\begin{itemize}
\tightlist
\item
  \texttt{carat} (weight),
\item
  \texttt{cut} (quality of the cut: Fair/Good/VeryGood/Premium/Ideal),
\item
  \texttt{colour} (from worst J to best D) and
\item
  \texttt{clarity} (from worst to best: I1, SI1, SI2, VS1, VVS1, VVS2,
  IF).
\end{itemize}

In addition there are five physical measurements:

\begin{itemize}
\tightlist
\item
  \texttt{depth} (total depth percentage, calculated from x, y and z),
\item
  \texttt{table} (width of top of diamond relative to widest point),
\item
  \texttt{xx} (length in mm),
\item
  \texttt{yy} (width in mm) and
\item
  \texttt{zz} (depth in mm)
\end{itemize}

explained with a drawing in Figure 3.1 of the Wickham (2016) book (ebook
available for free for NTNU students). There are no missing data. In
addition we have made two additional transformed variables:

\begin{itemize}
\tightlist
\item
  \texttt{logprice}: logarithm (base 10) of the price, and
\item
  \texttt{logcarat}: logarithm (base 10) of the carat.
\end{itemize}

To be able to do the analyses without too much hassle (running time for
analyses and plotting) we constructed a smaller version of the data set,
and divided it into a training set \texttt{dtrain} and a test set
\texttt{dtest}, to be loaded from the course web-page as follows.

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{all=}\KeywordTok{dget}\NormalTok{(}\StringTok{"https://www.math.ntnu.no/emner/TMA4268/2019v/data/diamond.dd"}\NormalTok{)}
\NormalTok{dtrain=all}\OperatorTok{$}\NormalTok{dtrain}
\NormalTok{dtest=all}\OperatorTok{$}\NormalTok{dtest}
\end{Highlighting}
\end{Shaded}

Our aim will be to predict the price of the diamond, based on some or
all of the covariates. We would also like to understand which of the
covariates are important in predicting the price and how the covariates
are related to the price.

\subsection{a) Know your data!}\label{a-know-your-data}

\textbf{Q1}: Would you choose \texttt{price} or \texttt{logprice} as
response variable? Justify your choice. Next, plot your choice of
response pairwise with \texttt{carat}, \texttt{logcarat},
\texttt{color}, \texttt{clarity} and \texttt{cut}. Comment.

\subsection{\texorpdfstring{b) Modelling \texttt{logprice} and
\texttt{carat}}{b) Modelling logprice and carat}}\label{b-modelling-logprice-and-carat}

Use the local regression model
\(\texttt{logprice} = \beta_0 + \beta_1 \texttt{carat}+ \beta_2 \texttt{carat}^2\)
weighted by the tricube kernel \(K_{i0}\).

\textbf{Q2:} What is the predicted price of a diamond weighting 1 carat.
Use the closest 20\% of the observations.\\
\textbf{Q3:} What choice of \(\beta_1\), \(\beta_2\) and \(K_{i0}\)
would result in KNN-regression?

\subsection{c) Best subset selection}\label{c-best-subset-selection}

\textbf{Q4:} Describe how you can perform model selection in regression
with AIC as criterion.\\
\textbf{Q5:} What are the main differences between using AIC for model
selection and using cross-validation (with mean squared test error
MSE)?\\
\textbf{Q6:} See the code below for performing model selection with
\texttt{bestglm()} based on AIC. What kind of contrast is used to
represent \texttt{cut}, \texttt{color} and \texttt{clarity}? Write down
the final best model and explain what you can interpret from the
model.\\
\textbf{Q7:} Calculate and report the MSE of the test set using the best
model (on the scale of \texttt{logprice}).

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{library}\NormalTok{(bestglm)}
\NormalTok{ds=}\KeywordTok{as.data.frame}\NormalTok{(}\KeywordTok{within}\NormalTok{(dtrain,\{}
\NormalTok{  y=logprice    }\CommentTok{# setting reponse}
\NormalTok{  logprice=}\OtherTok{NULL} \CommentTok{# not include as covariate}
\NormalTok{  price=}\OtherTok{NULL}    \CommentTok{# not include as covariate}
\NormalTok{  carat=}\OtherTok{NULL}    \CommentTok{# not include as covariate}
\NormalTok{  \}))}
\NormalTok{fit=}\KeywordTok{bestglm}\NormalTok{(}\DataTypeTok{Xy=}\NormalTok{ds, }\DataTypeTok{IC=}\StringTok{"AIC"}\NormalTok{)}\OperatorTok{$}\NormalTok{BestModel}
\KeywordTok{summary}\NormalTok{(fit)}
\end{Highlighting}
\end{Shaded}

\subsection{d) Variable selection with the
lasso}\label{d-variable-selection-with-the-lasso}

\textbf{Q8:} Build a model matrix for the covariates
\texttt{\textasciitilde{}logcarat+cut+clarity+color+depth+table+xx+yy+zz-1}.
What is the dimension of this matrix?\\
\textbf{Q9:} Fit a lasso regression to the diamond data with
\texttt{logprice} as the response and the model matrix given in Q8. How
did you find the value to be used for the regularization parameter?\\
\textbf{Q10:} Calculate and report the MSE of the test set (on the scale
of \texttt{logprice}).

\subsection{e) Regression tree}\label{e-regression-tree}

\textbf{Q11:} A regression tree to model is built using a \emph{greedy}
approach. What does that mean? Explain the strategy used for
constructing a regression tree.\\
\textbf{Q12:} Is a regression tree a suitable method to handle both
numerical and categorical covariates? Elaborate.\\
\textbf{Q13:} Fit a (full) regression tree to the diamond data with
\texttt{logprice} as the response (and the same covariates as for c and
d), and plot the result. Comment briefly on you findings.\\
\textbf{Q14:} Calculate and report the MSE of the test set (on the scale
of \texttt{logprice}).

\subsection{f) Random forest}\label{f-random-forest}

\textbf{Q15:} Explain the motivation behind bagging, and how bagging
differs from random forest? What is the role of bootstrapping?\\
\textbf{Q16:} What are the parameter(s) to be set in random forest, and
what are the rules for setting these?\\
\textbf{Q17:} Boosting is a popular method. What is the main difference
between random forest and boosting?\\
\textbf{Q18:} Fit a random forest to the diamond data with
\texttt{logprice} as the response (and the same covariates as before).
Comment on your choice of parameter (as decribed in Q16).\\
\textbf{Q19:} Make a variable importance plot and comment on the plot.
Calculate and report the MSE of the test set (on the scale of
\texttt{logprice}).

\subsection{g) Comparing results}\label{g-comparing-results}

\textbf{Q20:} Finally, compare the results from c (subset selection), d
(lasso), e (tree) and f (random forest): Which method has given the best
performance on the test set and which method has given you the best
insight into the relationship between the price and the covariates?

\section{Problem 2: Unsupervised learning {[}3
points{]}}\label{problem-2-unsupervised-learning-3-points}

We will use a data set called \texttt{nci} from the Friedman, Hastie,
and Tibshirani (2001) book. The data set consists of measurement on the
activity of \(p=6830\) genes for \(n=64\) samples from cancer tumors.
Our aim here is to use unsupervised methods to study the relationship
among the cancer samples. The data is on a transformed scale, where a
negative number means that the gene has a lower regulation than some
reference sample and a positive number gives a higer regularization. The
following names are given for the cancer tumors (R print-out). It is
known that K562 cells are leukemia cells and MCF7 are breast cancer
cells. Observe the one sample labelled as \texttt{UNKNOWN}. More
information: \url{http://genome-www.stanford.edu/nci60/}

Let \({\bf X}\) be a \(n\times p\) matrix of these observations, and
\({\bf Z}\) a standardized version, where column (gene) means are 0 and
column standard deviations are 1. Further, let \({\bf R}\) be the
\(p \times p\) covariance matrix of the gene-standardized observations,
which can be estimated as

\[ {\hat {\bf R}}=\frac{1}{n-1}\sum_{i=1}^n ({\bf Z}_i-\bar{\bf Z})({\bf Z}_i-\bar{\bf Z})^T\]
where \(\bar{\bf Z}=\frac{1}{n}\sum_{i=1}^n {\bf Z}_i\).

Studying the relationship between the different tumor samples is hard
due the the high dimension of the gene data, and we know that the gene
expressions of different genes may be correlated. One way to proceed is
to calculate \emph{principal components}, which we find using the
eigenvectors of the covariance matrix \({\hat {\bf R}}\).

\subsection{a) Understanding PCA}\label{a-understanding-pca}

\textbf{Q21:} What is the definition of a principal component score, and
how is the score related to the eigenvectors of the matrix
\({\hat {\bf R}}\).

Study the R-code below. The eigenvectors of \({\hat {\bf R}}\) are found
in \texttt{pca\$rotation} and the principal component scores in
\texttt{pca\$x}.

\textbf{Q22:} Explain what is given in the plot with title ``First
eigenvector''. Why are there only \(n=64\) eigenvectors and \(n=64\)
principal component scores?

\textbf{Q23:} How many principal components are needed to explain 80\%
of the total variance in \({\bf Z}\)? Why is
\texttt{sum(pca\$sdev\^{}2)=p}?

Hint: Total variance is given as the trace of \({\hat {\bf R}}\), the
trace of a square matrix is the sum of the diagonal elements.

\textbf{Q24}: Study the PC1 vs PC2 plot, and comment on the groupings
observed. What can you say about the placement of the \texttt{K262},
\texttt{MCF7} and \texttt{UNKNOWN} samples? Produce the same plot for
two other pairs of PCs and comment on your observations.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{library}\NormalTok{(ElemStatLearn)}
\NormalTok{X=}\KeywordTok{t}\NormalTok{(nci) }\CommentTok{#n times p matrix}
\NormalTok{ngroups=}\KeywordTok{length}\NormalTok{(}\KeywordTok{table}\NormalTok{(}\KeywordTok{rownames}\NormalTok{(X)))}
\NormalTok{cols=}\KeywordTok{rainbow}\NormalTok{(ngroups)}
\NormalTok{cols[}\KeywordTok{c}\NormalTok{(}\DecValTok{4}\NormalTok{,}\DecValTok{5}\NormalTok{,}\DecValTok{7}\NormalTok{,}\DecValTok{8}\NormalTok{,}\DecValTok{14}\NormalTok{)] =}\StringTok{ "black"}
\NormalTok{pch.vec =}\StringTok{ }\KeywordTok{rep}\NormalTok{(}\DecValTok{4}\NormalTok{,}\DecValTok{14}\NormalTok{)}
\NormalTok{pch.vec[}\KeywordTok{c}\NormalTok{(}\DecValTok{4}\NormalTok{,}\DecValTok{5}\NormalTok{,}\DecValTok{7}\NormalTok{,}\DecValTok{8}\NormalTok{,}\DecValTok{14}\NormalTok{)] =}\StringTok{ }\DecValTok{15}\OperatorTok{:}\DecValTok{19}

\NormalTok{colsvsnames=}\KeywordTok{cbind}\NormalTok{(cols,}\KeywordTok{sort}\NormalTok{(}\KeywordTok{unique}\NormalTok{(}\KeywordTok{rownames}\NormalTok{(X))))}
\NormalTok{colsamples=cols[}\KeywordTok{match}\NormalTok{(}\KeywordTok{rownames}\NormalTok{(X),colsvsnames[,}\DecValTok{2}\NormalTok{])] }
\NormalTok{pchvsnames=}\KeywordTok{cbind}\NormalTok{(pch.vec,}\KeywordTok{sort}\NormalTok{(}\KeywordTok{unique}\NormalTok{(}\KeywordTok{rownames}\NormalTok{(X))))}
\NormalTok{pchsamples=pch.vec[}\KeywordTok{match}\NormalTok{(}\KeywordTok{rownames}\NormalTok{(X),pchvsnames[,}\DecValTok{2}\NormalTok{])]}

\NormalTok{Z=}\KeywordTok{scale}\NormalTok{(X)}

\NormalTok{pca=}\KeywordTok{prcomp}\NormalTok{(Z)}
\KeywordTok{plot}\NormalTok{(pca}\OperatorTok{$}\NormalTok{x[,}\DecValTok{1}\NormalTok{],pca}\OperatorTok{$}\NormalTok{x[,}\DecValTok{2}\NormalTok{],}\DataTypeTok{xlab=}\StringTok{"PC1"}\NormalTok{,}\DataTypeTok{ylab=}\StringTok{"PC2"}\NormalTok{,}\DataTypeTok{pch=}\NormalTok{pchsamples,}\DataTypeTok{col=}\NormalTok{colsamples)}
\KeywordTok{legend}\NormalTok{(}\StringTok{"bottomright"}\NormalTok{,}\DataTypeTok{legend =}\NormalTok{ colsvsnames[,}\DecValTok{2}\NormalTok{],}\DataTypeTok{cex=}\FloatTok{0.55}\NormalTok{,}\DataTypeTok{col=}\NormalTok{cols,}\DataTypeTok{pch =}\NormalTok{ pch.vec)}
\end{Highlighting}
\end{Shaded}

\includegraphics{Compulsory2_files/figure-latex/unnamed-chunk-5-1.pdf}

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{plot}\NormalTok{(}\DecValTok{1}\OperatorTok{:}\KeywordTok{dim}\NormalTok{(X)[}\DecValTok{2}\NormalTok{],pca}\OperatorTok{$}\NormalTok{rotation[,}\DecValTok{1}\NormalTok{],}\DataTypeTok{type=}\StringTok{"l"}\NormalTok{,}\DataTypeTok{xlab=}\StringTok{"genes"}\NormalTok{,}\DataTypeTok{ylab=}\StringTok{"weight"}\NormalTok{,}\DataTypeTok{main=}\StringTok{"First eigenvector"}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\includegraphics{Compulsory2_files/figure-latex/unnamed-chunk-5-2.pdf}

\subsection{b) Hierarchical clustering}\label{b-hierarchical-clustering}

Hierarchical clustering can be used to group the cancer samples, and
then we may see which samples that are most similar to the five samples
named \texttt{K262}, \texttt{MCF7} and \texttt{UNKNOWN}.

\textbf{Q25:}: Explain what it means to use Euclidean distance and
average linkage for hierarchical clustering.

\textbf{Q26:}: Perform hierarchical clustering with Euclidean distance
and average linkage on the scaled gene expression in \texttt{Z}. Observe
where our samples labelled as K562, MCF7 and \texttt{UNKNOWN} are placed
in the dendrogram. Which conclusions can you draw from this?

\textbf{Q27:}: Study the R-code and plot below. Here we have performed
hierarchical clustering based on thefirst 64 principal component instead
of the gene expression data in \texttt{Z}. What is the difference
between using all the gene expression data and using the first 64
principal components in the clustering? We have plotted the dendrogram
together with a heatmap of the data. Explain what is shown in the
heatmap. What is given on the horizontal axis, vertical axis, value in
the pixel grid?

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{library}\NormalTok{(pheatmap)}
\NormalTok{npcs=}\DecValTok{64} 
\KeywordTok{pheatmap}\NormalTok{(pca}\OperatorTok{$}\NormalTok{x[,}\DecValTok{1}\OperatorTok{:}\NormalTok{npcs],}\DataTypeTok{scale=}\StringTok{"none"}\NormalTok{,}\DataTypeTok{cluster_col=}\OtherTok{FALSE}\NormalTok{,}\DataTypeTok{cluster_row=}\OtherTok{TRUE}\NormalTok{,}\DataTypeTok{clustering_distance_rows =} \StringTok{"euclidean"}\NormalTok{,}\DataTypeTok{clustering_method=}\StringTok{"average"}\NormalTok{,}\DataTypeTok{fontsize_row=}\DecValTok{5}\NormalTok{,}\DataTypeTok{fontsize_col=}\DecValTok{5}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\includegraphics{Compulsory2_files/figure-latex/unnamed-chunk-6-1.pdf}

\section{Problem 3: Flying solo with diabetes data {[}6
points{]}}\label{problem-3-flying-solo-with-diabetes-data-6-points}

A learning objective in this course is: \emph{The student knows, based
on an existing data set, how to choose a suitable statistical model,
apply sound statistical methods, and perform the analyses using
statistical software. The student knows how to present the results from
the statistical analyses, and which conclusions can be drawn from the
analyses.}

This problem addresses that objective.

We will use the classical data set of \emph{diabetes} from a population
of women of Pima Indian heritage in the US, available in the R
\texttt{MASS} package. The following information is available for each
woman:

\begin{itemize}
\tightlist
\item
  diabetes: \texttt{0}= not present, \texttt{1}= present
\item
  npreg: number of pregnancies
\item
  glu: plasma glucose concentration in an oral glucose tolerance test
\item
  bp: diastolic blood pressure (mmHg)
\item
  skin: triceps skin fold thickness (mm)
\item
  bmi: body mass index (weight in kg/(height in m)\(^2\))
\item
  ped: diabetes pedigree function.
\item
  age: age in years
\end{itemize}

We will use a training set (called \texttt{ctrain}) with 300
observations (200 non-diabetes and 100 diabetes cases) and a test set
(called \texttt{ctest}) with 232 observations (155 non-diabetes and 77
diabetes cases). Our aim is to make a classification rule for diabetes
(or not) based on the available data.

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{flying=}\KeywordTok{dget}\NormalTok{(}\StringTok{"https://www.math.ntnu.no/emner/TMA4268/2019v/data/flying.dd"}\NormalTok{)}
\NormalTok{ctrain=flying}\OperatorTok{$}\NormalTok{ctrain}
\NormalTok{ctest=flying}\OperatorTok{$}\NormalTok{ctest}
\end{Highlighting}
\end{Shaded}

\textbf{Q28:} Start by getting to know the \emph{training data}, by
producing summaries and plots. Write a few sentences about what you
observe and include your top 3 informative plots and/or outputs.

\textbf{Q29:} Use different methods to analyse the data. In particular
use

\begin{itemize}
\tightlist
\item
  one method from Module 4: Classification
\item
  one method from Module 8: Trees (and forests)
\item
  one method from Module 9: Support vector machines and, finally
\item
  one method from Module 11: Neural networks
\end{itemize}

For each method you

\begin{itemize}
\tightlist
\item
  clearly write out the model and model assumptions for the method
\item
  explain how any tuning parameters are chosen or model selection is
  performed
\item
  report (any) insight into the interpretation of the fitted model
\item
  evaluate the model using the test data, and report misclassifiation
  rate (cut-off 0.5 on probability) and plot ROC-curves and give the AUC
  (for method where class probabilities are given).
\end{itemize}

\textbf{Q30:} Conclude with choosing a winning method, and explain why
you mean that this is the winning method.

\section*{References}\label{references}
\addcontentsline{toc}{section}{References}

\hypertarget{refs}{}
\hypertarget{ref-ESL}{}
Friedman, Jerome, Trevor Hastie, and Robert Tibshirani. 2001. \emph{The
Elements of Statistical Learning}. Vol. 1. Springer series in statistics
New York.

\hypertarget{ref-ggplot2}{}
Wickham, Hadley. 2016. \emph{Ggplot2 Elegant Graphics for Data
Analysis}. Springer.


\end{document}
