---
subtitle: "Spring 2018"
title: "TMA4268 Statistical Learning"
author: "Mette Langaas, Department of Mathematical Sciences NTNU"
date: "22.05.2018"
output: #3rd letter intentation hierarchy
  html_document:
    toc: true
    toc_float: true
#   pdf_document:
#    toc: true
#    toc_depth: 2
#  beamer_presentation:
###    incremental: true # or >* for one at a time
---

# Aim

This is an overview of learning material in the course [TMA4268 Statistical learning](https://wiki.math.ntnu.no/tma4268) given at the [Department of Mathematical Sciences](https://www.ntnu.edu/imf) at [NTNU](https://www.ntnu.edu/). [Course description with formal information](https://www.ntnu.edu/studies/courses/TMA4268#tab=omEmnet).

# Module pages

These are links to (html-versions of) module pages in TMA4268 Statistical learning, that was held in the spring semester of 2018 at NTNU. (To see .Rmd and .pdf just use in place of .html for most of the module pages)

1. [Introduction](https://www.math.ntnu.no/emner/TMA4268/2018v/1Intro/1Intro.html)
2. [Statistical learning](https://www.math.ntnu.no/emner/TMA4268/2018v/2StatLearn/2StatLearn.html) 
3. [Linear regression](https://www.math.ntnu.no/emner/TMA4268/2018v/3LinReg/3LinReg.html)
4. [Classification](https://www.math.ntnu.no/emner/TMA4268/2018v/4Classif/4Classif.html)
5. [Resampling methods](https://www.math.ntnu.no/emner/TMA4268/2018v/5Resample/5Resample.html)
6. Linear model selection and regularization: [2 files](https://www.math.ntnu.no/emner/TMA4268/2018v/12Summing/12Final.html#6_linear_model_selection_and_regularization:)
7. [Moving beyond linearity](https://www.math.ntnu.no/emner/TMA4268/2018v/7BeyondLinear/7.html) 
8. [Tree-based methods](https://www.math.ntnu.no/emner/TMA4268/2018v/8Trees/8Trees.html)
9. [Support vector machines](https://www.math.ntnu.no/emner/TMA4268/2018v/9SVM/9SVM.html)
10. Unsupervised learning: [6 files](https://www.math.ntnu.no/emner/TMA4268/2018v/12Summing/12Final.html#10_unsupervised_learning:_6_files)
11. Neural networks [13 files](https://www.math.ntnu.no/emner/TMA4268/2018v/12Summing/12Final.html#11_neural_networks:_13_files)
12. [Summing up](https://www.math.ntnu.no/emner/TMA4268/2018v/12Summing/12Final.html)

# Exam

May 24, 4hrs:

* [English](https://www.math.ntnu.no/emner/TMA4268/Exam/V2018e.pdf) [Bokmål](https://www.math.ntnu.no/emner/TMA4268/Exam/V2018b.pdf)
[Nynorsk](https://www.math.ntnu.no/emner/TMA4268/Exam/V2018n.pdf)
* [Tentative solutions](https://www.math.ntnu.no/emner/TMA4268/Exam/e2018sol.pdf) 
* [Grading document](https://www.math.ntnu.no/emner/TMA4268/Exam/gradingdocumentV2018.pdf)

# Compulsory exercises

* Modules 2-4: [Compulsory1](https://www.math.ntnu.no/emner/TMA4268/2018v/Compulsory1.html) and [Short solutions](https://www.math.ntnu.no/emner/TMA4268/2018v/CompEx/Compulsory1withLF.html) 
* Modules 5-7: [Compulsory2](https://www.math.ntnu.no/emner/TMA4268/2018v/Compulsory2.html) and [Short solutions](https://www.math.ntnu.no/emner/TMA4268/2018v/CompEx/Compulsory2solutions.html) 
* Modules 8-11: [Compulsory3](https://www.math.ntnu.no/emner/TMA4268/2018v/Compulsory3.html) and [Short solutions](https://www.math.ntnu.no/emner/TMA4268/2018v/CompEx/Compulsory3solutions.html) 


# Solutions to recommended exercises

The recommended exercises are found in each of the module pages (mainly in the end of the pages).

2. Statistical learning [M2](https://www.math.ntnu.no/emner/TMA4268/2018v/2StatLearn/2RecEx-sol.html)
3. Linear regression [M3](https://www.math.ntnu.no/emner/TMA4268/2018v/3LinReg/3LinReg-sol.html)
4. Classification [M4](https://www.math.ntnu.no/emner/TMA4268/2018v/4Classif/4Classif-sol.html)
5. Resampling [M5](https://www.math.ntnu.no/emner/TMA4268/2018v/5Resample/5Resample-sol.pdf)
6. Linear model selection and regularization [M6](https://www.math.ntnu.no/emner/TMA4268/2018v/6selreg/selection_regularization_recommended_exercises.html)
7. Moving beyond linearity [M7](https://www.math.ntnu.no/emner/TMA4268/2018v/7BeyondLinear/7sol.html)
8. Tree-based methods [M8](https://www.math.ntnu.no/emner/TMA4268/2018v/8Trees/8Trees-sol.html)
10. Unsupervised methods [M10](https://www.math.ntnu.no/emner/TMA4268/2018v/10unsuper/unsupervised_learning_recommended_exercises.html)

# Planned changes for 2019

* Change the reading list for Module 11 and focus on basic concepts, use the theory presented in Module 11 above in the phd course MA8701 in the spring of 2019.
* Add more information on handling big data, maybe partly in Module 3.
* Have parts of a compulsory exericise where analysis methods are chosen by the students, that is, a data set with description and aim is given (or can be chosen by the student?). This will focus on data analysis skills in R.

# Connections to other statistics courses at IMF/NTNU

**Nice to have both before, after and at the same time as TMA4268:**

* (V)[TMA4267 Linear statistical models](https://www.math.ntnu.no/emner/TMA4267) Multiple linear regression. Analysis of variance. Experimental design. Multivariate normal distribution. Multiple testing. 
* (V)[TMA4180 Optimization 1](https://www.ntnu.edu/studies/courses/TMA4180). 
First and second order necessary and sufficient (Karush-Kuhn-Tucker) optimality conditions for unconstrained and constrained optimization problems in finite-dimensional vector spaces. Basics of convex analysis and Lagrangian duality theory and their application to optimization problems and algorithms. An overview of modern optimization techniques and algorithms for smooth problems (including line-search/trust-region, quasi-Newton, interior point and active set methods, SQP and augmented Lagrangian approaches). Basic derivative-free and non-smooth optimization methods.

**Expanding on topics mentioned in TMA4268**

* (V)[TMA4250 Spatial statistics](https://www.math.ntnu.no/emner/TMA4250). 
Parameter estimation, simulation and applications of Gaussian random fields, point fields and 
discrete Markov random fields. Examples from image analysis, and environmental and natural 
resource applications. 
* (V)[TMA4275 Lifetime analysis](https://www.math.ntnu.no/emner/TMA4275/). 
Basic concepts in lifetime modelling. Censored observations. Nonparametric estimation and 
graphical plotting for lifetime data (Kaplan-Meier, Nelson-plot). Estimation and testing in 
parametric lifetime distributions. Analysis of lifetimes with covariates (Cox-regression, accelerated lifetime testing). Modelling and analysis of recurrent events. Nonhomogeneous Poisson-processes. Nelson-Aalen estimators. 
* (H)[TMA4285 Time series models ](https://www.math.ntnu.no/emner/TMA4285/) 
Autoregressive and moving average based models for stationary and non-stationary time series. Parameter estimation. Model identification. Forecasting. ARCH and GARCH models for volatility. State space models (linear dynamic models) and the Kalman filter. 
* (H)[TMA4295 Statistical inference](https://www.math.ntnu.no/emner/TMA4295/). 
Transformations and moments of random variables. Families of distributions. Inequalities and convergence theorems. Sufficient statistics. Frequentist and Bayesian estimators. Methods of constructing point estimators, interval estimators and hypothesis tests, and optimality of these. Asymptotic properties of estimators and hypothesis tests. 
* (V)[TMA4300 Computational statistics](https://www.math.ntnu.no/emner/TMA4300/). 
Classical and Markov chain methods for stochastic simulation. Hierarchical Bayesian models and inference in these. The expectation maximisation (EM) algorithm. Bootstrapping, cross-validation and non-parametric methods. 
* (H)[TMA4315 Generalized linear models](https://www.math.ntnu.no/emner/TMA4315/). 
Univariate exponential family. Multiple linear regression. Logistic regression. Poisson regression. General formulation for generalised linear models with canonical link. Likelihood-based inference with score function and expected Fisher information. Deviance. AIC. Wald and likelihood-ratio test. Linear mixed effects models with random components of general structure. Random intercept and random slope. Generalised linear mixed effects models. Strong emphasis on programming in R. Possible extensions: quasi-likelihood, over-dispersion, models for multinomial data, analysis of contingency tables, quantile regression.

**PhD courses**

* (H)[MA8704 Asympotic methods](http://www.ntnu.no/studier/emner/MA8701/) every autumn. Requires TMA4295.
* (V)[MA8701 General statistical methods](http://www.ntnu.no/studier/emner/MA8701/) next time spring of 2019. We go in more detail into related topics to TMA4268. Requires TMA4295 and TMA4300. Nice to have TMA4180, TMA4285 and TMA4315.
* (V)[MA8702 Computational statistics 2](http://www.ntnu.no/studier/emner/MA8702/) next time spring of 2020. Requires TMA4300 and TMA4250.


