Classification and Regression Trees and Forests
December 9 & 10
Prof. Wei-Yin Loh, Department of Statistics, University of Wisconsin, Madison
Classification
and regression trees are essential tools for data mining, machine
learning, and statistical data analysis. This year marks the 50th
anniversary of the publication of the first journal article on the
subject. In a classification or regression tree model, the data and
sample space are split into two or more partitions and a simple
statistical model is fitted to each of them. The model is intuitive to
interpret because the partitions can be displayed as a decision tree.
Besides, the models often possess prediction accuracy as good as or
better than that of linear discriminant analysis and linear regression.
This course reviews the major techniques and discusses their relative
strengths, weaknesses, capabilities, and computational requirements.
Extensions to ensemble procedures, such as bagging and random forest,
are included. Concepts are explained with examples from business,
industry, science, and engineering.
Special emphasis is given to the instructor’s GUIDE algorithm
(http://www.stat.wisc.edu/~loh/guide.html). For classification, GUIDE
can construct decision trees with nonparametric models, such as nearest
neighbor and kernel discrimination, in the nodes. For regression, GUIDE
can use least squares, least median of squares, quantile, Poisson, and
proportional hazards loss functions and apply them to univariate,
multivariate, longitudinal, and censored response data. If time
permits, instruction on the use of GUIDE and other free software will
be given.
For a brief review of the subject, see Loh, W.-Y. (2011).
“Classification and regression trees.” Wiley
Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 1,
14-23. (http://www.stat.wisc.edu/~loh/treeprogs/guide/wires11.pdf).
For a list of sample applications in the scientific literature, see http://www.stat.wisc.edu/~loh/apps.html.
Course Notes
This short course is included in all conference registration plans except the one-day attendance.
Return to ICAS Home Page