{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Lab 1: Introduction to Machine Learning\n",
"\n",
"This lab introduces some basic concepts of machine learning with R. In this lab you will use the K-Nearest Neighbor (KNN) algorithm to classify the species of iris flowers, given meaurements of flower characteristics. By completing this lab you will have an overview of an end-to-end machine learning modeling process. \n",
"\n",
"By the completion of this lab, you will:\n",
"1. Follow and understand a complete end-to-end machine learning process including data exploration, data preparation, modeling, and model evaluation. \n",
"2. Develop a basic understanding of the principles of machine learning and associated terminology. \n",
"3. Understand the basic process for evaluating machine learning models. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Overview of KNN classification\n",
"\n",
"Before discussing a specific algorithm, it helps to know a bit of machine learning terminology. In supervised machine learning a set of ***cases*** are used to ***train***, ***test*** and ***evaluate*** the model. Each case is comprised of the values of one or more ***features*** and a ***label*** value. The features are variables used by the model to ***predict** the value of the label. Minimizing the ***errors*** between the true value of the label and the prediction supervises the training of this model. Once the model is trained and tested, it can be evaluated based on the accuracy in predicting the label of a new set of cases. \n",
"\n",
"In this lab you will use randomly selected cases to first train and then evaluate a k-nearest-neighbor (KNN) machine learning model. The goal is to predict the type or class of the label, which makes the machine learning model a ***classification*** model. \n",
"\n",
"The k-nearest-neighbor algorithm is conceptually simple. In fact, there is no formal traning step. Given a known set of cases, a new case is classified by majority vote of the K (where $k = 1, 2, 3$, etc.) points nearest to the values of the new case; that is, the nearest neighbors of the new case. \n",
"\n",
"The schematic figure below illustrates the basic concepts of a KNN classifier. In this case there are two features, the values of one shown on the horrizontal axis and the values of the other shown on the vertical axis. The cases are shown on the diagram as one of two classes, red triangles and blue circles. To summarize, each case has a value for the two features, and a class. The goal of thee KNN algorithm is to classify cases with unknown labels. \n",
"\n",
"Continuing with the example, on the left side of the diagram the $K = 1$ case is illustrated. The nearest neighbor is a red triangle. Therefore, this KNN algorithm will classify the unknown case, '?', as a red triangle. On the right side of the diagram, the $K = 3$ case is illustrated. There are three near neighbors within the circle. The majority of nearest neighbors for $K = 3$ are the blue circles, so the algorithm classifies the unknown case, '?', as a blue circle. Notice that class predicted for the unknown case changes as K changes. This behavior is inherent in the KNN method. \n",
"\n",
"\n",
" **KNN for k = 1 and k = 3**\n",
"\n",
"There are some additional considerations in creating a robust KNN algorithm. These will be addressed later in this course. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Examine the data set\n",
"\n",
"In this lab you will work with the Iris data set. This data set is famous in the history of statistics. The first publication using these data in statistics by the pioneering statistician Ronald A Fisher was in his 1936. Fisher proposed an algorthm to classify the species of iris flowers from physical measurements of their characteristics. The data set has been used as a teaching example ever since. \n",
"\n",
"Now, you will load and examine these data which are in the statsmodels.api package. Execute the code in the cell below and examine the first few rows of the data frame. "
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"C:\\Users\\StevePC2\\Anaconda3\\lib\\site-packages\\statsmodels\\compat\\pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.\n",
" from pandas.core import datetools\n"
]
},
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" Sepal_Length | \n",
" Sepal_Width | \n",
" Petal_Length | \n",
" Petal_Width | \n",
" Species | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 5.1 | \n",
" 3.5 | \n",
" 1.4 | \n",
" 0.2 | \n",
" setosa | \n",
"
\n",
" \n",
" | 1 | \n",
" 4.9 | \n",
" 3.0 | \n",
" 1.4 | \n",
" 0.2 | \n",
" setosa | \n",
"
\n",
" \n",
" | 2 | \n",
" 4.7 | \n",
" 3.2 | \n",
" 1.3 | \n",
" 0.2 | \n",
" setosa | \n",
"
\n",
" \n",
" | 3 | \n",
" 4.6 | \n",
" 3.1 | \n",
" 1.5 | \n",
" 0.2 | \n",
" setosa | \n",
"
\n",
" \n",
" | 4 | \n",
" 5.0 | \n",
" 3.6 | \n",
" 1.4 | \n",
" 0.2 | \n",
" setosa | \n",
"
\n",
" \n",
"
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" count | \n",
"
\n",
" \n",
" | Species | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" | setosa | \n",
" 50 | \n",
"
\n",
" \n",
" | versicolor | \n",
" 50 | \n",
"
\n",
" \n",
" | virginica | \n",
" 50 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" Sepal_Length | \n",
" Sepal_Width | \n",
" Petal_Length | \n",
" Petal_Width | \n",
" Species | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" -0.900681 | \n",
" 1.019004 | \n",
" -1.340227 | \n",
" -1.315444 | \n",
" 0 | \n",
"
\n",
" \n",
" | 1 | \n",
" -1.143017 | \n",
" -0.131979 | \n",
" -1.340227 | \n",
" -1.315444 | \n",
" 0 | \n",
"
\n",
" \n",
" | 2 | \n",
" -1.385353 | \n",
" 0.328414 | \n",
" -1.397064 | \n",
" -1.315444 | \n",
" 0 | \n",
"
\n",
" \n",
" | 3 | \n",
" -1.506521 | \n",
" 0.098217 | \n",
" -1.283389 | \n",
" -1.315444 | \n",
" 0 | \n",
"
\n",
" \n",
" | 4 | \n",
" -1.021849 | \n",
" 1.249201 | \n",
" -1.340227 | \n",
" -1.315444 | \n",
" 0 | \n",
"
\n",
" \n",
"
\n",
"