02-PCA-Exercise-Project

Principal Component Analysis - Project Exercise

GOAL: Figure out which handwritten digits are most differentiated with PCA.

Imagine you are working on an image recognition service for a postal service. It would be very useful to be able to read in the digits automatically, even if they are handwritten. (Quick note, this is very much how modern postal services work for a long time now and its actually more accurate than a human). The manager of the postal service wants to know which handwritten numbers are the hardest to tell apart, so he can focus on getting more labeled examples of that data. You will have a dataset of hand written digits (a very famous data set) and you will perform PCA to get better insight into which numbers are easily separable from the rest.

Data

Background:

E. Alpaydin, Fevzi. Alimoglu Department of Computer Engineering Bogazici University, 80815 Istanbul Turkey alpaydin '@' boun.edu.tr

Data Set Information from Original Authors:

We create a digit database by collecting 250 samples from 44 writers. The samples written by 30 writers are used for training, cross-validation and writer dependent testing, and the digits written by the other 14 are used for writer independent testing. This database is also available in the UNIPEN format.

We use a WACOM PL-100V pressure sensitive tablet with an integrated LCD display and a cordless stylus. The input and display areas are located in the same place. Attached to the serial port of an Intel 486 based PC, it allows us to collect handwriting samples. The tablet sends $x$ and $y$ tablet coordinates and pressure level values of the pen at fixed time intervals (sampling rate) of 100 miliseconds.

These writers are asked to write 250 digits in random order inside boxes of 500 by 500 tablet pixel resolution. Subject are monitored only during the first entry screens. Each screen contains five boxes with the digits to be written displayed above. Subjects are told to write only inside these boxes. If they make a mistake or are unhappy with their writing, they are instructed to clear the content of a box by using an on-screen button. The first ten digits are ignored because most writers are not familiar with this type of input devices, but subjects are not aware of this.

SOURCE: https://archive.ics.uci.edu/ml/datasets/Pen-Based+Recognition+of+Handwritten+Digits

Complete the Tasks in bold below

TASK: Run the cells below to import the libraries and relevant data set.

digits = pd.read_csv('../DATA/digits.csv')

digits

pixel_0_0
pixel_0_1
pixel_0_2
pixel_0_3
pixel_0_4

pixel_0_5
pixel_0_6
pixel_0_7
pixel_1_0
pixel_1_1
...
pixel_6_7
pixel_7_0
pixel_7_1
pixel_7_2
pixel_7_3
pixel_7_4
pixel_7_5
pixel_7_6
pixel_7_7
number_label
0
0.0
0.0
5.0
13.0
9.0
1.0
0.0
0.0
0.0
0.0
...
0.0
0.0
0.0
6.0
13.0
10.0
0.0
0.0
0.0
0
1
0.0
0.0
0.0
12.0
13.0
5.0
0.0
0.0
0.0
0.0
...
0.0
0.0
0.0
0.0
11.0
16.0
10.0
0.0
0.0
1
2
0.0
0.0
0.0
4.0
15.0
12.0
0.0
0.0
0.0
0.0
...
0.0
0.0
0.0
0.0
3.0
11.0
16.0
9.0
0.0
2
3
0.0
0.0
7.0
15.0
13.0
1.0
0.0
0.0
0.0
8.0
...
0.0
0.0
0.0
7.0
13.0
13.0
9.0
0.0
0.0
3
4
0.0
0.0
0.0
1.0
11.0
0.0
0.0
0.0
0.0
0.0
...
0.0
0.0
0.0
0.0
2.0
16.0
4.0
0.0
0.0
4
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
1792
0.0
0.0
4.0
10.0
13.0
6.0
0.0
0.0
0.0
1.0
...
0.0
0.0
0.0
2.0
14.0
15.0
9.0
0.0
0.0
9
1793
0.0
0.0
6.0
16.0
13.0
11.0
1.0
0.0
0.0
0.0
...
0.0
0.0
0.0
6.0
16.0
14.0
6.0
0.0
0.0
0
1794
0.0
0.0
1.0
11.0
15.0
1.0
0.0
0.0
0.0
0.0
...
0.0
0.0
0.0
2.0
9.0
13.0
6.0
0.0
0.0
8
1795
0.0
0.0
2.0
10.0
7.0
0.0
0.0
0.0
0.0
0.0
...
0.0
0.0
0.0
5.0
12.0
16.0
12.0
0.0
0.0
9
1796
0.0
0.0
10.0
14.0
8.0
1.0
0.0
0.0
0.0
2.0
...
0.0
0.0
1.0
8.0
12.0
14.0
12.0
1.0
0.0
8
1797 rows × 65 columns

*TASK: Create a new DataFrame called pixels that consists only of the pixel feature values by dropping the number_label column.*

#CODE HERE

Displaying an Image

TASK: Grab a single image row representation by getting the first row of the pixels DataFrame.

#CODE HERE

TASK: Convert this single row Series into a numpy array.

#CODE HERE

TASK: Reshape this numpy array into an (8,8) array.

#CODE HERE

TASK: Use Matplotlib or Seaborn to display the array as an image representation of the number drawn. Remember your palette or cmap choice would change the colors, but not the actual pixel values.

#CODE HERE

Now let's move on to PCA.

Scaling Data

TASK: Use Scikit-Learn to scale the pixel feature dataframe.

#CODE HERE

PCA

TASK: Perform PCA on the scaled pixel data set with 2 components.

TASK: How much variance is explained by 2 principal components.

#CODE HERE

TASK: Create a scatterplot of the digits in the 2 dimensional PCA space, color/label based on the original number_label column in the original dataset.

#CODE HERE

TASK: Which numbers are the most "distinct"?

# You should see label #4 as being the most separated group, 
# implying its the most distinct, similar situation for #2, #6 and #9.

Bonus Challenge

TASK: Create an "interactive" 3D plot of the result of PCA with 3 principal components. Lot's of ways to do this, including different libraries like plotly or bokeh, but you can actually do this just with Matplotlib and Jupyter Notebook. Search Google and StackOverflow if you get stuck, lots of solutions are posted online.

#CODE HERE

Great Job!