A. Please define the following terms as accurately, clearly, and concisely as possible. (50 points)

a. Overall Error Rate of a Classifier
b. Overall Accuracy Rate of a Classifier
c. The Sensitivity
d. The Specificity
e. False Positive Rate
f. False Negative Rate
g. Euclidean Distance (between two records, (x1, x2, …., xn1 ) and (y1, y2, …., yn))
h. Cutoff (Threshold)
i. ROC (Receiver Operating Characteristics)
j. Lift

B. A large number of insurance records are to be examined to develop a model for predicting fraudulent claims. Of the claims in the historical database, 1% were judged to be fraudulent. A sample database is taken to develop a model, and oversampling is used to provide a balanced sample in light of the very low response rate. When applied to this sample database (total number of records, N =800), the model ends up correctly classifying 310 frauds, and 270 non-frauds. It misses 90 frauds, and
classified 130 records incorrectly as frauds when they were not. Questions: (30 points)
a. Produce the classification matrix for the sample as it stands.
b. Find the adjusted misclassification rate (adjusting for the oversampling).
c. What percentage of new records would you expected to be classified as fraudulent?

C. Predicting housing median prices.
The file BostonHousing.xls contains information on over 500 census tracts in Boston, where for each tract 14 variables are recorded. The last column (CAT.MEDV) was derived from MEDV, such that it obtains the value 1 if MEDV>30 and 0 otherwise. Consider the goal of predicting the median value (MEDV) of a tract, given the information in the first 13 columns. Partition the data into training (60%) and validation (40%) sets. (For interpretation of the column names in BostonHousing.xls, please make reference to Table 2.2 on page 33 of the textbook, the 3rd edition, on page 27 of the 2nd edition)
a) Perform a k-nearest neighbors prediction with all 13 predictors (the CAT.MEDV column is the outcome or decision variable), trying values of k from 1 to 10. Make sure to normalize the data (click “normalize input data”). What is the best k chosen? What does it mean? (10 points)
b) Why is the validation data error overly optimistic compared to the error rate when applying this kNN predictor to new data? (10 points)

Note: 1. The file BostonHousing.xls is posted along Written Assignment #2.