Assigment:
In the second ML assignment you have to compare the performance of three different classification algorithms, namely Naive Bayes, SVM, and Random Forest. For this assignment you need to generate a random binary classification problem, and then train and test (using 10-fold cross validation) the three algorithms. For some algorithms inner cross validation (5-fold) for choosing the parameters is needed. Then, show the classification performace (per-fold and averaged) in the report, and briefly discussing the results.
Steps:
1. Create a classification dataset (n samples >=1000, n features >= 10)
![](https://i-blog.csdnimg.cn/blog_migrate/765d0f7aa6858656fba004cf9c76a224.png)
2. Split the dataset using 10-fold cross validation
![](https://i-blog.csdnimg.cn/blog_migrate/729b60d2dca35e332d4a7166d1d7ff33.png)
3. Train the algorithms
I GaussianNB
![](https://i-blog.csdnimg.cn/blog_migrate/efe96f94db6058066f6def83cd3d67b0.png)
I SVC (possible C values [1e-02, 1e-01, 1e00, 1e01, 1e02], RBF kernel)
![](https://i-blog.csdnimg.cn/blog_migrate/ab81d3ebe8c14c0d40f1fd5794968565.png)
I RandomForestClassifier (possible n estimators values [10, 100, 1000])
![](https://i-blog.csdnimg.cn/blog_migrate/b8d908f85086cbaa1ba2125b9501451b.png)
4. Evaluate the cross-validated performance
I Accuracy
![](https://i-blog.csdnimg.cn/blog_migrate/c40d61670e15588428c5a03696ad3293.png)
I F1-score
I AUC ROC
value error occurs...
5. Write a short report summarizing the methodology and the results
Scikit-learn allows to:
Load and generate datasets
Split them to perform cross-validation
Easily apply learning algorithms
Evaluate the platforms of such algorithms