支持向量机

以下是一个基于支持向量机的中文文本分类示例代码，包括数据集的读取、数据预处理、特征提取、模型训练、模型评估等步骤，可以直接应用到项目中：

import os
import jieba
import numpy as np
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

# 读取数据集
def load_dataset(path):
    texts, labels = [], []
    for label_name in os.listdir(path):
        label_path = os.path.join(path, label_name)
        label = label_name.split('.')[0]
        for file_name in os.listdir(label_path):
            file_path = os.path.join(label_path, file_name)
            with open(file_path, 'r', encoding='utf-8') as f:
                text = f.read()
                texts.append(text)
                labels.append(label)
    return texts, labels

# 数据预处理和分词
def preprocess(text):
    words = jieba.lcut(text)
    words = [word for word in words if len(word) > 1]
    return ' '.join(words)

# 加载数据集
train_path = 'train/'
test_path = 'test/'
train_texts, train_labels = load_dataset(train_path)
test_texts, test_labels = load_dataset(test_path)

# 数据预处理和分词
train_texts = [preprocess(text) for text in train_texts]
test_texts = [preprocess(text) for text in test_texts]

# 特征提取
vectorizer = TfidfVectorizer(max_features=10000)
train_features = vectorizer.fit_transform(train_texts)
test_features = vectorizer.transform(test_texts)

# 模型训练和评估
clf = SVC(kernel='linear')
scores = cross_val_score(clf, train_features, train_labels, cv=5, scoring='accuracy')
print('交叉验证结果：', scores)
clf.fit(train_features, train_labels)
pred_labels = clf.predict(test_features)
acc = accuracy_score(test_labels, pred_labels)
precision = precision_score(test_labels, pred_labels, average='macro')
recall = recall_score(test_labels, pred_labels, average='macro')
f1 = f1_score(test_labels, pred_labels, average='macro')
print('准确率：', acc)
print('精确率：', precision)
print('召回率：', recall)
print('F1得分：', f1)

以上代码中，我们首先使用load_dataset函数读取数据集，然后使用preprocess函数对文本进行预处理和分词。我们使用TfidfVectorizer类将文本转换为TF-IDF特征向量，并使用SVC类训练支持向量机模型。我们使用cross_val_score函数进行交叉验证，并使用accuracy_score、precision_score、recall_score和f1_score函数对模型进行评估。

在使用此代码时，需要将数据集放在train和test文件夹中，并按照类别分别存储。例如，train/体育/1.txt表示体育类别下的第一个文本。