常用机器学习工具库

最新推荐文章于 2024-04-07 10:47:18 发布

梦想照进灵魂

最新推荐文章于 2024-04-07 10:47:18 发布

阅读量100

点赞数

文章标签：机器学习人工智能

本文链接：https://blog.csdn.net/wow20008/article/details/133612874

版权

我一般学习新的框架或工具库喜欢看着官方文档来码代码，所以本文就以这个方式介绍下自己常用的一些机器学习工具库。

机器学习

Numpy

NumPy是一个用于数值计算的库，提供了多维数组和矩阵操作，适用于处理大规模数学计算和数据处理。

numpyhttps://www.numpy.org.cn/reference/

import numpy as np

a = np.arange(12)
print(a.view())
print('\nafter reshape...\n')
a.shape = 3, 4
print(a.view())

boardcast = a**3
print('\n广播')
print(boardcast)


Output:
[ 0  1  2  3  4  5  6  7  8  9 10 11]

after reshape...

[[ 0  1  2  3]
 [ 4  5  6  7]
 [ 8  9 10 11]]

广播
[[   0    1    8   27]
 [  64  125  216  343]
 [ 512  729 1000 1331]]

Pandas

Pandas是一个数据分析库，提供了数据结构和数据分析工具，适用于处理格式化的数据文件。

pandashttps://www.pypandas.cn/docs/

import pandas as pd

s = pd.Series([1, 3, 5, np.nan, 6, 8])
print('Series\n')
print(s)

df = pd.DataFrame({
    'A': 1.,
    'B': pd.Timestamp('20130102'),
    'C': pd.Series(1, index=list(range(4)), dtype='float32'),
    'D': np.array([3] * 4, dtype='int32'),
    'E': pd.Categorical(["test", "train", "test", "train"]),
    'F': 'foo'})
print('\nDataFrame\n')
print(df)


Output:
Series:
0    1.0
1    3.0
2    5.0
3    NaN
4    6.0
5    8.0
dtype: float64

DataFrame:
     A          B    C  D      E    F
0  1.0 2013-01-02  1.0  3   test  foo
1  1.0 2013-01-02  1.0  3  train  foo
2  1.0 2013-01-02  1.0  3   test  foo
3  1.0 2013-01-02  1.0  3  train  foo

Gensim

Gensim是一个用于主题建模和文档相似性的库，适用于处理文本数据。

gensimhttps://radimrehurek.com/gensim/apiref.html#api-reference

text_corpus = [
    "Human machine interface for lab abc computer applications",
    "A survey of user opinion of computer system response time",
    "The EPS user interface management system",
    "System and human system engineering testing of EPS",
    "Relation of user perceived response time to error measurement",
    "The generation of random binary unordered trees",
    "The intersection graph of paths in trees",
    "Graph minors IV Widths of trees and well quasi ordering",
    "Graph minors A survey",
]


# Create a set of frequent words
stoplist = set('for a of the and to in'.split(' '))
# Lowercase each document, split it by white space and filter out stopwords
texts = [[word for word in document.lower().split() if word not in stoplist]
         for document in text_corpus]

# Count word frequencies
from collections import defaultdict
frequency = defaultdict(int)
for text in texts:
    for token in text:
        frequency[token] += 1

# Only keep words that appear more than once
processed_corpus = [[token for token in text if frequency[token] > 1] for text in texts]
print(processed_corpus)

from gensim import corpora

dictionary = corpora.Dictionary(processed_corpus)
print(dictionary)
print(dictionary.token2id)
new_doc = "Human response interaction"
new_vec = dictionary.doc2bow(new_doc.lower().split())
print(new_vec)


Output:
processed_corpus:
[['human', 'interface', 'computer'], ['survey', 'user', 'computer', 'system', 'response', 'time'], ['eps', 'user', 'interface', 'system'], ['system', 'human', 'system', 'eps'], ['user', 'response', 'time'], ['trees'], ['graph', 'trees'], ['graph', 'minors', 'trees'], ['graph', 'minors', 'survey']]

dictionary:
Dictionary<12 unique tokens: ['computer', 'human', 'interface', 'response', 'survey']...>

dictionary.token2id:
{'computer': 0, 'human': 1, 'interface': 2, 'response': 3, 'survey': 4, 'system': 5, 'time': 6, 'user': 7, 'eps': 8, 'trees': 9, 'graph': 10, 'minors': 11}

new_vec:
[(1, 1), (3, 1)]