自己开发了一个股票软件,功能很强大,需要的点击下面的链接获取:
https://www.cnblogs.com/bclshuai/p/11380657.html
1.1 tensorflow预定义经典卷积神经网络和数据集
1.1.1 预定义模型tf.keras.applications
tensorflow有很多已经定义好的模型,而且模型参数已经训练过,可以直接下载模型参数文件,载入参数,使用模型。预定义模型在tf.keras.applications。
# This file is MACHINE GENERATED! Do not edit.
# Generated by: tensorflow/python/tools/api/generator/create_python_api.py script.
"""Keras Applications are canned architectures with pre-trained weights.
"""
from __future__ import print_function as _print_function
import sys as _sys
from . import densenet
from . import efficientnet
from . import imagenet_utils
from . import inception_resnet_v2
from . import inception_v3
from . import mobilenet
from . import mobilenet_v2
from . import nasnet
from . import resnet
from . import resnet50
from . import resnet_v2
from . import vgg16
from . import vgg19
from . import xception
from tensorflow.python.keras.applications.densenet import DenseNet121
from tensorflow.python.keras.applications.densenet import DenseNet169
from tensorflow.python.keras.applications.densenet import DenseNet201
from tensorflow.python.keras.applications.efficientnet import EfficientNetB0
from tensorflow.python.keras.applications.efficientnet import EfficientNetB1
from tensorflow.python.keras.applications.efficientnet import EfficientNetB2
from tensorflow.python.keras.applications.efficientnet import EfficientNetB3
from tensorflow.python.keras.applications.efficientnet import EfficientNetB4
from tensorflow.python.keras.applications.efficientnet import EfficientNetB5
from tensorflow.python.keras.applications.efficientnet import EfficientNetB6
from tensorflow.python.keras.applications.efficientnet import EfficientNetB7
from tensorflow.python.keras.applications.inception_resnet_v2 import InceptionResNetV2
from tensorflow.python.keras.applications.inception_v3 import InceptionV3
from tensorflow.python.keras.applications.mobilenet import MobileNet
from tensorflow.python.keras.applications.mobilenet_v2 import MobileNetV2
from tensorflow.python.keras.applications.nasnet import NASNetLarge
from tensorflow.python.keras.applications.nasnet import NASNetMobile
from tensorflow.python.keras.applications.resnet import ResNet101
from tensorflow.python.keras.applications.resnet import ResNet152
from tensorflow.python.keras.applications.resnet import ResNet50
from tensorflow.python.keras.applications.resnet_v2 import ResNet101V2
from tensorflow.python.keras.applications.resnet_v2 import ResNet152V2
from tensorflow.python.keras.applications.resnet_v2 import ResNet50V2
from tensorflow.python.keras.applications.vgg16 import VGG16
from tensorflow.python.keras.applications.vgg19 import VGG19
from tensorflow.python.keras.applications.xception import Xception
del _print_function
预定模型种类说明
可以在官网查看https://keras.io/zh/applications/
|
|
|
|
|
|
模型 | 大小 | Top-1 准确率 | Top-5 准确率 | 参数数量 | 深度 |
88 MB | 0.790 | 0.945 | 22,910,480 | 126 | |
528 MB | 0.713 | 0.901 | 138,357,544 | 23 | |
549 MB | 0.713 | 0.900 | 143,667,240 | 26 | |
98 MB | 0.749 | 0.921 | 25,636,712 | - | |
171 MB | 0.764 | 0.928 | 44,707,176 | - | |
232 MB | 0.766 | 0.931 | 60,419,944 | - | |
98 MB | 0.760 | 0.930 | 25,613,800 | - | |
171 MB | 0.772 | 0.938 | 44,675,560 | - | |
232 MB | 0.780 | 0.942 | 60,380,648 | - | |
96 MB | 0.777 | 0.938 | 25,097,128 | - | |
170 MB | 0.787 | 0.943 | 44,315,560 | - | |
92 MB | 0.779 | 0.937 | 23,851,784 | 159 | |
215 MB | 0.803 | 0.953 | 55,873,736 | 572 | |
16 MB | 0.704 | 0.895 | 4,253,864 | 88 | |
14 MB | 0.713 | 0.901 | 3,538,984 | 88 | |
33 MB | 0.750 | 0.923 | 8,062,504 | 121 | |
57 MB | 0.762 | 0.932 | 14,307,880 | 169 | |
80 MB | 0.773 | 0.936 | 20,242,984 | 201 | |
23 MB | 0.744 | 0.919 | 5,326,716 | - | |
343 MB | 0.825 | 0.960 | 88,949,818 |
1.1.2 数据集tensorflow_datasets
(1)安装方式,可以打开anaconda界面,用命令行去添加,也可以直接在pycharm里面为环境添加。
输入activate tensoflow 激活环境
然后输入pip install tensorflow_datasets 安装数据集库
(2)或者在pycharm里面安装库
点击pycharm工程的file-》setting
在输入框内输入tensorflow_datasets,出现安装库,然后选中列表中的tensorflow_datasets,点击左下角的install Package
之后再py文件中输入import tensorflow_datasets as dataset就可以使用数据集了。
(3)运行出现HDF5库和h5py版本不匹配的问题
安装完成后开始编译下载数据,出现如下错误
h5py is running against HDF5 1.10.5 when it was built against 1.10.6, this may cause problems
因为下载数据用到的HDF5和h5py的版本冲突不匹配,可以在anaconda中先输入pip uninstall h5py 然后再输入pip install h5py。下载最新版本的h5py,问题解决。
(4)tensorflow_datasets 包含的数据集名称
可以用下面的语句打印显示
import tensorflow_datasets as data
print(data.list_builders())#打印显示所有的数据集名称,用load加载
['abstract_reasoning', 'accentdb', 'aeslc', 'aflw2k3d', 'ag_news_subset', 'ai2_arc', 'ai2_arc_with_ir', 'amazon_us_reviews', 'anli', 'arc', 'bair_robot_pushing_small', 'bccd', 'beans', 'big_patent', 'bigearthnet', 'billsum', 'binarized_mnist', 'binary_alpha_digits', 'blimp', 'bool_q', 'c4', 'caltech101', 'caltech_birds2010', 'caltech_birds2011', 'cars196', 'cassava', 'cats_vs_dogs', 'celeb_a', 'celeb_a_hq', 'cfq', 'cherry_blossoms', 'chexpert', 'cifar10', 'cifar100', 'cifar10_1', 'cifar10_corrupted', 'citrus_leaves', 'cityscapes', 'civil_comments', 'clevr', 'clic', 'clinc_oos', 'cmaterdb', 'cnn_dailymail', 'coco', 'coco_captions', 'coil100', 'colorectal_histology', 'colorectal_histology_large', 'common_voice', 'coqa', 'cos_e', 'cosmos_qa', 'covid19sum', 'crema_d', 'curated_breast_imaging_ddsm', 'cycle_gan', 'd4rl_mujoco_ant', 'd4rl_mujoco_halfcheetah', 'dart', 'davis', 'deep_weeds', 'definite_pronoun_resolution', 'dementiabank', 'diabetic_retinopathy_detection', 'div2k', 'dmlab', 'dolphin_number_word', 'downsampled_imagenet', 'drop', 'dsprites', 'dtd', 'duke_ultrasound', 'e2e_cleaned', 'efron_morris75', 'emnist', 'eraser_multi_rc', 'esnli', 'eurosat', 'fashion_mnist', 'flic', 'flores', 'food101', 'forest_fires', 'fuss', 'gap', 'geirhos_conflict_stimuli', 'gem', 'genomics_ood', 'german_credit_numeric', 'gigaword', 'glue', 'goemotions', 'gpt3', 'gref', 'groove', 'gtzan', 'gtzan_music_speech', 'hellaswag', 'higgs', 'horses_or_humans', 'howell', 'i_naturalist2017', 'imagenet2012', 'imagenet2012_corrupted', 'imagenet2012_real', 'imagenet2012_subset', 'imagenet_a', 'imagenet_r', 'imagenet_resized', 'imagenet_v2', 'imagenette', 'imagewang', 'imdb_reviews', 'irc_disentanglement', 'iris', 'kitti', 'kmnist', 'lambada', 'lfw', 'librispeech', 'librispeech_lm', 'libritts', 'ljspeech', 'lm1b', 'lost_and_found', 'lsun', 'lvis', 'malaria', 'math_dataset', 'mctaco', 'mlqa', 'mnist', 'mnist_corrupted', 'movie_lens', 'movie_rationales', 'movielens', 'moving_mnist', 'multi_news', 'multi_nli', 'multi_nli_mismatch', 'natural_questions', 'natural_questions_open', 'newsroom', 'nsynth', 'nyu_depth_v2', 'ogbg_molpcba', 'omniglot', 'open_images_challenge2019_detection', 'open_images_v4', 'openbookqa', 'opinion_abstracts', 'opinosis', 'opus', 'oxford_flowers102', 'oxford_iiit_pet', 'para_crawl', 'patch_camelyon', 'paws_wiki', 'paws_x_wiki', 'pet_finder', 'pg19', 'piqa', 'places365_small', 'plant_leaves', 'plant_village', 'plantae_k', 'qa4mre', 'qasc', 'quac', 'quickdraw_bitmap', 'race', 'radon', 'reddit', 'reddit_disentanglement', 'reddit_tifu', 'resisc45', 'robonet', 'rock_paper_scissors', 'rock_you', 's3o4d', 'salient_span_wikipedia', 'samsum', 'savee', 'scan', 'scene_parse150', 'schema_guided_dialogue', 'scicite', 'scientific_papers', 'sentiment140', 'shapes3d', 'siscore', 'smallnorb', 'snli', 'so2sat', 'speech_commands', 'spoken_digit', 'squad', 'stanford_dogs', 'stanford_online_products', 'star_cfq', 'starcraft_video', 'stl10', 'story_cloze', 'sun397', 'super_glue', 'svhn_cropped', 'tao', 'ted_hrlr_translate', 'ted_multi_translate', 'tedlium', 'tf_flowers', 'the300w_lp', 'tiny_shakespeare', 'titanic', 'trec', 'trivia_qa', 'tydi_qa', 'uc_merced', 'ucf101', 'vctk', 'vgg_face2', 'visual_domain_decathlon', 'voc', 'voxceleb', 'voxforge', 'waymo_open_dataset', 'web_nlg', 'web_questions', 'wider_face', 'wiki40b', 'wiki_bio', 'wiki_table_questions', 'wiki_table_text', 'wikiann', 'wikihow', 'wikipedia', 'wikipedia_toxicity_subtypes', 'wine_quality', 'winogrande', 'wmt13_translate', 'wmt14_translate', 'wmt15_translate', 'wmt16_translate', 'wmt17_translate', 'wmt18_translate', 'wmt19_translate', 'wmt_t2t_translate', 'wmt_translate', 'wordnet', 'wsc273', 'xnli', 'xquad', 'xsum', 'xtreme_pawsx', 'xtreme_xnli', 'yelp_polarity_reviews', 'yes_no', 'youtube_vis']
1.1.3 预定义模型和数据集使用实例
import tensorflow as tf
import tensorflow_datasets as data
#(3)定义训练参数和模型对象,数据集对象
num_epochs = 5
batch_size = 19#一批数据的数量
learning_rate = 0.001#学习率
#根据第一个参数名称来下载数据集
print(data.list_builders())#打印显示所有的数据集名称,用load加载
dataset = data.load("tf_flowers",split=data.Split.TRAIN,as_supervised=True)#创建数据源对象,下载数据
dataset=dataset.map(lambda img,label:(tf.image.resize(img,(224,224))/255.0,label)).shuffle(1024).batch(batch_size)
model = tf.keras.applications.MobileNetV2(weights=None,classes=5)#创建模型
optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)#创建优化器,用于参数学习优化
#开始训练参数
#arryindex=np.arange(num_batches)
#arryloss=np.zeros(num_batches)
#通过梯度下降法对模型参数进行训练,优化模型
for e in range(num_epochs):
for images,labels in dataset:
with tf.GradientTape() as tape:
label_pred=model(images,trainable=True)
# 计算损失函数
loss = tf.keras.losses.sparse_categorical_crossentropy(y_true=labels, y_pred=label_pred)
# 计算损失函数的均方根值,表示误差大小
loss = tf.reduce_mean(loss)
print("第%d次训练后:误差%f" % (batch_index, loss.numpy()))
grads = tape.gradient(loss, model.variables)
# 将梯度值调整模型参数
print(label_pred)
optimizer.apply_gradients(grads_and_vars=zip(grads, model.variables))
运行下载数据出现断开连接的错误
Connection broken: ConnectionResetError(10054, '远程主机强迫关闭了一个现有的连接。', None, 10054, None)"
网上因为下载数据太多,时间超时,远程主机以为是受到攻击,自动断开。没找到解决方法。