安装和使用anaconda3+tensorflow遇到的各种问题

原本是想解决在构建CNN网络模型时遇到的一些问题的,结果解决完了这个问题又出现了那个问题,总之就是问题层出不穷,最后只好重新安装这两个了,但是在安装的过程中也出现了很多问题,下面将这些问题汇总一下,没准可能也许会用到呢.

ValueError: Error when checking input: expected conv1d_1_input to have 3 dimensions, but got array with shape (29322, 14)

在conv1d_1层的期望输入值为三维,但是所获取的形状为二维(29332,14)
train.shape=(29322,14)
train_lb.shape=(29322,)
此时要对训练集reshape或者对训练集新增一维
新增一维:
train = np.expand_dims(train, axis=2)
参考博客
https://blog.csdn.net/cjw838982809/article/details/105377410
http://www.111com.net/phper/187111.htm

ValueError: Error when checking target: expected dense_2 to have shape (5,) but got array with shape (1,)
 history = model.fit(train, train_lb, batch_size=100, epochs=30, callbacks=[reduce_lr]) #错误点

在使用categorical_crossentropy作为loss的时候,应该对标签进行one-hot化,因此:
解决办法:
train = keras.utils.to_categorical(train, num_classes)
train_lb = keras.utils.to_categorical(train_lb, num_classes)

详细见:https://blog.csdn.net/ouening/article/details/85065709

ValueError: invalid literal for int() with base 10: 'protocol'

主要原因是因为所处理的数据中包含了数字以外的类型,因此在将字符串转换为整型的时候会报错
解决的办法
https://blog.csdn.net/u012928324/article/details/82186530

python Status(error::Code::INVALID_ARGUMENT, "Unsupported data format") stat

查找资料说是因为版本问题,将tensorflow版本改为1.6(原来版本为2.2.0),还是有错误,又更新为最新版本2.3后此错误没有,但是出现其他错误:

#错误1.
AttributeError: module 'tensorflow' has no attribute 'get_default_graph'
#错误处:
model = Sequential()

原因:版本不兼容?

#错误2.
Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory

https://blog.csdn.net/qq_28193019/article/details/103146116
(按照博客中的方法之后发现还是有错误,错误的提示下还有一句:如果未使用GPU可以忽略此错误)

本来想要解决一下版本问题的,但是没解决
所以最后不得不重新安装
卸载anaconda3

# 删除安装文件夹
sudo rm -rf 路径/anaconda3
# 删除配置文件中的路径分别位于
.bashrc和/etc/profile
# 保存文件
source ~/.bashrc

详细见:https://blog.csdn.net/hang916/article/details/79530108

卸载pycharm

# 删除安装文夹
sudo rm -rf 安装路径
# 删除配置文件
~/.config/JetBrains/PyCharmCE2020.2

详见:https://blog.csdn.net/weixin_43424058/article/details/107977014

linux下安装anaconda3和tensotflow以及如何在pycharm中使用的教程:
下载anaconda3
https://www.anaconda.com/products/individual#linux
bash XXXXX (XXXX为anaconda3的路径)
安装过程中一直选择yes即可
#查看是否安装成功
conda --version
#添加tensorflow的环境,自己可以定义python的版本,这里的版本为3.6
conda create -n tensorflow python=3.6
#激活环境
source activate tensorflow
#安装tensorflow(下面是我安装所需要的安装包,根据自己安装的python环境在下面的网址中选择你需要的安装包)
pip install --ignore-installed --upgrade tensorflow-1.0.0-cp36-cp36m-linux_x86_64.whl
下载tensorflow
https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/tensorflow/
安装完了之后进行测试:
在tensorflow环境下输入python

import tensorflow as tf
hello = tf.constant('first tensorflow')
sess = tf.Session()
print (sess.run(hello))

若输出first tensorflow则表示安装成功.

安装pycharm
#解压
tar zxvf pycharm-professional-2018.2.4.tar.gz
#进入bin文件夹
cd /tmp/pycharm-2018.2.4/bin
#启动pycharm
source pycharm.sh

Tensorflow环境嵌入到编辑器中
参考:https://blog.csdn.net/hitzijiyingcai/article/details/83342905#t12

安装完tensorflow1.1.0版本后出现错误

/home/thm/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:475: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
# 解决办法
pip install numpy==1.16.0

https://blog.csdn.net/bigdream123/article/details/99467316
(按照第一条评论的方法,但是在pycharm里面运行程序的时候还是会有错误,想到之前是在anaconda3环境下面直接安装numpy1.16.0的版本,因此换了一种方法,在pycharm/settings里面查看之前安装的版本并没有生效,所以重新安装1.16.0版本之后运行就可以了)

ImportError: cannot import name 'Sequential'
#出错点
from keras import Sequential

原因是因为tensotflow的版本太低了,应该升级到2.0.0+的版本
解决办法:

from tensorflow.keras import Sequential
pip install tensorflow==2.2.0

出现了另外两个错误
1）出错句改为"from tensorflow.contrib.rnn.python.ops import rnn_cell"(未解决)
2）或重新安装tensorflow:

pip install tensorflow-gpu==0.12.1
pip install tensorflow==0.12.1(解决)

KeyError: "None of [Int64Index([  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,\n            ...\n            826, 828, 829, 830, 831, 833, 834, 835, 836, 837],\n           dtype='int64', length=670)] are in the [columns]"

参考博客:
https://blog.csdn.net/weixin_40238600/article/details/96604879
是因为索引问题,但是我解决下面这个问题后这个问题就解决了

问题:

ModuleNotFoundError: No module named 'tensorflow.contrib.tensorboard'

参考博客
https://stackoverflow.com/questions/60983118/no-module-named-tensorflow-contrib
原本的keras版本为2.0.6,应该将其版本升级为2.3.0(直接在pycharm里面找到keras安装或者pip install keras==2.3.0)

归根结底以上的问题都是因为版本问题(不兼容):

反复安装n次之后…
此时的版本分别为:tensorflow = 1.14.0 keras = 2.2.5 numpy =1.16.0
虽然能够正常运行,但是控制台会报很多警告
问题1:
https://blog.csdn.net/weixin_42081389/article/details/98190324
版本不对,将1.14.0改为1.5.0(keras也要改为对应的版本)

WARNING:tensorflow:From /home/thm/anaconda3/envs/tensorflow/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:66: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

WARNING:tensorflow:From /home/thm/anaconda3/envs/tensorflow/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:541: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /home/thm/anaconda3/envs/tensorflow/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:4432: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

WARNING:tensorflow:From /home/thm/anaconda3/envs/tensorflow/lib/python3.6/site-packages/keras/optimizers.py:793: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

WARNING:tensorflow:From /home/thm/anaconda3/envs/tensorflow/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3576: The name tf.log is deprecated. Please use tf.math.log instead.

问题二:tensorflow不是已源码安装的,因此会提示少SSE4.1 SSE4.2 AVX AVX2 FMA等指令,可以添加以下代码忽视:
import os
os.environ[“TF_CPP_MIN_LOG_LEVEL”]=“3”
或者用源码重新安装tensorflow
https://blog.csdn.net/weixin_43159148/article/details/89609870

2020-09-21 10:54:32.381543: I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in non-MKL-DNN operations, rebuild TensorFlow with the appropriate compiler flags.
2020-09-21 10:54:32.405263: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3399905000 Hz
2020-09-21 10:54:32.405476: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x563a901ad280 executing computations on platform Host. Devices:
2020-09-21 10:54:32.405495: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2020-09-21 10:54:32.405960: I tensorflow/core/common_runtime/process_util.cc:115] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
2020-09-21 10:54:32.469443: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.

问题三:(或者直接忽略,并不影响运行)
参考博客
https://blog.csdn.net/qq_33373858/article/details/82771740

WARNING:tensorflow:From /home/thm/PycharmProjects/code/Extraction/4_Attacks_detection/step/6_mymodel.py:100: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.convert_variables_to_constants`
WARNING:tensorflow:From /home/thm/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/graph_util_impl.py:270: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`

问题四:
如下图展示,有很多冗余信息输出在控制台中,如果想要减少这些信息的输出,可以:

import os
os.environ['KMP_WARNINGS'] = '0'
#或者
#os.environ['KMP_WARNINGS'] = 'off'

https://www.it1352.com/1525953.html
https://stackoverflow.com/questions/57385766/disable-tensorflow-log-information