poeticHadoop的博客

原创 Linux配置matplotlib中文显示

在python工程的虚拟空间环境里：1.SimHei.ttf 文件拷贝到指定的ttf文件夹之下。2.在matplotlibrc 末尾添加如下几行。3.清除缓存 rm -rf /root/.cache/matplotlib 文件夹。4.reboot服务器。参考链接

原创 运行JoinQuant的环境

下载 聚宽的python环境。pip install jqdatasdk所有的下载日志信息：Microsoft Windows [版本 10.0.19042.1288]© Microsoft Corporation。保留所有权利。(venv) C:\Users\IMO_0117\PycharmProjects\JoinQuantity\Demo1>pip install jqdatasdkCollecting jqdatasdkDownloading jqdatasdk-1.8.10-

原创 Mac OS 官网装64位 Python3.5.2

Python 3.5.2 - 2016-06-27 (64位)Download Mac OS X 64-bit/32-bit installer (32位)Download Mac OS X 32-bit i386/PPC installer 有 64-bit 这个就是 64 位的。

原创 [TensorFlow] demo1 完整的代码和运行结果

import tensorflow as tfimport numpy as npx_data = np.random.rand(100).astype(np.float32)y_data = x_data*0.1 + 0.3#print(x_data)#print(y_data)Weights = tf.Variable(tf.random_uniform([1], -1.0,

原创 [TensorFlow] demo1 tf.initialize_all_variables函数换成新的tf.global_variables_initializer函数

WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow/python/util/tf_should_use.py:107: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be re

原创 [TensorFlow] demo1 tf.square 函数

tensorflow/python/ops/math_ops.py.就是求平方的函数库

原创 [TensorFlow] demo1 tf.zeros 函数

定义的文件路径：tensorflow/python/ops/array_ops.py.举个例子：tf.zeros([3, 4], tf.int32)  # [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]可以看出，创建一个数组并初始化它们都为0Args:shape: A list of integers, a tuple of integ

原创 [TensorFlow] demo1 tf.random_uniform 函数

tensorflow/tensorflow/python/ops的 random_ops.py 文件有定义# pylint: disable=protected-accessdef random_normal(shape, mean=0.0, stddev=1.0, dtype=dty

原创 [TensorFlow] git clone 官方的tensorflow代码

原创 [TensorFlow] demo1 导入tensorflow相关的包，并生成Weights和biases

import tensorflow as tfimport numpy as npx_data = np.random.rand(100).astype(np.float32)y_data = x_data*0.1 + 0.3#print(x_data)#print(y_data)Weights = tf.Variable(tf.random_uniform([1], -1.0, 1.

原创 [TensorFlow] demo1 创建100个float32的随机数y_data

import numpy as npx_data = np.random.rand(100).astype(np.float32)y_data = x_data*0.1 + 0.3print(x_data)print(y_data)

原创 [TensorFlow] demo1 创建100个float32的随机数x_data

import numpy as npx_data = np.random.rand(100).astype(np.float32)print(x_data)运行效果如下：

原创 [TensorFlow] python3-tk 库的安装作为tensorflow的可视化库

jhczz@vaio:~$ suPassword: root@vaio:/home/jhczz# apt-get install python3-tkReading package lists... DoneBuilding dependency tree       Reading state information... DoneThe following packag

原创 [TensorFlow] windows 安装TensorFlow

C:\Users\Administrator>pip install tensorflowCollecting tensorflow  Downloading tensorflow-1.4.0-cp36-cp36m-win_amd64.whl (28.3MB)    100% |████████████████████████████████| 28.3MB 72kB/sColle

原创 [TensorFlow] Ubuntu16.04 更新最新的pip3 正确方式

root@xxx:/home/czz# pip3 install --upgrade pipCollecting pip  Using cached pip-9.0.1-py2.py3-none-any.whlInstalling collected packages: pip  Found existing installation: pip 8.1.1    Not uni

原创 [TensorFlow] ubuntu16.04 pip3安装 TensorFlow

[08:40:08]root@xxx:/home/czz# pip3 install tensorflow[08:40:15]Collecting tensorflow[08:40:16]  Downloading tensorflow-1.4.1-cp35-cp35m-manylinux1_x86_64.whl (40.7MB)[08:42:15]    100% |||||||||

原创 [TensorFlow] TensorFlow学习的方向

教程本节包含演示如何在 TensorFlow 中执行特定任务的教程。如果您是 TensorFlow 的新手，那么在阅读这些教程之前，我们建议阅读“入门”一节中的文档。[1]以下教程解释了 TensorFlow 系统上CPU和GPU的交互：(1)使用GPU[2]以下教程介绍了图像识别的不同方面：(1)图像识别，它引入了图像识别领域和一个用于识别图像的模型（Incept

原创 ubuntu16.04安装python3 selenium

# -*- coding: utf-8 -*-from selenium import webdriverfrom selenium.webdriver.common.keys import Keysdef open100Chrome(): begin = 1 end = 3 for flag in range(begin, end): brow

原创 Ubuntu 16.04 安装 redis 分布式爬虫

(ENV3.5) jhczz@vaio:~/work/Python/Vietualenv/ENV3.5$ pip3 install redisCollecting redis Downloading redis-2.10.6-py2.py3-none-any.whl (64kB) 100% |████████████████████████████████| 71kB 180kB/s

原创 ubuntu 16.04 安装mysqlclient数据库驱动

jhczz@vaio:~/work/MySQL$ sudo apt-get install libmysqlclient-devReading package lists... DoneBuilding dependency tree Reading state information... DoneThe following packages were automatical

原创 ubuntu16.04 破解navicat for MySQL 教程

jhczz@vaio:~$ cd .navicat64/jhczz@vaio:~/.navicat64$ lltotal 304drwxrwxr-x 5 jhczz jhczz 4096 Sep 12 13:42 ./drwx------ 31 jhczz jhczz 12288 Sep 12 13:47 ../drwxrwxr-x 2 jhczz jhczz 4096 S

原创 Ubuntu 16.04 安装 MySQL

jhczz@vaio:~/work/MySQL$ sudo apt-get install mysql-server[sudo] password for jhczz: Reading package lists... DoneBuilding dependency tree Reading state information... DoneThe following pac

原创 创建Scrapy项目，Ubuntu 16.04 Python3.5 pip3

jhczz@vaio:~$ cd work/Python/Vietualenv/jhczz@vaio:~/work/Python/Vietualenv$ virtualenv -p /usr/bin/python3.5 ENV3.5Running virtualenv with interpreter /usr/bin/python3.5Using base prefix '/usr'Ne

原创 The Google File System : part8 RELATED WORK and CONCLUSIONS and ACKNOWLEDGMENTS

8.RELATED WORKLike other large distributed file systems such as AFS,GFS provides a location independent namespace which enables data to be moved transparently for load balance or fault tolerance. 

原创 The Google File System : part7 EXPERIENCES

7. EXPERIENCESIn the process of building and deploying GFS, we have experienced a variety of issues, some operational and some technical.Initially, GFS was conceived as the backend file system f

原创 The Google File System : part6 MEASUREMENTS

6. MEASUREMENTSIn this section we present a few micro-benchmarks to illustrate the bottlenecks inherent in the GFS architecture and implementation, and also some numbers from real clusters in use at

原创 The Google File System : part5 FAULT TOLERANCE AND DIAGNOSIS

5.FAULT TOLERANCE AND DIAGNOSISOne of our greatest challenges in designing the system is dealing with frequent component failures. The quality and quantity of components together make these proble

原创 The Google File System : part4 MASTER OPERATION

4. MASTER OPERATIONThe master executes all namespace operations. In addition, it manages chunk replicas throughout the system: it makes placement decisions, creates new chunks and hence replicas

原创 The Google File System : part3 SYSTEM INTERACTIONS

3. SYSTEM INTERACTIONSWe designed the system to minimize the master’s involvement in all operations. With that background, we now describe how the client, master, and chunkservers interact to impl

原创 The Google File System : part2 DESIGN OVERVIEW

2.DESIGN OVERVIEW2.1 AssumptionsIn designing a file system for our needs, we have been guided by assumptions that offer both challenges and opportunities. We alluded to some key observations ear

原创 The Google File System : part1 ABSTRACT and INTRODUCTION

ABSTRACTWe have designed and implemented the Google File System, a scalable distributed file system for large distributed data-intensive applications. It provides fault tolerance while running on

原创 Bigtable: A Distributed Storage System for Structured Data : part11 Conclusions and Acknowledgements

11 ConclusionsWe have described Bigtable, a distributed system for storing structured data at Google. Bigtable clusters have been in production use since April 2005, and we spent roughly seven per

原创 Bigtable: A Distributed Storage System for Structured Data : part10 Related Work

10 Related WorkThe Boxwood project [24] has components that overlap in some ways with Chubby, GFS, and Bigtable, since it provides for distributed agreement, locking, distributed chunk storage, and

原创 Bigtable: A Distributed Storage System for Structured Data : part9 Lessons

9 LessonsIn the process of designing, implementing, maintaining,and supporting Bigtable, we gained useful experience and learned several interesting lessons.One lesson we learned is that large dis

原创 Bigtable: A Distributed Storage System for Structured Data : part8 Real Applications

8 Real ApplicationsAs of August 2006, there are 388 non-test Bigtable clusters running in various Google machine clusters, with a combined total of about 24,500 tablet servers. Table 1 shows a

原创 Bigtable: A Distributed Storage System for Structured Data : part7 Performance Evaluation

7 Performance EvaluationWe set up a Bigtable cluster with N tablet servers to measure the performance and scalability of Bigtable as N is varied. The tablet servers were configured to use 1 GB of

原创 Bigtable: A Distributed Storage System for Structured Data : part6 Refinements

6 RefinementsThe implementation described in the previous section required a number of refinements to achieve the high performance, availability, and reliability required by our users. This sectio

原创 Bigtable: A Distributed Storage System for Structured Data : part5 Implementation

5 ImplementationThe Bigtable implementation has three major components: a library that is linked into every client, one master server, and many tablet servers. Tablet servers can be dynamically

原创 Bigtable: A Distributed Storage System for Structured Data : part4 Building Blocks

4 Building BlocksBigtable is built on several other pieces of Google infrastructure. Bigtable uses the distributed Google File System (GFS) to store log and data files. A Bigtable cluster typica

原创 Bigtable: A Distributed Storage System for Structured Data : part3 API

3 APIThe Bigtable API provides functions for creating and deleting tables and column families. It also provides functions for changing cluster, table, and column family metadata, such as access co