1. python 有关
1.1 virtualenv
用于创建一个隔离的Python环境,一个专属于项目的python环境。用virtualenv 来保持一个干净的环境非常有用
https://virtualenv.pypa.io/en/latest/
http://www.virtualenv.org/en/latest/index.html
一些博文:
用pyenv和virtualenv搭建单机多版本python虚拟开发环境 http://www.it165.net/pro/html/201405/13603.html
1.2 pyenv
https://github.com/yyuu/pyenv-virtualenv
## INSTALL pyenv
git clone https://github.com/yyuu/pyenv.git ~/.pyenv
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile
echo 'eval "$(pyenv init -)"' >> ~/.bash_profile
## INSTALL pyenv-virtualenv
git clone https://github.com/yyuu/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv
echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bash_profile
--------
$ pyenv install 2.7.11
$ pyenv rehash
To create a virtualenv for the Python version used with pyenv, run pyenv virtualenv, specifying the Python version you want and the name of the virtualenv directory. For example,
$ pyenv virtualenv 2.7.10 my-virtual-env-2.7.10
will create a virtualenv based on Python 2.7.10 under ~/.pyenv/versions in a folder called my-virtual-env-2.7.10.
$ pyenv virtualenv 2.7.11 ve-ansible-2.7.11
http://my.oschina.net/lionets/blog/267469
CONFIGURE_OPTS="--enable-shared" pyenv install -v 2.7.5
1.3 buildout
http://www.buildout.org/en/latest/
1.4 devpi
1.5 fabric
用于最小化将本地开发的代码部署到远程服务器上这一繁琐的重复性工作。
fabric 用来自动化远程部署项目,非常的方便。可以根据需要在本地、远程依次执行一系列shell 命令、程序等。比如从代码库更新代码,执行数据迁移脚本,重启服务进程,完成自动化的部署。
https://pypi.python.org/pypi/fabric-virtualenv/
1.6 pip
最先进的Python软件包管理工具。pip 用来解决项目依赖问题。将项目所有依赖的第三方库写在一个requirements.txt 中用pip 批量安装。一般和virtualenv 配合使用,将所有包安装在virtualenv 的环境中。
https://github.com/mitsuhiko/pipsi 优点是可以把每一个命令都隔离开,不会再污染系统的包了,包和包之间的依赖也不会冲突了。
1.7 ansible
ansible是新出现的自动化运维工具,基于Python开发,集合了众多运维工具(puppet、cfengine、chef、func、fabric)的优点,实现了批量系统配置、批量程序部署、批量运行命令等功能。ansible是>基于模块工作的,本身没有批量部署的能力。真正具有批量部署的是ansible所运行的模块,ansible只是提供一种框架。主要包括:
(1)、连接插件connection plugins:负责和被监控端实现通信;
(2)、host inventory:指定操作的主机,是一个配置文件里面定义监控的主机;
(3)、各种模块核心模块、command模块、自定义模块;
(4)、借助于插件完成记录日志邮件等功能;
(5)、playbook:剧本执行多个任务时,非必需可以让节点一次性运行多个任务。
特性
(1)、no agents:不需要在被管控主机上安装任何客户端;
(2)、no server:无服务器端,使用时直接运行命令即可;
(3)、modules in any languages:基于模块工作,可使用任意语言开发模块;
(4)、yaml,not code:使用yaml语言定制剧本playbook;
(5)、ssh by default:基于SSH工作;
(6)、strong multi-tier solution:可实现多级指挥。
优点
(1)、轻量级,无需在客户端安装agent,更新时,只需在操作机上进行一次更新即可;
(2)、批量任务执行可以写成脚本,而且不用分发到远程就可以执行;
(3)、使用python编写,维护更简单,ruby语法过于复杂;
(4)、支持sudo。
To install from source.
$ git clone git://github.com/ansible/ansible.git --recursive
$ cd ./ansible
Using Bash:
$ source ./hacking/env-setup
Using Fish:
$ . ./hacking/env-setup.fish
If you want to suppress spurious warnings/errors, use:
$ source ./hacking/env-setup -q
If you don’t have pip installed in your version of Python, install pip:
$ sudo easy_install pip
Ansible also uses the following Python modules that need to be installed [1]:
$ sudo pip install paramiko PyYAML Jinja2 httplib2 six
Note when updating ansible, be sure to not only update the source tree, but also the “submodules” in git which point at Ansible’s own modules (not the same kind of modules, alas).
$ git pull --rebase
$ git submodule update --init --recursive
Once running the env-setup script you’ll be running from checkout and the default inventory file will be /etc/ansible/hosts. You can optionally specify an inventory file (see Inventory) other than /etc/ansible/hosts:
$ echo "127.0.0.1" > ~/ansible_hosts
$ export ANSIBLE_INVENTORY=~/ansible_hosts
NOTE ANSIBLE_INVENTORY is available starting at 1.9 and substitutes the deprecated ANSIBLE_HOSTS
You can read more about the inventory file in later parts of the manual.
Now let’s test things with a ping command:
$ ansible all -m ping --ask-pass
$ ansible all -m ping -i ~/ansible_hosts --ask-pass
You can also use “sudo make install” if you wish.
Ansible is not just about running commands, it also has powerful configuration management and deployment features.
2. golang 有关
2.1 surfer
surfer [high level concurrency downloader] 是一款Go语言编写的高并发爬虫下载器,拥有surf与phantom两种下载内核,支持固定UserAgent自动保存cookie与随机大量UserAgent禁用cookie两种模式,高度模拟浏览器行为,可实现模拟登录等功能。
http://git.oschina.net/henrylee/surfer
2.2 etcd
2016-05
3. rust 有关
rust 中文教程 https://www.gitbook.com/book/kaisery/rust-book-chinese/details
3.1 log
2016-05
3.2 命令行处理工具
2016-05
4. mysql 有关
mysql 支持json了 http://dev.mysql.com/doc/refman/5.7/en/json.html
各版本 MySQL 并行复制的实现及优缺点 http://www.oschina.net/news/70126/mysql-compare
5. php
5.1 phpenv
phpenv https://github.com/phpenv/phpenv
php-build https://github.com/php-build/php-build
## INSTALL phpenv
cd
git clone git://github.com/phpenv/phpenv.git .phpenv
echo 'export PATH="$HOME/.phpenv/bin:$PATH"' >> ~/.bash_profile
echo 'eval "$(phpenv init -)"' >> ~/.bash_profile
exec $SHELL
phpenv rehash
## INSTALL phpenv-build
git clone git://github.com/php-build/php-build.git $HOME/.phpenv/plugins/php-build
You can list the available PHP releases by running:
$ phpenv install --releases
To build one of the listed releases run:
$ phpenv install php-5.3.20
To install multiple builds of the same release simply add a unique name for your additional builds after the release identifier.
$phpenv install php-5.3.20 debug
Will use the configuration options source file located at .phpenv/etc/php-5.3.20-debug.Darwin.source if installing on a Mac OS X environment and installs the version to .phpenv/versions/5.3.20-debug.
The build is kept in tact at location phpenv/php-src to simplify fault finding and allowing you to continue the installation process in the event of a failed build.
To continue from a previous step in the installation process use the --continue option.
$phpenv install php-5.3.20 -c 4
To start from the configuring stage of the installation process and rerun ./configure using the updated information from your configuration options source file.
When restarting an installation from scratch it may be useful to clean previously build and generated files, use
$phpenv install --clean
When installing a different release version it may be useful to do a deep clean and purge all previously build and generated files including those from custom extension located at .php-env/php-ext and purge the ccache (if used), use
$phpenv install --deep-clean
Running phpenv install with no arguments will output its usage, for detailed help documentation, use
$phpenv install --help
phpenv global
Sets the global version of PHP to be used in all shells by writing the version name to the ~/.phpenv/version file. This version can be overridden by a per-project .phpenv-version file, or by setting the PHPENV_VERSION environment variable.
$ phpenv global 5.4.0
The special version name system tells phpenv to use the system PHP (detected by searching your $PATH).
When run without a version number, phpenv global reports the currently configured global version.
phpenv local
Sets a local per-project PHP version by writing the version name to a .phpenv-version file in the current directory. This version overrides the global, and can be overridden itself by setting the PHPENV_VERSION environment variable or with the phpenv shell command.
$ phpenv local 5.3.8
When run without a version number, phpenv local reports the currently configured local version. You can also unset the local version:
$ phpenv local --unset
php logging
z. 其他
好程序员 + 烂技术 = 痛苦 http://www.oschina.net/news/69989/the-miserable-programmer-paradox