1 构建含指定包的python镜像
1.1 下载基础镜像
docker pull python:3.6
(1)创建容器【创建完以后,会直接进入容器】
docker run -it python:3.6 /bin/bash
(2)用-it创建的容器,退出后,会停止运行
[root@cf18727fd569 /]# exit退出容器
(3)查看安装的包
[root@cf18727fd569 /]python3
[root@cf18727fd569 /]pip list
1.2 构建dockerfile
文件dockerfile
FROM python:3.6
MAINTAINER bingbing <123456789@qq.com>
RUN pip3 install pip -U
RUN pip3 config set global.index-url http://mirrors.aliyun.com/pypi/simple
RUN pip3 config set install.trusted-host mirrors.aliyun.com
RUN pip3 install pandas==1.0.0
RUN pip3 install scikit-learn==0.24.2
RUN pip3 install redis==4.0.0
创建并存储用于后期加载
docker build -f dockerfile -t mypython36:1.0 .
docker save -o mypython36.tar mypython36:1.0打包
docker load -i mypython36.tar加载
2 外部调用容器里的命令
2.1 宿主机中创建文件
文件helloworld.py位于宿主机/root/myuse的下面
mkdir /root/myuse
vi helloworld.py
print("hello,world")
chmod a+x helloworld.py
2.2 调用方式
docker run
-v /root/myuse:/usr/src/myapp
-w /usr/src/myapp
mypython36:1.0
python3 helloworld.py
命令说明:
(1)-v /root/myuse:/usr/src/myapp:
将主机中目录下的内容挂载到容器的/usr/src/myapp。
映射时,宿主机目录有什么,容器对应目录中也有什么。
(2)-w /usr/src/myapp:
指定容器的/usr/src/myapp目录为工作目录。
(3)mypython36:1.0
镜像
(4)python3 helloworld.py:
使用容器的python命令来执行工作目录中的helloworld.py文件。
3 python调用shell命令
直接调用容器中的内容
3.1 subprocess.Popen()方式
访问返回到stdout的内容。
3.1.1 use.py
import sys
if __name__ == "__main__":
print("直接输出")
3.1.2 main.py
import subprocess
cmd_list = ["docker","run","-v","/root/myuse:/usr/src/myapp","-w","/usr/src/myapp","mypython36:1.0","python3","use.py"]
child = subprocess.Popen(cmd_list,stdout=subprocess.PIPE)
print(child.stdout.read().decode())
3.1.3 main2.py
x = sys.stdin
for line in x:
print(line)
break
送到容器的stdin。
import subprocess
with open("use.txt","r",encoding="utf-8") as fr:
data_json = fr.read() # 字符串
data_bytes = data_json.encode(encoding="utf-8")
cmd_str = "echo '{}'|docker run -i -v /root/myuse:/usr/src/myapp -w /usr/src/myapp mypython36:2.0 python3 __init__.py".format(data_json)
child = subprocess.Popen(cmd_str,shell=True,stdout=subprocess.PIPE)
re = child.stdout.read().decode()
print(re)
3.2 sys.argv[1]方式
直接传参的方式。
3.2.1 use.py
import sys
if __name__ == "__main__":
print("子进程",sys.argv[1])
3.2.2 main.py
import subprocess
with open("use.txt","r",encoding="utf-8") as fr:
data_json = fr.read() # 字符串
cmd_str = "docker run -v /root/myuse:/usr/src/myapp -w /usr/src/myapp mypython36:1.0 python3 use.py '{}'".format(data_json)
re = subprocess.getoutput(cmd_str)
print("主进程",re)
3.3 os.popen()方式
3.3.1 use.py
import sys
x = sys.stdin
for line in x:
print("receive",line)
break # 执行一次后退出循环,否则会一直持续等待运行
3.3.2 main.py
一、调用宿主机的环境
import os
with open("use.txt","r",encoding="utf-8") as fr:
data_json = fr.read() # 字符串
re = os.popen("echo '{}'|python3 test.py".format(data_json)).read()
print(re)
3.4 docker容器从stdin读取数据
需要使--interactive或-i以交互模式运行容器。
3.4.1 use.py
import sys
x = sys.stdin
for line in x:
print("receive",line)
break # 执行一次后退出循环,否则会一直持续等待运行
3.4.2 main.py
import os
with open("use.txt","r",encoding="utf-8") as fr:
data_json = fr.read() # 字符串
re = os.popen("echo '{}'| docker run -i -v /root/myuse:/usr/src/myapp -w /usr/src/myapp mypython36:1.0 python3 test.py".format(data_json)).read()
print(re)
4 图计算结构
4.1 sys.argv[1]传参方式
def main():
line = sys.argv[1]
start_time = time.time()
graph_data = Graph.from_json(line)
try:
g = graph_data.graph
x = g.topoSort()
e = Executor(g.nodes, g.edges)
e.run(x, graph_data)
except Exception as err: # 这里会触发 Too broad exception clause 校验
message = "".join(traceback.format_exc())
graph_data.exceptions.append("设备{0}数据处理异常: {1}。".format(
graph_data.datasource["config"]["devid"], message))
try:
graph_data.timeconsuming = str(time.time() - start_time)
sys.stdout.write(graph_data.to_json() + '\n')
except Exception as err:
message = "".join(
traceback.format_exception_only(
err.__class__, err))
graph1 = Graph.from_json(line)
graph1.exceptions.append("设备{0}数据处理异常: {1}。".format(
graph1.datasource["config"]["devid"], message))
sys.stdout.write(graph1.to_json() + '\n')
if __name__ == '__main__':
main()