如何将TensorFlow模型封装成服务（网页服务）

版本： 

tensorflow == 1.14.0

python == 3.7

Flask == 1.1.1

对于下面程序，x为传入参数，封装成网页

import tensorflow as tf

def main():

    x = tf.placeholder("float")
    y = x * 2
    with tf.Session() as sess:
        print(sess.run(y, feed_dict={x:2}))

if __name__ == '__main__':
    main()

方法一：（函数传参）

main.py

import tensorflow as tf

def main(m):

    x = tf.placeholder("float")
    y = x * 2
    with tf.Session() as sess:
        res = sess.run(y, feed_dict={x:m})
        return str(res)

run.py

from flask import Flask, request
from main import main

app = Flask(__name__)

@app.route('/')
def index():
    x = int(request.args.get('x', 0))
    return main(x)

if __name__ == '__main__':
    app.run()

运行结果： 

 

该方法比较简单，多线程运行互不干扰，但每次调用需要重现构建张量图，模型复杂时开销较大

方法二： （管道多进程）

main.py

import tensorflow as tf

def main(_):

    def get_pipe():
        from multiprocessing.managers import BaseManager
        BaseManager.register('get_pipe')
        m = BaseManager(address=('localhost', 30030), authkey=b'awadffwea')
        m.connect()
        return m.get_pipe()

    x = tf.placeholder("float")
    y = x * 2
    with tf.Session() as sess:
        while True:
            try:
                pipe = get_pipe()
                m = pipe.recv()
                res = sess.run(y, feed_dict={x:m})
                pipe.send(str(res))
            except:
                pipe.send('error')

run.py

from flask import Flask, request
import tensorflow as tf
from main import main
from multiprocessing.managers import BaseManager
import multiprocessing
parent_conn, child_conn = multiprocessing.Pipe()

app = Flask(__name__)

def get_parent_pipe():
    from multiprocessing.managers import BaseManager
    BaseManager.register('get_parent_pipe')
    m = BaseManager(address=('localhost', 30030), authkey=b'awadffwea')
    m.connect()
    return m.get_parent_pipe()

def register():
    BaseManager.register('get_pipe', callable=lambda: child_conn)
    BaseManager.register('get_parent_pipe', callable=lambda: parent_conn)
    m = BaseManager(address=('localhost', 30030), authkey=b'awadffwea')
    s = m.get_server()
    s.serve_forever()

@app.route('/')
def index():
    x = int(request.args.get('x', 0))
    parent_conn = get_parent_pipe()
    parent_conn.send(x)
    return parent_conn.recv()

if __name__ == '__main__':
    multiprocessing.Process(target=register, args=()).start()
    multiprocessing.Process(target=tf.app.run, args=(main,)).start()
    app.run(threaded=False)

运行结果：

该方法不需要每次调用构建张量图，但是程序比较复杂，需要预防死锁，只能串行调用

方法三：（协程）

main.py

#tensorflow == 1.14.0 python == 3.7 Flask == 1.1.1
import tensorflow as tf

def main():

    x = tf.placeholder("float")
    y = x * 2
    with tf.Session() as sess:
        while True:
            try:
                m = yield
                res = sess.run(y, feed_dict={x:m})
                yield str(res)
            except:
                yield 'error'

if __name__ == '__main__':
    main()

run.py

from flask import Flask, request
import tensorflow as tf
from main import main

app = Flask(__name__)
m = main()
next(m)

@app.route('/')
def index():
    x = int(request.args.get('x', 0))
    res = m.send(x)
    next(m)
    return res

if __name__ == '__main__':
    app.run(threaded=False)

运行结果：

注：

1、yield在接收参数的同时将当前运行结果返回出去，所以第一次必须调用next(m)开启迭代器，同时将第一个yield的结果返回

2、run.py中的程序运行，将参数传入第一个yield中，在main.py执行程序，将结果返回到第二个yield中，如图所示：

3、多线程调用需要加锁，否则出现ValueError: generator already executing

参考资料:

https://blog.csdn.net/buptgshengod/article/details/106861118

https://blog.csdn.net/stu_py/article/details/115067192