BuildStrategy
fluid.BuildStrategy:https://www.paddlepaddle.org.cn/documentation/docs/zh/api_cn/fluid_cn/BuildStrategy_cn.html
计算图的建造方法
import numpy as np
import paddle.fluid as fluid
import os
# 设置使用4个CPU
os.environ["CPU_NUM"]='2'
use_cuda=True
place=fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace()
# 输入是1个维度
data = fluid.layers.data(name="x", shape=[1], dtype="float32")
# 映射到10个维度
hidden = fluid.layers.fc(input=data, size=10)
loss = fluid.layers.mean(hidden)
fluid.optimizer.SGD(learning_rate=0.01).minimize(loss)
start_p=fluid.default_startup_program()
main_p=fluid.default_main_program()
exe=fluid.executor.Executor(place=place)
exe.run(start_p)
# 创建 创建策略
build_strategy = fluid.BuildStrategy()
# 为True时可用于减少总内存消耗,False表示不使用
build_strategy.memory_optimize = True
# build_strategy.reduce_strategy = fluid.BuildStrategy.ReduceStrategy.Reduce
build_strategy.reduce_strategy = fluid.BuildStrategy.ReduceStrategy.AllReduce
CompiledProgram
paddle.fluid.CompiledProgram:https://www.paddlepaddle.org.cn/documentation/docs/zh/api_cn/fluid_cn/CompiledProgram_cn.html
CompiledProgram支持根据 build_strategy 的配置,将输入的Program进行转换和优化(比如with_data_parallel后支持多CPU/GPU并行)
# 对原来的main_p,进行转换升级
# Equivalence fluid.compiler.CompiledProgram()
main_program = fluid.CompiledProgram(main_p)
# 升级后的program,可以进行高级的策略配置build_strategy,和并行计算places=[,,]
# 没有GPU换成fluid.CPUPlace,这里设置在2个GPU上并行计算
main_program = main_program.with_data_parallel(loss_name=loss.name,
build_strategy=build_strategy,
places=[fluid.CUDAPlace(0),fluid.CUDAPlace(1)])
# 喂的数据是10个1的维度的数据
x = np.random.random(size=(10, 1)).astype('float32')
# 那么h就是10个10维度的映射
l,h=exe.run(program=main_program,
feed={'x':x},
fetch_list=[loss.name,hidden.name])
# 损失是多个设备返回的,所以实际计算的时候记得加个np.mean
#l是对h求平均的,但是使用了2个CPU/GPU,所以l是2份数据:array([-0.1789905 , -0.12210217], dtype=float32)
# h.shape :(10, 10)