php整合apollo配置中心——k8s从入门到高并发系列教程（十六）

 ads：

关注以下公众号查看更多文章

安装apollo

mysql导入apollo相关数据库

apolloportaldb.sql

apolloconfigdb.sql

helm 导入apollo仓库源

helm repo add apollo https://charts.apolloconfig.com

部署 configService 和 adminService 

helm install apollo-service-dev \
    --set configdb.host=192.168.205.1 \
    --set configdb.dbName=ApolloConfigDB \
    --set configdb.userName=apollo \
    --set configdb.password=apollo \
    --set configdb.connectionStringProperties="characterEncoding=utf8&useSSL=false" \
    --set configdb.service.enabled=true \
    --set configService.replicaCount=1 \
    --set adminService.replicaCount=1 \
    -n apollo \
    apollo/apollo-service

执行完后可以在控制台看到如下输出，可以看到config service地址是 http://apollo-service-dev-apollo-configservice.apollo:8080

Meta service url for current release:
  echo http://apollo-service-dev-apollo-configservice.apollo:8080

For local test use:
  export POD_NAME=$(kubectl get pods --namespace apollo -l "app=apollo-service-dev-apollo-configservice" -o jsonpath="{.items[0].metadata.name}")
  echo http://127.0.0.1:8080
  kubectl --namespace apollo port-forward $POD_NAME 8080:8080

Urls registered to meta service:
Config service: http://apollo-service-dev-apollo-configservice.apollo:8080
Admin service: http://apollo-service-dev-apollo-adminservice.apollo:8090

部署portal服务 

helm install apollo-portal \
    --set portaldb.host=192.168.205.1 \
    --set portaldb.dbName=ApolloPortalDB \
    --set portaldb.userName=apollo \
    --set portaldb.password=apollo \
    --set portaldb.connectionStringProperties="characterEncoding=utf8&useSSL=false" \
    --set portaldb.service.enabled=true \
    --set config.envs="dev" \
    --set config.metaServers.dev=http://apollo-service-dev-apollo-configservice.apollo:8080 \
    --set replicaCount=1 \
    -n apollo \
    apollo/apollo-portal

执行完后可以在控制台看到如下输出

Portal url for current release:
  export POD_NAME=$(kubectl get pods --namespace apollo -l "app=apollo-portal" -o jsonpath="{.items[0].metadata.name}")
  echo "Visit http://127.0.0.1:8070 to use your application"
  kubectl --namespace apollo port-forward $POD_NAME 8070:8070

打开如下地址，可以看到apollo管理界面

http://apollo-portal.apollo.svc.cluster.local:8070/signin

输入用户名apollo，密码admin后登录

添加了两个项目，一个作为基础命名空间，另一个继承这个命名空间 

开放api

拿对应appid的多个命名空间的消息更新

【config service 地址】/notifications/v2?appId=【appid】&cluster=【集群地址】&notifications=%5B%7B%22namespaceName%22%3A%22【命名空间1】%22%2C%22notificationId%22%3A-1%7D%2C%7B%22namespaceName%22%3A%22【命名空间2】%22%2C%22notificationId%22%3A-1%7D%5D

返回如下

[{
	"namespaceName": "application",
	"notificationId": 4618,
	"messages": {
		"details": {
			"ruby-live+default+application": 4618
		}
	}
}, {
	"namespaceName": "ruby.app",
	"notificationId": 4606,
	"messages": {
		"details": {
			"ruby+default+ruby.app": 4606,
			"ruby-live+default+ruby.app": 2323
		}
	}
}]

拉取配置信息

【config service 地址】/configs/【appid】/【集群地址】/【命名空间1】?ip=127.0.0.1&releaseKey=%

返回格式如下

{
	"appId": "ruby-live",
	"cluster": "default",
	"namespaceName": "application",
	"configurations": {

	},
	"releaseKey": "20220929203203-026c76296579b6ac"
}

php集成apollo配置中心

ApolloClient类负责和apollo上述两个接口打交道，把拉取的apollo配置信息写到本地文件，用到了并发curl请求特性

<?php

namespace Ruby\Config;


class ApolloClient
{
    protected $configServer; //apollo服务端地址
    protected $appId; //apollo配置项目的appid
    protected $cluster = 'default';
    protected $clientIp = '127.0.0.1'; //绑定IP做灰度发布用
    protected $notifications = [];
    protected $pullTimeout = 10; //获取某个namespace配置的请求超时时间
    protected $intervalTimeout = 60; //每次请求获取apollo配置变更时的超时时间
    public $save_dir; //配置保存目录

    /**
     * ApolloClient constructor.
     * @param string $configServer apollo服务端地址
     * @param string $appId apollo配置项目的appid
     * @param array $namespaces apollo配置项目的namespace
     */
    public function __construct($configServer, $appId, array $namespaces)
    {
        $this->configServer = $configServer;
        $this->appId = $appId;
        foreach ($namespaces as $namespace) {
            $this->notifications[$namespace] = ['namespaceName' => $namespace, 'notificationId' => -1];
        }
        $this->save_dir = dirname($_SERVER['SCRIPT_FILENAME']);
    }

    public function setCluster($cluster)
    {
        $this->cluster = $cluster;
    }

    public function setClientIp($ip)
    {
        $this->clientIp = $ip;
    }

    public function setPullTimeout($pullTimeout)
    {
        $pullTimeout = intval($pullTimeout);
        if ($pullTimeout < 1 || $pullTimeout > 300) {
            return;
        }
        $this->pullTimeout = $pullTimeout;
    }

    public function setIntervalTimeout($intervalTimeout)
    {
        $intervalTimeout = intval($intervalTimeout);
        if ($intervalTimeout < 1 || $intervalTimeout > 300) {
            return;
        }
        $this->intervalTimeout = $intervalTimeout;
    }

    private function _getReleaseKey($config_file)
    {
        $releaseKey = '';
        if (file_exists($config_file)) {
            $last_config = require $config_file;
            is_array($last_config) && isset($last_config['releaseKey']) && $releaseKey = $last_config['releaseKey'];
        }
        return $releaseKey;
    }

    //获取单个namespace的配置文件路径
    public function getConfigFile($namespaceName)
    {
        return $this->save_dir . DIRECTORY_SEPARATOR . 'apolloConfig.' . $namespaceName . '.php';
    }

    //获取单个namespace的配置-无缓存的方式
    public function pullConfig($namespaceName)
    {
        $base_api = rtrim($this->configServer, '/') . '/configs/' . $this->appId . '/' . $this->cluster . '/';
        $api = $base_api . $namespaceName;

        $args = [];
        $args['ip'] = $this->clientIp;
        $config_file = $this->getConfigFile($namespaceName);
        $args['releaseKey'] = $this->_getReleaseKey($config_file);

        $api .= '?' . http_build_query($args);

        $ch = curl_init($api);
        curl_setopt($ch, CURLOPT_TIMEOUT, $this->pullTimeout);
        curl_setopt($ch, CURLOPT_HEADER, false);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

        $body = curl_exec($ch);
        $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
        $error = curl_error($ch);
        curl_close($ch);

        if ($httpCode == 200) {
            $result = json_decode($body, true);
            $content = '<?php return ' . var_export($result, true) . ';';
            file_put_contents($config_file, $content);
            echo 'get content' . $content . "\r\n";

        } elseif ($httpCode != 304) {
            echo $body ?: $error . "\n";
            return false;
        }
        return true;
    }

    //获取多个namespace的配置-无缓存的方式
    public function pullConfigBatch(array $namespaceNames)
    {
        if (!$namespaceNames) return [];
        $multi_ch = curl_multi_init();
        $request_list = [];
        $base_url = rtrim($this->configServer, '/') . '/configs/' . $this->appId . '/' . $this->cluster . '/';
        $query_args = [];
        $query_args['ip'] = $this->clientIp;
        foreach ($namespaceNames as $namespaceName) {
            $request = [];
            $config_file = $this->getConfigFile($namespaceName);
            $request_url = $base_url . $namespaceName;
            $query_args['releaseKey'] = $this->_getReleaseKey($config_file);
            $query_string = '?' . http_build_query($query_args);
            $request_url .= $query_string;
            $ch = curl_init($request_url);
            curl_setopt($ch, CURLOPT_TIMEOUT, $this->pullTimeout);
            curl_setopt($ch, CURLOPT_HEADER, false);
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
            $request['ch'] = $ch;
            $request['config_file'] = $config_file;
            $request_list[$namespaceName] = $request;
            curl_multi_add_handle($multi_ch, $ch);
        }

        $active = null;
        // 执行批处理句柄
        do {
            $mrc = curl_multi_exec($multi_ch, $active);
        } while ($mrc == CURLM_CALL_MULTI_PERFORM);

        while ($active && $mrc == CURLM_OK) {
            if (curl_multi_select($multi_ch) == -1) {
                usleep(100);
            }
            do {
                $mrc = curl_multi_exec($multi_ch, $active);
            } while ($mrc == CURLM_CALL_MULTI_PERFORM);

        }

        // 获取结果
        $response_list = [];
        foreach ($request_list as $namespaceName => $req) {
            $response_list[$namespaceName] = true;
            $result = curl_multi_getcontent($req['ch']);
            $code = curl_getinfo($req['ch'], CURLINFO_HTTP_CODE);
            $error = curl_error($req['ch']);
            curl_multi_remove_handle($multi_ch, $req['ch']);
            curl_close($req['ch']);
            if ($code == 200) {
                $result = json_decode($result, true);
                $content = '<?php return ' . var_export($result, true) . ';';
                file_put_contents($req['config_file'], $content);
                echo 'get content' . $content . "\r\n";
            } elseif ($code != 304) {
                echo 'pull config of namespace[' . $namespaceName . '] error:' . ($result ?: $error) . "\n";
                $response_list[$namespaceName] = false;
            }
        }
        curl_multi_close($multi_ch);
        return $response_list;
    }

    protected function _listenChange(&$ch, $callback = null)
    {
        $base_url = rtrim($this->configServer, '/') . '/notifications/v2?';
        $params = [];
        $params['appId'] = $this->appId;
        $params['cluster'] = $this->cluster;
        do {
            $params['notifications'] = json_encode(array_values($this->notifications));
            $query = http_build_query($params);
            curl_setopt($ch, CURLOPT_URL, $base_url . $query);
            $response = curl_exec($ch);
            $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
            $error = curl_error($ch);
            if ($httpCode == 200) {
                $res = json_decode($response, true);
                $change_list = [];
                foreach ($res as $r) {
                    if ($r['notificationId'] != $this->notifications[$r['namespaceName']]['notificationId']) {
                        $change_list[$r['namespaceName']] = $r['notificationId'];
                    }
                }
                echo "config change:" . $response . "\r\n";
                $response_list = $this->pullConfigBatch(array_keys($change_list));
                foreach ($response_list as $namespaceName => $result) {
                    $result && ($this->notifications[$namespaceName]['notificationId'] = $change_list[$namespaceName]);
                }
                //如果定义了配置变更的回调，比如重新整合配置，则执行回调
                ($callback instanceof \Closure) && call_user_func($callback);
            } elseif ($httpCode != 304) {
                throw new \Exception($response ?: $error);
            }
        } while (true);
    }

    /**
     * @param $callback 监听到配置变更时的回调处理
     * @return mixed
     */
    public function start($callback = null)
    {
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_TIMEOUT, $this->intervalTimeout);
        curl_setopt($ch, CURLOPT_HEADER, false);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        try {
            $this->_listenChange($ch, $callback);
        } catch (\Exception $e) {
            curl_close($ch);
            return $e->getMessage();
        }
    }
}

Config类对 illuminate/config 基础上，把apollo的配置信息整合到自己管理的配置数组中，进行读取和设置

<?php

namespace Ruby\Config;

class Config extends \Illuminate\Config\Repository
{
    private $configServer;

    private $appId;

    private $namespaces;

    private $clientIp;

    private $cluster;

    /**
     * Config constructor.
     * @param $configServer
     * @param $appId
     * @param $namespaces
     * @param $clientIp
     * @param $savePath
     * @throws \Exception
     */
    public function __construct($configServer, $appId, $namespaces, $savePath = null, $clientIp = null, $cluster = 'default')
    {
        parent::__construct();
        $this->setConfigServer($configServer);
        $this->setAppId($appId);
        $this->setNamespaces($namespaces);
        $this->setSavePath($savePath);
        $this->setClientIp($clientIp);
        $this->loadLocalConfig();
        $this->cluster = $cluster;
    }

    private $savePath;


    /**
     * 读取apollo配置
     * @param null $callback
     */
    public function run($callback = null)
    {
        if (is_null($callback)) {
            $callback = function () {
                echo "config changed sync config \r\n";
                $this->loadLocalConfig();
                reload_jobs();
            };
        } else {
            $callback = function () use ($callback) {
                $this->loadLocalConfig();
                $callback();
            };
        }
        $apollo = new ApolloClient($this->getConfigServer(), $this->getAppId(), $this->getNamespaces());

        if ($this->getClientIp()) {
            $apollo->setClientIp($this->getClientIp());
        }
        $cluster = $this->getCluster();
        $apollo->setCluster($cluster);
        $apollo->save_dir = $this->getSavePath();
        ini_set('memory_limit', '128M');
        $pid = getmypid();
        echo 'cluster is :' . $cluster . "\r\n";
        echo 'save_path is :' . $this->getSavePath() . "\r\n";
        echo 'config server is :' . $this->getConfigServer() . "\r\n";
        echo 'app_id is :' . $this->getAppId() . "\r\n";
        echo 'namespaces is :' . implode(',', $this->getNamespaces()) . "\r\n";
        echo "start [$pid]\r\n";
        $restart = true; //auto start if failed
        do {
            $error = $apollo->start($callback);
            if ($error) {
                echo('error:' . $error . "\r\n");
            } else {
                echo "get config success\r\n";
            }
            sleep(1);
            echo "restarting apollo config client daemon\r\n";
        } while ($restart);
    }

    public function loadLocalConfig()
    {
        $pattern = $this->getSavePath() . DIRECTORY_SEPARATOR . 'apolloConfig.*';
        $list = glob($pattern);
        foreach ($list as $l) {
            $config = require $l;
            if (is_array($config) && isset($config['configurations'])) {
                foreach ($config['configurations'] as $name => $value) {
                    if (is_json($value)) {
                        $value = json_decode($value, true);
                    }
                    data_set($this->items, $name, $value);
                }
            }
        }
        return $this->all();
    }

    public function merge($config)
    {
        $this->items = array_merge_recursive($config, $this->items);
    }

    /**
     * @return mixed
     */
    public function getClientIp()
    {
        return $this->clientIp;
    }

    /**
     * @param mixed $clientIp
     */
    public function setClientIp($clientIp)
    {
        $this->clientIp = $clientIp ? $clientIp : null;
    }

    /**
     * @return mixed
     */
    public function getNamespaces()
    {
        return $this->namespaces;
    }

    /**
     * @param mixed $namespaces
     */
    public function setNamespaces($namespaces)
    {
        !is_array($namespaces) && $namespaces = explode(',', $namespaces);
        $this->namespaces = $namespaces;
    }

    /**
     * @return mixed
     */
    public function getAppId()
    {
        return $this->appId;
    }

    /**
     * @param mixed $appId
     */
    public function setAppId($appId)
    {
        $this->appId = $appId;
    }

    /**
     * @return mixed
     */
    public function getConfigServer()
    {
        return $this->configServer;
    }

    /**
     * @param mixed $configServer
     */
    public function setConfigServer($configServer)
    {
        $this->configServer = $configServer;
    }

    /**
     * @return mixed
     */
    public function getSavePath()
    {
        return $this->savePath;
    }

    /**
     * @param mixed $savePath
     */
    public function setSavePath($savePath)
    {
        if (empty($savePath)) {
            $savePath = '/home/config/' . $this->getAppId();
        }
        if (!is_dir($savePath)) {
            @mkdir($savePath, 0777, true);
        }
        $this->savePath = $savePath;
    }

    public function getCluster()
    {
        $cluster = getenv('APOLLO_CLUSTER');
        if ($cluster !== false) {
            return $cluster;
        }
        return $this->cluster;
    }
}

Container类用单例模式保证Config类被唯一实例化，并且允许使用环境变量来覆盖apollo的配置信息

<?php

namespace Ruby\Config;

use Illuminate\Support\Str;

class Container
{
    private static $config;

    private function __construct()
    {
        throw new \Exception('not allowed construct');
    }

    /**
     * 调用实例化
     * @param $configServer
     * @param $appId
     * @param $namespaces
     * @param null $savePath
     * @param null $clientIp
     * @return Config
     * @throws \Exception
     */
    public static function register($configServer, $appId, $namespaces, $savePath = null, $clientIp = null, $cluster = 'default')
    {
        self::$config = new Config($configServer, $appId, $namespaces, $savePath, $clientIp, $cluster);
        self::registerOsEnv();
        return self::$config;
    }

    private static function registerOsEnv()
    {
        //register os env
        $osEnvs = getenv();
        if (!empty($osEnvs)) {
            foreach ($osEnvs as $key => $item) {
                if (Str::startsWith($key, 'RUBY_')) {
                    $key = str_replace('_', '.', strtolower(ltrim($key, 'MELO_')));
                    if (Str::contains($key, '\.')) {
                        $key = str_replace('\.', '_', $key);
                    }
                    self::$config->set($key, $item);
                }
            }
        }
    }

    public static function instance()
    {
        if (self::$config) {
            return self::$config;
        }
        throw new \Exception("config is not registered, call register before ");
    }

    private function __clone()
    {
        throw new \Exception('not allowed clone');
    }
}

ConfigCommand 封装了 illuminate/console 的一个命令，然后注册到lumen框架中去 

<?php

namespace Ruby\Config\Lumen;

use Illuminate\Console\Command;
use Melo\Config\Container;

class ConfigCommand extends Command
{
    protected $signature = 'config:sync';

    protected $description = 'sync config from apollo';

    public function handle()
    {
        $config = Container::instance();
        $callback = null;
        $config->run($callback);
    }
}

helpers.php封装一个小助手函数来读取apollo配置中心的内容（也有可能是覆盖apollo的环境变量）

<?php

if (!function_exists('conf')) {
    /**
     * Get / set the specified configuration value.
     *
     * If an array is passed as the key, we will assume you want to set an array of values.
     *
     * @param  array|string|null $key
     * @param  mixed $default
     * @return mixed
     * @throws Exception
     */
    function conf($key = null, $default = null)
    {
        if (is_null($key)) {
            return \Melo\Config\Container::instance()->all();
        }

        if (is_array($key)) {
            return \Melo\Config\Container::instance()->set($key);
        }

        return \Melo\Config\Container::instance()->get($key, $default);
    }
}

用php写一个可执行文件来长期运行拉取apollo配置信息到本地并及时更新的重要任务

#!/usr/bin/env php
<?php

use Ruby\Config\Container;

$dir = __DIR__ . '/src';

$dir = __DIR__ . '/..';

if (!file_exists($dir . '/autoload.php')) {
    $dir = __DIR__ . '/../vendor';
}

if (!file_exists($dir . '/autoload.php')) {
    $dir = __DIR__ . '/../../..';
}

if (!file_exists($dir . '/autoload.php')) {
    echo 'Autoload not found.';
    exit(1);
}

require $dir . '/autoload.php';

$config = [
    'config_server' => getenv('APOLLO_CONFIG_SERVER'),
    'app_id' => getenv('APOLLO_APP_ID'),
    'namespaces' => getenv('APOLLO_NAMESPACES'),
    'save_path' => getenv('APOLLO_SAVE_PATH') ?? null,
    'client_ip' => getenv('APOLLO_CLIENT_IP') ?? null,
    'cluster' => getenv('APOLLO_CLUSTER') ?? null,
];

$config = array_merge($config, parse_cli_params($argv));

if (empty($config['config_server'])) {
    error('config_server is not passed');
}
if (empty($config['app_id'])) {
    error('app_id is not passed');
}
if (empty($config['namespaces'])) {
    error('namespaces is not passed');
} else {
    $config['namespaces'] = explode(',', $config['namespaces']);
}

$client = Container::register(
    $config['config_server'],
    $config['app_id'],
    $config['namespaces'],
    $config['save_path'] ?? null,
    $config['client_ip'] ?? null
);
$client->run();

function error($msg)
{
    print_params();
    echo '[ERROR]: ' . $msg;
    exit(-1);
}

function print_params()
{
    echo "usage: config config_server=sample app_id=sample namespaces=sample1,sample2 client_ip=sample save_path=absolute_path \r\n";
    echo "or set os_env APOLLO_CONFIG_SERVER APOLLO_APP_ID APOLLO_NAMESPACES APOLLO_CLIENT_IP APOLLO_SAVE_PATH \r\n";
}

把这些封装成一个php composer包，供内网其他项目使用

{
  "name": "ruby/config",
  "description": "config for projects use apollo",
  "type": "library",
  "keywords": [
    "client",
    "config",
    "apollo"
  ],
  "license": "MIT",
  "authors": [{
    "name": "ruby",
    "email": "rubyruby@gmail.com"
  }],
  "require": {
    "php": ">=7.1",
    "illuminate/config": ">=6.0",
    "illuminate/console": ">=6.0",
    "ext-curl": "*",
    "ext-json": "*"
  },
  "autoload": {
    "files": [
      "src/helpers.php"
    ],
    "psr-4": {
      "RUBY\\Config\\": "src/"
    }
  },
  "bin": [
    "bin/config"
  ]
}

 相关链接

手把手教你部署nginx+php

php和nginx镜像合并 && 代码打包到镜像

nginx-php镜像安装常用软件

yaf && yar微服务/hprose微服务 镜像初始化

常用开发工具：php_codesniffer代码规范检查&修复、phpstan语法检查、phpunit单元测试

.gitlab-ci.yaml自动镜像打包&&互联网企业规范化上线流程（上）

kustomize/kubectl自动镜像部署&&互联网企业规范化上线流程（下）

apisix网关、JMeter压测  

prometheus/grafana监控数据收集与展示

k8s容器网络性能调优

supervisor进程管理

安装opcache和apcu

APM性能监测工具skywalking

链路跟踪工具zipkin

phpfpm和nginx配置

php整合apollo配置中心

php rdkafka操作kafka消息队列