php使用weiboad/kafka-php推送错误Could not write xxx bytes to stream解决

线上使用php的客户端是 https://github.com/weiboad/kafka-php 版本:dev-master
线上经常报错误:
Could not write 1422 bytes to stream
追查源码后发现在CommonSocket.php 文件中 writeBlocking方法

 public function writeBlocking(string $buffer): int
    {
        // fwrite to a socket may be partial, so loop until we
        // are done with the entire buffer
        $failedAttempts = 0;
        $bytesWritten   = 0;

        $bytesToWrite = strlen($buffer);

        while ($bytesWritten < $bytesToWrite) {
            // wait for stream to become available for writing
            $writable = $this->select([$this->stream], $this->sendTimeoutSec, $this->sendTimeoutUsec, false);

            if ($writable === false) {
                throw new Exception\Socket('Could not write ' . $bytesToWrite . ' bytes to stream');
            }

            if ($writable === 0) {
                $res = $this->getMetaData();
                if (! empty($res['timed_out'])) {
                    throw new Exception('Timed out writing ' . $bytesToWrite . ' bytes to stream after writing ' . $bytesWritten . ' bytes');
                }

                throw new Exception\Socket('Could not write ' . $bytesToWrite . ' bytes to stream');
            }

            if ($bytesToWrite - $bytesWritten > self::MAX_WRITE_BUFFER) {
                // write max buffer size
                $wrote = fwrite($this->stream, substr($buffer, $bytesWritten, self::MAX_WRITE_BUFFER));
            } else {
                // write remaining buffer bytes to stream
                $wrote = fwrite($this->stream, substr($buffer, $bytesWritten));
            }

            if ($wrote === -1 || $wrote === false) {
                throw new Exception\Socket('Could not write ' . strlen($buffer) . ' bytes to stream, completed writing only ' . $bytesWritten . ' bytes');
            }

            if ($wrote === 0) {
                // Increment the number of times we have failed
                $failedAttempts++;

                if ($failedAttempts > $this->maxWriteAttempts) {
                    throw new Exception\Socket('After ' . $failedAttempts . ' attempts could not write ' . strlen($buffer) . ' bytes to stream, completed writing only ' . $bytesWritten . ' bytes');
                }
            } else {
                // If we wrote something, reset our failed attempt counter
                $failedAttempts = 0;
            }

            $bytesWritten += $wrote;
        }

        return $bytesWritten;
    }

w r i t a b l e = = = f a l s e 这 一 行 抛 出 的 异 常 , 追 查 代 码 发 现 调 用 的 是 s t r e a m s e l e c t ( writable === false 这一行抛出的异常, 追查代码发现调用的是 stream_select( writable===false,streamselect(null, $sockets, $null, $timeoutSec, $timeoutUsec); 方法

stream_select 是一个事件模型, 看文档知道 需要传入$socktes, readSocket, writeSocket, ExceptionSocket,超时秒数 和超时毫秒数 那为什么会返回false 呢
文档解释
https://www.php.net/manual/en/function.stream-select.php

成功时,stream_select()返回已修改数组中包含的流资源的数量,如果超时在任何有趣的事件发生之前到期,则可能为零。FALSE 返回错误并引发警告(如果系统调用被传入信号中断,则可能发生此情况)。

那说明有错误引起的
据需看代码看 到创建socket

 protected function createSocket(string $remoteSocket, $context, ?int &$errno, ?string &$errstr)
    {
        return stream_socket_client(
            $remoteSocket,
            $errno,
            $errstr,
            $this->sendTimeoutSec + ($this->sendTimeoutUsec / 1000000),
            STREAM_CLIENT_CONNECT,
            $context
        );
    }

传入的timeout时间是 t h i s − &gt; s e n d T i m e o u t S e c + ( this-&gt;sendTimeoutSec + ( this>sendTimeoutSec+(this->sendTimeoutUsec / 1000000), 计算默认默认值是0.1s
这个值太小了 会导致socket超时直接断开, 在stream_select时候 socket已经断开 返回false

将值改大后实验 不在报错误了 确实是这个值引起的

将源码值改为:

**
     * Send timeout in seconds.
     *
     * @var int
     */
    protected $sendTimeoutSec = 300;

    /**
     * Send timeout in microseconds.
     *
     * @var int
     */
    protected $sendTimeoutUsec = 100000;

    /**
     * Recv timeout in seconds
     *
     * @var int
     */
    protected $recvTimeoutSec = 300;

    /**
     * Recv timeout in microseconds
     *
     * @var int
     */
    protected $recvTimeoutUsec = 750000;

附我的kafka配置:

 $config = \Kafka\ProducerConfig::getInstance();
        $config->setMetadataRefreshIntervalMs(10000);
        $config->setMessageMaxBytes(1000000000);
        $config->setMetadataBrokerList($kafka_config['url']);
        $config->setBrokerVersion('1.1.1');
        $config->setSecurityProtocol($kafka_config['securityprotocol']);
        $config->setSaslMechanism($kafka_config['saslmechanism']);
        $config->setSaslUsername($kafka_config['saslusername']);
        $config->setSaslPassword($kafka_config['saslpassword']);
        $config->setRequiredAck(1);
        $config->setIsAsyn(false);
        $config->setProduceInterval(500);
        $config->setRequestTimeout(900000);
        $config->setTimeout(900000); //kafka 请求超时控制
        $producer = new \Kafka\Producer();
        $producer->setLogger($logger);

默认值都设置最大吧 保证不会出错 哈哈哈

我将项目拉了一份修改后放到了自己的github中
https://github.com/jeevi-cao/kafka-php

码字不易

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值