缓冲区读取问题

一次http请求（post），会报错：

HTTP request to `http://127.0.0.1:9200/my-index-000001?pretty` failed at try 1/1 with bytes read: 0/unknown. Error: DB::Exception: Unexpected state of istream at offset 0. (Current backoff wait is 100/1600 ms)

已确定服务端成功完成工作并返回response，因为可以通过下列代码获得Received response with content length：

std::streamsize HTTPMessage::getContentLength() const
{
	const std::string& contentLength = get(CONTENT_LENGTH, EMPTY);
	if (!contentLength.empty())
	{
		if (sizeof(std::streamsize) == sizeof(Poco::Int64))
			return static_cast<std::streamsize>(NumberParser::parse64(contentLength));
		else
			return static_cast<std::streamsize>(NumberParser::parse(contentLength));
	}
	else return UNKNOWN_CONTENT_LENGTH;
}
if (response.hasContentLength())
        LOG_DEBUG(log, "Received response with content length: {}", response.getContentLength());

会打印出个log：Received response with content length: 91
说明response长度已经可以被解析。

但是最终报错：

HTTP request to `http://127.0.0.1:9200/my-index-000001?pretty` failed at try 1/1 with bytes read: 0/unknown. Error: DB::Exception: Unexpected state of istream at offset 0. (Current backoff wait is 100/1600 ms)

经过定位failed at try 1/1 with bytes read: 0/unknown 发现异常抛出点。这里read_range是个unkown。大概率是buffer没读对。同时，offset_from_begin_pos是0。

catch (const Poco::Exception & e)
        {
            /// Too many open files - non-retryable.
            if (e.code() == POCO_EMFILE)
                throw;

            /** Retry request unconditionally if nothing has been read yet.
                    * Otherwise if it is GET method retry with range header.
                    */
            bool can_retry_request = !offset_from_begin_pos || method == Poco::Net::HTTPRequest::HTTP_GET;
            if (!can_retry_request)
                throw;

            LOG_ERROR(
                log,
                "HTTP request to `{}` failed at try {}/{} with bytes read: {}/{}. "
                "Error: {}. (Current backoff wait is {}/{} ms)",
                uri.toString(),
                i + 1,
                settings.http_max_tries,
                getOffset(),
                read_range.end ? toString(*read_range.end) : "unknown",
                e.displayText(),
                milliseconds_to_wait,
                settings.http_retry_max_backoff_ms);

            on_retriable_error();
            exception = std::current_exception();
        }

经过定位问题Unexpected state of istream at offset，发现现在，gcount() = 0，并且istr.fail() = false

bool ReadBufferFromIStream::nextImpl()
{
    istr.read(internal_buffer.begin(), internal_buffer.size());
    size_t gcount = istr.gcount();

    if (!gcount)
    {
        if (istr.eof())
            return false;

        if (istr.fail())
            throw Exception(ErrorCodes::CANNOT_READ_FROM_ISTREAM, "Cannot read from istream at offset {}", count());

        throw Exception(ErrorCodes::CANNOT_READ_FROM_ISTREAM, "Unexpected state of istream at offset {}", count());
    }
    else
        working_buffer.resize(gcount);

    return true;
}

大概率是buffer读取的问题。回到上层检查buffer读取。要么是http请求用的不对导致post返回的数据没有正确读取，要么是buffer本事的读取函数有问题。http请求调用与buffer读取代码如下。point576可以被打印，尝试虚修改读取函数。

if (!rq.body.empty())
    {
        std::cout << "point" << std::endl;
        auto write_body_callback = [rq](std::ostream & os) { os << rq.body+"\n"; };
        ReadWriteBufferFromHTTP buf_test(
            uri,
            rq.http_method,
            write_body_callback,
            connection_parameters.timeouts,
            creds,
            {},
            {},
            {},
            {{"Content-Type", "application/json"}});
        std::string character;
        std::cout<<"point:576"<<std::endl;
        readStringBinary(character, buf_test);
        std::cout << "response_body:" << character << std::endl;
]

定位到读取函数后，观察到用ReadWriteBufferFromHTTP时缓冲区不能正常读写，但是他的子类PooledReadWriteBufferFromHTTP缓冲区可以正常读写。因为ReadWriteBufferFromHTTP本身对于一部分请求可以正常发送读取，因此开始的时候误以为是使用方式出错。PooledReadWriteBufferFromHTTP可以发送读取无header的请求，不能满足条件，因此决定通过继承ReadWriteBufferFromHTTPBase实现类似于PooledReadWriteBufferFromHTTP的类。并且通过.Parent父类多态来将参数传入（这部分C++）.

class TestReadWriteBufferFromHTTP : public detail::ReadWriteBufferFromHTTPBase<std::shared_ptr<UpdatableSession<PooledSessionFactory>>>
{
    using SessionType = UpdatableSession<PooledSessionFactory>;
    using Parent = detail::ReadWriteBufferFromHTTPBase<std::shared_ptr<SessionType>>;

public:
    explicit TestReadWriteBufferFromHTTP(
        Poco::URI uri_,
        const std::string & method_ = {},
        OutStreamCallback out_stream_callback_ = {},
        HTTPHeaderEntries http_header_entries_ = {},
        const ConnectionTimeouts & timeouts_ = {},
        const Poco::Net::HTTPBasicCredentials & credentials_ = {},
        size_t buffer_size_ = DBMS_DEFAULT_BUFFER_SIZE,
        const UInt64 max_redirects = 0,
        size_t max_connections_per_endpoint = DEFAULT_COUNT_OF_HTTP_CONNECTIONS_PER_ENDPOINT);
}

TestReadWriteBufferFromHTTP::TestReadWriteBufferFromHTTP(
    Poco::URI uri_,
    const std::string & method_,
    OutStreamCallback out_stream_callback_,
    HTTPHeaderEntries http_header_entries_,
    const ConnectionTimeouts & timeouts_,
    const Poco::Net::HTTPBasicCredentials & credentials_,
    size_t buffer_size_,
    const UInt64 max_redirects,
    size_t max_connections_per_endpoint)
    : Parent(
        std::make_shared<SessionType>(uri_, max_redirects, std::make_shared<PooledSessionFactory>(timeouts_, max_connections_per_endpoint)),
        uri_,
        credentials_,
        method_,
        out_stream_callback_,
        buffer_size_,
        {},
        http_header_entries_)
{
}

使用TestReadWriteBufferFromHTTP类既可以实现有header发送也可以实现无header发送，至此，一个卡点问题解决。