转mjpeg协议到httpflv

知识点

mjpeg

mjpeg是动态jpeg，Motion Joint Photographic Experts Group）是一种视频压缩格式，其中每一帧图像都分别使用JPEG编码，不使用帧间编码，压缩率通常在20:1-50:1范围内。比不上h264，h265，但是有一个非常好的地方，就是可以直接在http里面显示.

httpflv

httpflv 其实最终是要变成fmp4格式来使用浏览器来展示的，这一点一定要清楚，如果不清楚fmp4，一定要搞清楚这种格式。

接收mjpeg

准备一个http客户端，接收mjpeg服务器的码流，实际上，他是一张张的jpg图片在http通道里面传输过来。以下是使用asio制作的一个客户端，链接http server之后，每次都去获取mjpeg帧的边界，叫 Content-Type:.*boundary=，还有一个关键就是获取每一帧的content-length,这个都是使用http协议解析来获取。

void capture_start(s_param *param)
{
	//开始视频采集
	if (param != NULL)
	{
		asio::io_context io_service;
		tcp::resolver resolver(io_service);
		tcp::resolver::query query(param->host, param->port);
		tcp::resolver::iterator endpoint_iterator = resolver.resolve(query);
		tcp::resolver::iterator end;
		tcp::socket socket(io_service);
		//socket.io_control(asio::ip::tcp::socket::non_blocking(true));
		std::error_code error = asio::error::host_not_found;
		while (error && endpoint_iterator != end) {
			socket.close();
			//int timeout = 3000;
			//int nRet = setsockopt(socket.native_handle(), SOL_SOCKET, SO_CONNECT_TIME, (const char*)&timeout, sizeof(timeout));
			socket.connect(*endpoint_iterator++, error);
		}
		if (error) {
			cout << "can not connect the camera!" << endl;
			return;
			//throw asio::system_error(error);
		}

		// Form the request. We specify the "Connection: close" header so that the
		// server will close the socket after transmitting the response. This will
		// allow us to treat all data up until the EOF as the content.
		asio::streambuf request;
		ostream request_stream(&request);
		request_stream << "GET " <<param->route << " HTTP/1.0\r\n";
		request_stream << "Host: " << param->host << "\r\n";
		request_stream << "Accept: */*\r\n";
		request_stream << "Connection: close\r\n\r\n";

		// Send the request.
		asio::write(socket, request);

		// Read the response status line. The response streambuf will automatically
		// grow to accommodate the entire line. The growth may be limited by passing
		// a maximum size to the streambuf constructor.
		asio::streambuf response;
		asio::read_until(socket, response, "\r\n");

		// Check that response is OK.
		istream response_stream(&response);
		string http_version;
		response_stream >> http_version;
		unsigned int status_code;
		response_stream >> status_code;
		string status_message;
		getline(response_stream, status_message);
		if (!response_stream || http_version.substr(0, 5) != "HTTP/") {
			cout << "Invalid response\n";
			return;
		}
		if (status_code != 200) {
			cout << "Response returned with status code " << status_code << "\n";
			return;
		}

		// Read the response headers, which are terminated by a blank line.
		asio::read_until(socket, response, "\r\n\r\n");

		// Get the MIME multipart boundary from the headers.
		regex rx_content_type("Content-Type:.*boundary=(.*)");
		regex rx_content_length("Content-Length: (.*)");

		smatch match;
		string header;
		string boundary;

		while (getline(response_stream, header) && header != "\r") {
			//cout << "HTTP HEADER: " << header << endl;
			if (regex_search(header, match, rx_content_type)) {
				boundary = match[1];
				//cout << "BOUNDARY SELECTED: " << boundary << endl;
			}
		}

		// Abort if a boundary was not found.
		if (boundary == "") {
			cout << "Not a MJPEG stream" << endl;
			return;
		}


		std::vector<uint8_t> buff;
		while (1) {
			asio::read_until(socket, response, boundary);
			while (getline(response_stream, header)) {
				if (header.find(boundary + "\r") != string::npos)
				{
					break;
				}
			}
			uint32_t content_length;
			while (getline(response_stream, header) && header != "\r") {
				if (regex_search(header, match, rx_content_length)) {
					std::ssub_match base_sub_match = match[1];
					content_length = std::atoi(base_sub_match.str().c_str());
				}
			}

			if (response.size() < content_length) {
				asio::read(socket, response, asio::transfer_at_least(
					content_length - response.size()));
			}
			if (buff.size() < content_length) {
				buff.resize(content_length);
			}

			response.sgetn((char*)&buff[0], content_length);
			cv::Mat mat(buff);
			cv::Mat dec = imdecode(Mat(buff), IMREAD_COLOR);
			v_mut_src.lock();
			cv::resize(dec, v_srcImg, cv::Size(CANVAS_WIDTH, CANVAS_HEIGHT));
			v_mut_src.unlock();
			//cout << "RESPONSE SIZE, BEFORE JPEG CONSUME: " << response.size() << endl;
			cv::imshow("windows", v_srcImg);
			cv::waitKey(2);
			response.consume(content_length);
		}
	}
	
	std::cout << "capture break...." << std::endl;
}

可以看出来，我们使用opencv来展示画面，并让画面缩放到我们指定的宽度和高度，为了解耦合，我们加锁了图像，把图像放到了v_srcImg中。

解码和编码h264

jpeg内存文件或者文件的标记码由两个字节构成，其前一个字节是固定值0xFF，后一个字节则根据不同意义有不同数值。在每个标记码之前还可以添加数目不限的无意义的0xFF填充，也就说连续的多个0xFF可以被理解为一个0xFF，并表示一个标记码的开始。而在一个完整的两字节的标记码后，就是该标记码对应的压缩数据流，记录了关于文件的诸种信息。解码可以使用

cv::Mat dec = imdecode(Mat(buff), IMREAD_COLOR);

使用opencv的原因是我们还可以使用opencv来做一些算法来分析图像，然后再画图像并且编码到h264，转flvserver，其中经过h264编码后再经过rtp发送到我们的flvserver。

转送flvserver

制作了一个flvserver，具体使用udp rtp包接收，然后再结果flvmux，具体flvmux我们下一章再讲