HLS第四课（UG1233纪要）

Migrating HLS Video Library to xfOpenCV

The HLS video library will soon be deprecated .All the functions and most of the infrastructure
available in HLS video library are now available in xfOpenCV with their names changed and some
modifications.

All the functions imported from HLS video library now take xf::Mat (in sync with xfOpenCV
library) to represent image data instead of hls::Mat.
The main difference between these two is
that the hls::Mat uses hls::stream to store the data whereas xf::Mat uses a pointer. Therefore,
hls:: Mat cannot be exactly replaced with xf::Mat for migrating.

Infrastructure files available in HLS Video Library
hls_video_core.h, hls_video_mem.h, hls_video_types.h
are moved to
xf_video_core.h, xf_video_mem.h, xf_video_types.h
in xfOpenCV Library

All functions are defined in the xf namespace

All the functions are designed as templates and all arguments that are images, must be
provided as xf::Mat

Infrastructure Functions：
read()
----Readout a value from a given location and return it as a packed (for multipixel/clock) value.
Write()
----Writes a packed (for multi-pixel/clock)value into the given location

OpenCV interface functions:
two functions are available in xfOpenCV Library:
----cvMat2AXIvideo
and
----AXIvideo2cvMat
located in “xf_axi.h” file.

AXI4-Stream I/O Functions:
----xf::AXIvideo2xfMat
and
----xf:: xfMat2AXIvideo
to facilitate the xf::Mat to/from conversion. To use these functions, the header file “xf_infra.h” must be included.

Read the image using cv::imread()
Copy the data to xf::Mat
Call the processing function(s) in xfOpenCV
Copy the data from xf::Mat to cv::Mat
Write the output to image using cv::imwrite()

all the calls to xfOpenCV functions are moved to hardware.
The image containers for xfOpenCV library functions are xf::Mat objects.

+++++++++++++++++++++++++++++++++++++++++++++++++
xf::window
A template class to represent the 2D window buffer. It has three parameters to specify the
number of rows, columns in window buffer and the pixel data type

Window<K_ROWS, K_COLS, unsigned char> kernel;

xf::LineBuffer
A template class to represent 2D line buffer. It has three parameters to specify the number of
rows, columns in window buffer and the pixel data type.

LineBuffer<3, 1920, XF_8UC3, RAM_S2P_URAM,1> buff;

++++++++++++++++++++++++++++++++++++++++++++++++++++
Using the xfOpenCV Library

include folder constitutes all the necessary components to build a Computer Vision or Image
Processing pipeline using the library. The folders common and core contain the infrastructure
that the library functions need for basic functions, Mat class, and macros. The library functions
are categorized into three folders, features, video and imgproc based on the operation they
perform. The names of the folders are self-explanatory.

To work with the library functions, you need to include the path to the The xfOpenCV library is
structured as shown in the following table.
You can include relevant header files for the library functions you will be working with after you source the include folder’s path to the compiler.
For example, if you would like to work with Harris
Corner Detector and Bilateral Filter, you must use the following lines in the host code:

#include “features/xf_harris.hpp” //for Harris Corner Detector
#include “imgproc/xf_bilateral_filter.hpp” //for Bilateral Filter
#include “video/xf_kalmanfilter.hpp

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Getting Started with HLS

Use of appropriate compile-time options - When using the xfOpenCV functions in HLS, the
-D__SDSVHLS__
and
-std=c++0x
options need to be provided at the time of compilation:

Specifying interface pragmas to the interface level arguments - For the functions with top level
interface arguments as pointers (with more than one read/write access), the m_axi Interface
pragma must be specified.

void lut_accel(xf::Mat<TYPE, HEIGHT, WIDTH, NPC1> &imgInput,
xf::Mat<TYPE, HEIGHT, WIDTH, NPC1> &imgOutput, unsigned char *lut_ptr)
{
#pragma HLS INTERFACE m_axi depth=256 port=lut_ptr offset=direct bundle=lut_ptr
	xf::LUT< TYPE, HEIGHT, WIDTH, NPC1> (imgInput,imgOutput,lut_ptr);
}

这个函数有一个lut_ptr，HLS无法推断它的读写方向以及访问寻址机制，所以要指定为AXIMM。

Tcl Script Mode
Setting flags for source files:
Setting flags for testbench files:

add_files xf_dilation_accel.cpp -cflags "-I<path-to-include-directory> -D__SDSVHLS__ -std=c++0x "
add_files -tb xf_dilation_tb.cpp -cflags "-I<path-to-include-directory> -D__SDSVHLS__ -std=c++0x"

GUI Mode
Right click on the created project and select Project Settings
In the opened tab, select Simulation
Files added under the Test Bench section will be displayed.
Select a file and click Edit
CFLAGS.
Enter
-I<path-to-include-directory> -D__SDSVHLS__ -std=c++0x

Note: When using Vivado HLS in the Windows operating system, make sure to provide the -std=c+
+0x flag only for C-Sim and Co-Sim. Do not include the flag when performing synthesis

Select Synthesis and repeat the above step for all the displayed files.

+++++++++++++++++++++++++++++++++++++++++++++++++++
AXI Video Interface Functions

xfOpenCV has functions that will transform the xf::Mat into Xilinx® Video Streaming interface
and vice-versa.

xf::AXIvideo2xfMat()
and
xf::xfMat2AXIVideo()
act as video interfaces to the IPs of the xfOpenCV functions in the Vivado® IP integrator.

cvMat2AXIvideoxf <NPC> and AXIvideo2cvMatxf<NPC> are used on the host side.

----AXIvideo2xfMat
The AXIvideo2xfMat function receives a sequence of images using the AXI4 Streaming Video
and produces an xf::Mat representation.
用在可综合的模块内，入口第一个函数，将AXISVIDEO转换成XFMAT。

template<int W,int T,int ROWS, int COLS,int NPC>
int AXIvideo2xfMat(
					hls::stream< ap_axiu<W,1,1,1> >& AXI_video_strm,
					xf::Mat<T,ROWS, COLS, NPC>& img)

----xfMat2AXIvideo
The Mat2AXI video function receives an xf::Mat representation of a sequence of images and
encodes it correctly using the AXI4 Streaming video protocol.
用在可综合的模块内，出口最后一个函数，将XFMAT转换成AXISVIDEO。

template<int W, int T, int ROWS, int COLS,int NPC>
int xfMat2AXIvideo(
				xf::Mat<T,ROWS, COLS,NPC>& img,
				hls::stream<ap_axiu<W,1,1,1> >& AXI_video_strm)

----cvMat2AXIvideoxf
The cvMat2Axivideoxf function receives image as cv::Mat representation and produces the
AXI4 streaming video of image.
用在Testbench中，将CVMAT转换成AXISV的激励源，送给DUT。

template<int NPC,int W>
void cvMat2AXIvideoxf(
				cv::Mat& cv_mat, 
				hls::stream<ap_axiu<W,1,1,1> >& AXI_video_strm)

----AXIvideo2cvMatxf
The Axivideo2cvMatxf function receives image as AXI4 streaming video and produces the
cv::Mat representation of image
用在Testbench中，将DUT的输出结果AXISV转换成CVMAT，存放在TB中。

template<int NPC,int W>
void AXIvideo2cvMatxf(
				hls::stream<ap_axiu<W,1,1,1> >& AXI_video_strm,
				cv::Mat& cv_mat)

+++++++++++++++++++++++++++++++++++++++++++++++++
xf::Mat Image Container Class

The xf::Mat image container class is similar to the cv::Mat class of the OpenCV library。

----read(int index)
Readout a value from a given location and return it as a packed (for multi-pixel/clock) value.

----write(int index, XF_TNAME(T,NPC) val)
Writes a packed (for multi-pixel/clock) value into the given location.

----copyTo(* fromData)
Copies the data from Data pointer into physically contiguous memory allocated inside the constructor.

----depth()
Returns the depth of the image

----channels()
Returns number of channels of the image

++++++++++++++++++++++++++++++++++++++++++++++++
Pixel-Level Parallelism
XF_NPPC1
XF_NPPC2
XF_NPPC4
XF_NPPC8

The XF_NPIXPERCYCLE(flags) macro resolves to the number of pixels processed per cycle.
e.g.
XF_NPIXPERCYCLE(XF_NPPC4) resolves to 4

The XF_BITSHIFT(flags) macro resolves to the number of times to shift the image size to right to arrive at the final data transfer size for parallel processing.
e.g.
XF_BITSHIFT(XF_NPPC4) resolves to 2

+++++++++++++++++++++++++++++++++++++++++
Pixel Types
For example, for an 8-bit pixel - unsigned - 1 channel the data type is XF_8UC1
Option : Number of bits per Pixel : Unsigned/ Signed/ Float Type : Number of Channels
XF_8UC1 8 Unsigned 1
XF_16UC1 16 Unsigned 1
XF_16SC1 16 Signed 1
XF_32UC1 32 Unsigned 1
XF_32FC1 32 Float 1
XF_32SC1 32 Signed 1
XF_8UC2 8 Unsigned 2
XF_8UC4 8 Unsigned 4
XF_8UC3 8 Unsigned 3
XF_2UC1 2 Unsigned 1

XF_TNAME(TYPE,NPPC) resolves to the data type of the data member of the xf::Mat
object.
For instance,
XF_TNAME(XF_8UC1,XF_NPPC8) resolves to ap_uint<64>.

XF_DTUNAME(TYPE,NPPC) resolves to the data type of the pixel.
For instance,
XF_DTUNAME(XF_32FC1,XF_NPPC1) resolves to float.

XF_PTSNAME(TYPE,NPPC) resolves to the ‘C’ data type of the pixel.
For instance,
XF_PTSNAME (XF_16UC1,XF_NPPC2) resolves to unsigned short.

+++++++++++++++++++++++++++++++++++++++++++++++++++
gaussian filter

在头文件xf_config_params.h中，定义各种typedef，方便后续使用。

#define FILTER_SIZE_3 1
#define FILTER_SIZE_5 0
#define FILTER_SIZE_7 0
#define RO 0
#define NO 1

#if NO
#define NPC1 XF_NPPC1
#endif
#if RO
#define NPC1 XF_NPPC8
#endif

在头文件xf_gaussian_filter.hpp中，定义DUT的原型模版代码。

template<int FILTER_SIZE, int BORDER_TYPE, int SRC_T, int ROWS, int COLS,int NPC = 1>
void GaussianBlur(
				xf::Mat<SRC_T, ROWS, COLS, NPC> & _src,
				xf::Mat<SRC_T, ROWS, COLS, NPC> & _dst, 
				float sigma)
{
	//function body
}

模版函数是最重要的功能代码，只是以模版的形式出现，方便代码复用。

在源码文件xf_gaussian_filter_accel.cpp中，实现DUT。

#include "xf_gaussian_filter_config.h"

void gaussian_filter_accel(	
				xf::Mat<XF_8UC1,HEIGHT,WIDTH,NPC1>&imgInput,
				xf::Mat<XF_8UC1,HEIGHT,WIDTH,NPC1>&imgOutput,
				float sigma)
{
	xf::GaussianBlur<FILTER_WIDTH, XF_BORDER_CONSTANT, XF_8UC1, HEIGHT,
			WIDTH, NPC1>(imgInput, imgOutput, sigma);
}

实现函数是一个wrapper，在里面对模版函数进行具象实例化。

在源码文件xf_gaussian_filter_tb.cpp中，定义TB。

int main(int argc, char **argv)
{
	cv::Mat in_img, out_img, ocv_ref;
	cv::Mat in_gray, in_gray1, diff;
	
	in_img = cv::imread(argv[1], 1); // reading in the color image
	extractChannel(in_img, in_gray, 1);
	
	xf::Mat<XF_8UC1, HEIGHT, WIDTH, NPC1>  imgInput(in_img.rows,in_img.cols);
	xf::Mat<XF_8UC1, HEIGHT, WIDTH, NPC1>  imgOutput(in_img.rows,in_img.cols);
	
	imgInput.copyTo(in_gray.data);
	
	gaussian_filter_accel(imgInput,imgOutput,sigma);
	
	xf::imwrite("hls_out.jpg",imgOutput);
}

main函数就是TB，
首先通过imread读取文件，并存储到cvmat中，
然后通过extractchannel，抽取出灰度图片，存在另一个cvmat中，

然后定义了两个xfmat对象，用来以xfmat的格式存储图片，
然后是使用copyto成员函数，将cvmat中的数据读取到xfmat中。

然后是调用DUT，传递激励对象和响应对象的句柄给DUT。
当DUT运行结束后，DUT输出的结果已经在响应对象中了。

最后，通过imwrite将xfmat的中图片，写入文件。

+++++++++++++++++++++++++++++++++++++
Additional Utility Functions for Software

----xf::imread
The function xf::imread loads an image from the specified file path, copies into xf::Mat and
returns it.If the image cannot be read (because of missing file, improper permissions, unsupported or invalid format), the function exits with a non-zero return code and an error statement.

template<int PTYPE, int ROWS, int COLS, int NPC>
xf::Mat<PTYPE, ROWS, COLS, NPC> imread (char *filename, int type)

In an HLS standalone mode like Cosim, use cv::imread followed by copyTo function, instead of
xf::imread.

in_img = cv::imread("hls_input_image.jpg", 1); // reading in the color image
imgInput.copyTo(in_gray.data);

-----xf::imwrite
The function xf::imwrite saves the image to the specified file from the given xf::Mat.
This function internally uses cv::imwrite for the processing. Therefore, all the limitations of cv::imwrite are also applicable to xf::imwrite.

template <int PTYPE, int ROWS, int COLS, int NPC>
void imwrite(const char *img_name, xf::Mat<PTYPE, ROWS, COLS, NPC> &img)

++++++++++++++++++++++++++++++++++++++++
xfOpenCV Library Functions
The xfOpenCV library is a set of select OpenCV functions optimized for Zynq-7000 and Zynq
UltraScale+ MPSoC devices. The following table lists the xfOpenCV library functions.

+++++++++++++++++++++++++++++++++++++++++++
Video Processing Functions
The following table summarizes the video processing functions ported from HLS Video Library
into xfOpenCV Library along with the API modifications.

All the functions except Reduce can process N-pixels per clock where N is power of 2