CS 259 Accelerating Convolutional Neural Network

Fall 2024 
CS 259 Lab 1 
Accelerating Convolutional Neural Network (CNN) on FPGAs using 
Merlin Compiler 
Due October 9 11:59pm 
Description 
Your task is to accelerate the computation of two layers in a convolutional neural network 
(CNN) using a high-level synthesis (HLS) tool on an FPGA. We encourage you to start with 
using the Merlin Compiler. For an input image with 228 × 228 pixels and 256 channels, you 
are going to calculate the tensor after going through a 2D convolution layer and a 2D max 
pooling layer. The convolution layer has 256 filters of shape 256 × 5 × 5, uses the ReLU 
activation relu(x) = max{x, 0} with a bias value for each output channel. The 2D maxpooling
 layer operates on 2 × 2 non-overlapping windows. You will need to implement this 
function using HLS: 
void CnnKernel(const float* input, const float* weight, const float* bias, float* 
output) 
where input is the input image of size [256][228][228], weight stores the weights of the 
convolution filters of size [256][256][5][5], bias stores the offset values of size [256] that 
will be added to the output channels, and output should be written to by you as defined 
above to store the result of maxpool(relu(conv2d(input, weight) + bias)). The output 
size is [256][112][112]. 
How-To 
FPGA accelerator compilation typically involves three (3) stages: high-level synthesis (HLS), 
bitstream generation, and onboard execution. The last two stages can take days to 
complete. Therefore, in this lab, we only focus on the first stage: HLS. Your performance will 
only be assessed using the estimation in the HLS reports, which is usually accurate. 
However, you are welcome to try out the last two steps if you are interested. 
 
 
 
Connecting to the Server: Method 1 
In this method, you won’t be able to run Merlin directly from your /home directory, so you’ll need 
to copy files back and forth. 
1. Connect to the server (VPN may be required). You can find VPN details here: 
https://www.it.ucla.edu/it-support-center/services/virtual-private-network-vpn-clients  
ssh <username>@brimstone.cs.ucla.edu 
 
2. Start the Docker container and share your home with –v: 
 
docker run -v /d0/class/:/home -it vitis2021 /bin/bash 
 
3. Source Vitis, navigate to the desired directory and clone the repository: 
 
source /tools/Xilinx/Vitis_HLS/2021.1/settings64.sh 
cd /opt 
git clone https://github.com/UCLA-VAST/cs-259-f24.git 
cd cs-259-f24/lab1 
 
4. Copy the necessary file to your home directory: 
 
cp /opt/cs-259-f24/lab1/cnn-krnl.cpp /home/<username> 
Connecting to the Server: Method 2 
In this method, you can run Merlin directly from your /home directory, but make sure to export your 
home directory. 
 
1. Connect to the server (VPN may be required). You can find VPN details here: 
https://www.it.ucla.edu/it-support-center/services/virtual-private-network-vpn-clients 
 
ssh <username>@brimstone.cs.ucla.edu 
 
2. Start the Docker container and share your home with –v: 
 
docker run --user $(id -u):100 -v /d0/class/:/home -it vitis2021 /bin/bash 
 
3. Export your home directory: 
 
export HOME=/home/<username> 
 
4. Source Vitis, navigate to your home directory and clone the repository: 
 
source /tools/Xilinx/Vitis_HLS/2021.1/settings64.sh 
cd /home/<username> 
git clone https://github.com/UCLA-VAST/cs-259-f24.git 
cd cs-259-f24/lab1 
Build and Run Baseline with Software Simulation 
We have prepared the starter kit for you. Please run: make 
This command will perform a software simulation of the provided starter FPGA HLS kernel. It 
should show “PASS”. You need to use FPGA Developer AMI in this lab unless you are using 
a computer with Xilinx Vitis HLS installation. However, you are still suggested to develop code 
and run software simulation locally to test the correctness. You can move to AWS once you 
enter the tuning stage. 
Understand the automatic Merlin’s optimization 
Before modifying the kernel and adding pragmas, synthesize the CNN kernel with Merlin and 
describe in your report the automatic optimizations made by Merlin and how this reduces 
latency. 
Modify the HLS CNN Kernel 
If you have successfully built and run the baseline HLS CNN kernel, you can now optimize 
the code to design your CNN kernel. Your task is to implement a fast, parallel version of the 
CNN kernel on FPGA. You should start with the provided starter kit. You should edit cnnkrnl.cpp
for this task. When editing, please use the given types input_t, weight_t, bias_t, 
and output_t for the corresponding data, and compute_t for your intermediate values. 
You can use them as if they are float numbers. 
Parallelism should be exploited by using Merlin pragmas and tiling. You are encouraged to 
focus on Merlin pragmas (#pragma ACCEL parallel, #pragma ACCEL pipeline and #pragma 
ACCEL tile). You can explicitly modify the code (tiling, loop permutation, …) but make sure 
the code modified is correct. 
In the starter kit, we simply wrap a sequential CNN code with #pragma ACCEL kernel, and 
Merlin automatically performs data caching, memory coalescing, pipelining and 
parallelization, which yield about 10 GFLOPs. 
Although the skeleton kernel is provided, you are also free to create your own by removing 
the header file inclusion of “lib/cnn-krnl.h” and implement the basic kernel from scratch. 
However, this would require specific expertise in Xilinx FPGA architecture and is not 
recommended for this course. 
Test Your HLS CNN Kernel with Software Simulation 
To perform software emulation of your FPGA implementation of CNN kernel: 
make 
If you see something similar to the following message, your implementation is incorrect. 
Found 2120190 errors 
FAIL Since the software simulation step uses the CPU to emulate the hardware behavior, it only 
serves as correctness test and its execution time doesn’t reflect that of actual hardware. Your 
estimated execution time should be retrieved using the command below: 
make estimate 
This command will print out the estimated latency and resource usage of your kernel: 
+---------------------------+------------------------+----------+----------+---------+--------+-------+------+ 
| Kernel | Cycles | LUT | FF | BRAM | DSP | URAM |Detail| 
+---------------------------+------------------------+----------+----------+---------+--------+-------+------+ 
|CnnKernel (cnn-krnl.cpp:12)|4179564052 (16718.256ms)|49558 (4%)|49381 (2%)|810 (18%)|202 (2%)|25 (2%)|- | 
+---------------------------+------------------------+----------+----------+---------+--------+-------+------+ 
The time highlighted in yellow is the estimated execution time of your FPGA kernel. You can 
get the performance by “kNum*kNum*kImSize*kImSize*kKernel*kKernel*2/latency”, or 
164.4/latency (in s) to get the performance in GFLOPS. 
IMPORTANT: Please make sure that all your loops have fixed loop bounds. If any of the loop 
bounds are variable, a performance estimation will not be shown and you will receive no 
performance grade. 
IMPORTANT: The “make estimate” command should finish in 30 minutes, or in two hours 
with highly-complex optimizations. Our recommendation is to halt your estimation using 
Ctrl-C when the time exceeds 30 minutes, except for your last step (after you reach ~100 
GOPS). More than 12 hours in the estimation will result in zero for the performance score. 
As your kernel design becomes more complex, the software simulation and the estimation 
will start to take a significantly longer time. 
IMPORTANT: As you apply more optimizations, your resource usage will also increase. 
Ideally, you should keep applying optimization until your kernel occupies about 80% of these 
resources. The remaining 20% should be reserved for the interfaces (DRAM/PCI-e controller) 
and the downstream flows. Please make sure that resource utilization is less than 80% for all 
FPGA resources. If any of the resources are over this limit, you will receive no performance 
grade. 
IMPORTANT: You can check the HLS report by opening merlin.rpt with a text editor. This 
file will be generated with the command make estimate. You must submit this file with your 
final submission. You should not modify this file in your submission, and it will be all verified 
after submission due. Any modification to this file in your submission constitutes academic 
misconduct and will be reported. 
Advanced Tips for HLS 
Kernel Profiling: If you want to “profile” your kernel, you can open merlin.rpt with a text 
editor and scroll down to Performance Estimate. You can see the trip count, accumulated 
cycles and cycles per call, as well as pipeline initiation interval and parallel factor for each 
loop in the table. For resource usage, you can go to Resource Estimate. No loop level 
information is available, though. If you want to check the resource usage of a code region, 
you can wrap it with a function then run again. 
Kernel after transformation: If you want to see the kernel after being transformed by Merlin, 
you can look for that in .merlin_prj/run/implement/exec/hls/kernel. Annotation for Profiling: If you find the loops in your report hard to read, you can name the 
loops you are interested in using a goto label. For example, this_loop: for (int i = 0; 
i < n; i++); 
Debugging Pipelining: If you are not sure about why you cannot achieve a specific initiation 
interval as you expected, you can open the file below and read the logs. HLS usually gives out 
a reason. 
.merlin_prj/run/implement/exec/hls/_x/logs/CnnKernel/CnnKernel/vitis_hls.log 
Long Synthesis Time In Pipelining: You will experience long HLS synthesis time (for 
generating the estimation) if you pipeline a loop with a large loop body. Besides, please note 
that as all loops inside a pipeline will be unrolled, it may be automatically a large loop body. 
In this case, you may want to exchange the order of pipelining and unrolling and see if the time 
can get improved. 
Use Functions for Shorter Synthesis Time: If you experience long synthesis time, you may try 
wrapping some loops into a function and specify #pragma HLS inline off inside the 
function body. However, this may lead to inaccurate dependency analysis or memory port 
analysis and cause lower performance sometimes. There might be some workarounds, or 
not. For example, if you have access to A[k + i][j] inside the function, passing A + k to 
the function and accessing A’[i][j] can allow HLS to understand the array partitioning 
better than passing A. You need to do experiments. 
General Tips 
● When you develop on AWS, to resume a session in case you lose your connection, you 
can run screen after login. You can recover your session with screen -DRR. You should 
stop your AWS instance if you are going to come back and resume your work in a few 
hours or days. Your data will be preserved but you will be charged for the EBS storage 
for $0.10 per GB per month (with default settings). You should terminate your instance 
if you are not going to come back and resume your work. Data on the instance will be 
lost. 
● You are recommended to use private repositories provided by GitHub to backup your 
code. Never put your code in a public repo to avoid potential plagiarism. To check in 
your code to a private GitHub repo, create a repo first. 
git branch -m upstream 
git checkout -b main # skip these two lines if you are reusing the folder in Lab 1 
... // your modifications 
git add cnn-krnl.cpp merlin.rpt 
git commit -m "lab1: first version" # change commit message accordingly 
# please replace the URL with your own URL 
git remote add origin git@github.com:YourGitHubUserName/your-repo-name.git 
git push -u origin main 
● You are recommended to git add and git commit often so that you can keep track of 
the history and revert whenever necessary. 
● Make sure your code produces correct results! 
(Optional) Modify the HLS CNN Kernel using Vitis Pragmas 
You are encouraged to use mainly Merlin pragmas. If needed, you can use Vitis pragmas for 
finer-grained control and optimization. The list of pragmas in Vitis can be found here. You can simply write Vitis pragmas and Merlin pragmas in the same file (cnn-krnl.cpp), but note 
that, to apply an HLS pragma to a loop, you need to put the pragma inside the loop body 
instead of before it. 
Submission 
You need to report the estimated performance results of your FPGA-based implementation on 
a Xilinx Ultrascale+ VU9P FPGA (the FPGA we are using, specified in the makefile). Please 
express your performance in GFLOPS and the speedup compared with the starter-kit version. 
Your report should also include: 
● Please run the input C file through the Merlin Compiler, identify the code 
transformation and HLS pragmas that Merlin added, and discuss why. 
● Please explain the parallelization and optimization strategies you have applied for 
each loop in the CNN program (convolution, max pooling, etc) in this lab. Include the 
pragmas (if any) or code segments you have added to achieve your strategy. 
● Please incrementally evaluate each parallelization/optimization that you have applied 
and explain why it improves the performance. 
● Please report the FPGA resources (LUT/FF/DSP/BRAM) usages, in terms of resource 
count and percentage of the total. Which resource has been used most, in terms of 
percentage? 
● Optional: The challenges you faced, and how you overcame them. 
● (Bonus +5pts): Analyze your code and check if the DSP/BRAM resource usage 
matches your expectation. Only the adders, multipliers, and size of arrays need to be 
considered. Please attach related code segments to your report and show how you 
computed the expected number. Provide a discussion on possible reasons if they 
differ significantly. 
You also need to submit your optimized kernel code. Do not modify code in the lib directory. 
Please submit on Gradescope. Your final submission should contain and only contain these 
files individually: 
 ├ cnn-krnl.cpp 
 ├ merlin.rpt 
 └ lab1-report.pdf 
File lab1-report.pdf must be in PDF format. 
Grading Policy 
Your submission will only be graded if it complies with the formatting requirements. 
Missing reports/code or compilation errors will result in 0 for the corresponding 
category(ies). 
Correctness (40%) 
Please check the correctness using the command “make”. Performance (40%) 
Your performance will be evaluated based on the estimation report generated using the 
command “make estimate”. The performance point will be added only if you have the 
correct result, so please prioritize the correctness over performance. Your performance will 
be evaluated based on the ranges of throughput (GOPS). Ranges A+ and A++ will be defined 
after all the submissions are graded: 
● Range A++, better than Range A+ performance: 40 points + 20 points (bonus) 
● Range A+, better than Range A performance: 40 points + 10 points (bonus) 
● Range A GFLOPS [200, 280]: 40 points 
● Range B GFLOPS [120, 200): 30 points 
● Range C GFLOPS [60, 120): 20 points 
● Range D GFLOPS [30, 60): 10 points 
● Lower than range F [0, 30): 0 points 
 
Report (20%) 
Points may be deducted if your report misses any of the sections described above. 
Academic Integrity 
All work is to be done individually, and any sources of help are to be explicitly cited. You must 
not modify the HLS report merlin.rpt in your submission. Any instance of academic 
dishonesty will be promptly reported to the Office of the Dean of Students. Academic 
dishonesty includes, but is not limited to, cheating, fabrication, plagiarism, copying code from 
other students or from the internet, modifying the software-generated report, or facilitating 
academic misconduct. We’ll use automated software to identify similar sections between 
different student programming assignments, against previous students’ code, or against 
Internet sources. We’ll run HLS on all submissions and compare the reproduced HLS 
report with the submitted report. Students are not allowed to post the lab solutions on public 
websites (including GitHub). Please note that any version of your submission must be your 
own work and will be compared with sources for plagiarism detection. 
Late policy: Late submission will be accepted for 24 hours with a 10% penalty. No late 
submission will be accepted after that (you lost all points after the late submission time).