Computer Vision 2023 Assignment 1: Image Filtering

Computer Vision 2023 Assignment 1: Image Filtering

1. Overview

In this prac you will research, implement and test some image filtering operations. Image filtering Q1703105484 by convolution is a fundamental step in many computer vision tasks and you will find it useful to have a firm grasp of how it works. For example, later in the course we will come across Convolutional Neural Networks (CNNs) which are built from convolutional image filters.

The main aims of the prac are:

• • to understand the basics of how images are stored and processed in memory;
  • to gain exposure to several common image filters, and understand how they work;
  • to get practical experience implementing convolutional image filters;
  • to test your intuition about image filtering by running some experiments;
  • to report your results in a clear and concise manner.

This assignment relates to the following ACS CBOK areas: abstraction, design, hardware and software, data and information, HCI and programming.

1. Setup

In this assignment, you will write a report and some code. While you are required to submit both, you will be graded based primarily on your report.

There are many software libraries that can perform image filtering supporting various languages and environments. For this prac you should use python, and to that end we provide code stubs and and a jupyter notebook (ipynb file) to get you started. You must use this setup so that we can verify code and/or provide help if you get your setup wrong.

The notebook can be uploaded and run in a google colab environment, with all processing and file management happening on a remote server, mostly invisible to you. Google colab provides a web interface/environment for your python code development that looks a lot like the jupyter notebook interface. This will be the easiest way to do the work.

However if you prefer to be able to run locally, you need to set up python and jupyter notebooks on your local computer. The easiest way to do this is to install anaconda, a nicely packaged version of python that comes with Jupyter notebooks built in. You run jupyter or jupyter-lab as a server on your local machine and then communicate with that through a web- browser which provides the interface to the IDE and the language. There may be additional packages you have to install, such as opencv, which do not come as part of anaconda.

More detailed instructions on these setups is provided in a separate document available through the MyUni course pages, and will also be discussed in workshops. There are, of course, also lots of online tutorials about colab, jupyter notebooks and anaconda.

1. Instructions

This assignment is divided into several tasks. For each task, you should write a corresponding section in your report that addresses all questions posed. In most cases, this will be one or more images and a small amount of text and maybe code fragment to explain them and describe what you did.

Make sure to read through the whole of the assignnment before tackling the tasks in order.

Task 1: Loading and displaying an image (10%)

Create or download several (minimum 3) test images that you will apply your filters to. Choose your images so that they will test different aspects of your filters. Include at least one greyscale image and one RGB colour image. Show each image in your report, and explain why you have chosen it.

1. Write some code to read an image from file and display it. Apply it to your test images and then apply some simple point processes such as those described in Lecture 1, and observe the results. What happens when a processed pixel value becomes < 0 or > 255, and what effect does this have on later processing?
2. Repeat the above for a RGB test image, but this time, apply the point process only to one channel of the image (red, green or blue). Display the resulting RGB image.

Task 2: Image processing

Now that you can load and display images, let’s try some operations on them. Note that there are stubs for each of these functions in a1code.py

1. Implement a crop() function to create a new image that is a small crop from the larger image
2. Implement a resize() function that works by sampling from the original image
3. Implement a change contrast() function
4. Implement a greyscale() to convert an RGB image into monochrome
5. Implement a binary() function that thresholds an image based on an intensity thresh- old. What do you observe when you change the threshold of the binary function?

Apply all these functions with different parameters on your own test images.

Task 3: Image filtering

See section 3.2 of the textbook [1] for background information on this question.

1. Using the definition of 2D convolution from week 1, implement the convolution oper- ation in the function conv2D().
2. In the function conv, extend your function conv2D to work on RGB images, by applying the 2D convolution to each channel independently.

1. Use the gauss2D function provided in a1code.py to create a Gaussian kernel, and apply it to your images with convolution. You will obtain marks for trying different tests and analysing the results, for example:
   • try varying the image size, and the size and variance of the filter
   • subtract the filtered image from the original What do you observe and why?
2. Define a horizontal and vertical Sobel edge filter kernel and test them on your images. You will obtain marks for testing them and displaying results in interesting ways, for example:
   • apply them to an image at different scales
   • considering how to display positive and negative gradients
   • apply different combinations of horizontal and vertical filters, such as applying the vertical Sobel filter to the output of the horizontal Sobel filter

Task 4: Image sampling and pyramids

NOTE: image sampling will be covered in Lecture 2. See sections 3.5.1 and 3.5.2 of the textbook [1] for this question.

1. Apply your resize() function to reduce an image to 0.5 height and width (i.e. down- sample the image)

Repeat the above procedure, but apply a Gaussian blur filter to your original image before downsampling it. How does the result compare to your previous output, and to the original image? Why?

1. Create a Gaussian pyramid as described in week2’s lecture on an image.

Apply a Gaussian kernel to an image I, and resize it with ratio 0.5, to get I1. Repeat this step to get I2, I3 and I4.

Display these four images in a manner analogus to the example shown in the lectures.

1. Assessment

Hand in your report and supporting code/images via the MyUni submission page for the Assignment. Upload two files:

1. report.pdf, a PDF version of your report
2. code.zip a zip file containing your code and images

Make sure you include all the code you’ve written and your test images. Your mark will be based primarily on your report. The code will be used to check your work and ensure it has been done independently and not plagiarised.

The prac is worth 20% of your overall mark for the course. It is due on Thursday March 23 at 11.59pm.

Ian Reid March 2023

References

[1] Richard Szeliski, Computer Vision: Algorithms and Applications, Springer, 2011. http:

//szeliski.org/Book/