CSIL machines for multi-threaded programming

Assignment 1
Due Thursday by 11:59pm Points 60 Submitting a file upload
Available Sep 2 at 3pm - Dec 24 at 11:59pm
Start Assignment
Assignment 1 [60 Points]
Due: Friday September 12 at 11:59 PM
This assignment is a refresher for multi-threaded programming using pthreads. You have one problem in
this assignment and you are provided the source code (with a driver program for executing the solution),
the test script, and the expected output to be generated by your solution. You are expected to fill the
various functions (explained in detail below) in order to complete the assignment. You can find a tutorial
for pthreads at: https://computing.llnl.gov/tutorials/pthreads/
(https://computing.llnl.gov/tutorials/pthreads/) for a quick review.
You need to test and debug your program on CSIL machines. You can find instructions on how to run
remotely on CSIL machines at: https://www.sfu.ca/computing/about/support/csil/remote?access.html (https://www.sfu.ca/computing/about/support/csil/remote-access.html)
For a list of available CSIL machines to use:
https://www.sfu.ca/computing/about/support/csil/unix/how-to-use-csil-linux-cpu-server.html
(https://www.sfu.ca/computing/about/support/csil/unix/how-to-use-csil-linux-cpu-server.html)
Note: For future assignments, you may need to run on a private cluster that we'll provide information
about soon.
General Instructions
1. You are provided with source code to start with here: assignment1.tar.gz
(https://canvas.sfu.ca/courses/84236/files/24448327?wrap=1)
(https://canvas.sfu.ca/courses/84236/files/24448327/download?download_frd=1) . The compiler used is
`g++ 9.4.0` and is already installed on CSIL machines. To run the program, follow the steps below:
a. Run
 make producer_consumer
This creates a binary file called `producer_consumer`.
 b. Test the binary:
 Assignment 1
1/6
 ./producer_consumer --nItems 1000000 --nProducers 4 --nConsumers 2 --bufferSize 50000

 Note that `--nProducers` and `--nConsumers` are used to specify the number of producer and
consumer threads. So, the above command will create 6 threads. (Detailed description about the various
arguments for `producer_consumer` can be found below.)
2. You will be asked to print the time spent by different threads on specific code regions. The time spent
by any code region can be computed as follows:
timer t1;
? ? t1.start();
? ? /* ---- Code region whose time is to be measured --- */
 double time_taken = t1.stop();
3. Sample outputs for all the programs can be found in `sample_outputs` directory. Programs will be
evaluated and graded automatically. Please make sure that your program output strictly follows
the sample output format.
4. We have provided test scripts for you to quickly test your solutions during your development process.
You can test your code using the test script, `test_scripts/solution_tester.py`. Note that these test scripts
only validate the output formats, and a different evaluation script will be used for grading the
assignments.
Producer-Consumer Problem
In this problem, we want to create `p` producer threads that produce `n` items in total, and `q` consumer
threads consuming those items. Each producer thread produces n/p items (integer division). Any
remaining items will be produces by producer thread 0. For example, if n=1000 and p = 3, producers 1
and 2 will produce 333 items while producer 0 will produce 334 items. Each item has a unique value that
is determined by the producing thread ID and the total number of producers. In the previous example,
thread 0 will generate item values 0, 3, 6, 9,...; thread 1 will generate item values 1, 4, 7, 10,...; and
thread 2 will generate item values 2, 5, 8, 11,...
The produced items belong to one of three types (type 0, type 1 and type 2). Type 0 represents half the
items produced by each producer, Type 1 represents a third, and Type 2 represents the rest. For
example, if a thread needs to produce "items_to_produce" then it needs to produce
"items_to_produce/2" type 0 items, "items_to_produce/3" type 1 items, and the remaining from
items_to_produce are type 2 items.
The produced items are enqueued in a buffer queue called `production_buffer` (the implementation for
the queue is available in `core/circular_queue.h`), and are dequeued for consumption. Each element in
the queue has an item value, an item type, and its producer thread ID. Both producer and consumer
threads operate concurrently. Since multiple producer threads enqueue the items and multiple consumer
 Assignment 1
2/6
threads dequeue the items from the *same queue*, you should use mutex when accessing the queue
(critical section).
In addition, care must be taken to carefully handle when the `production_buffer` is full or empty to avoid
overflow (items cannot be produced when queue is full) and underflow (items cannot be consumed when
queue is empty). Each producer should correctly produce its items (total n for all producers) and the total
number of items consumed by all the consumers should be equal to the total number of items produced.
You are expected to use conditional variables as required to ensure correctness and to prevent any
deadlocks or infinite waiting.
In order to evaluate correctness, we also compute the *value* of items produced/consumed by each
thread. For producers, the *value produced* is the sum of all the items produced (and enqueued) and for
consumers, the *value consumed* is the sum of all the items consumed. Each consumer also needs to
keep track of the total value it consumed from each type. The total value of items produced for a certain
type should equal the total number of items consumed from that type by all consumer threads.
The logic for both the producer and consumer threads is shown below. Make sure your solution retains
the *long* datatype for variables, as shown below:
**Producer threads:**

 // also update item count and value for item type produced
else {
 // production_buffer is full, so block on conditional variable waiting for consumer
to signal.}
// After production is completed:
// Update the number of producers that are currently active.
 --active_producer_count;
// The producer that was last active (can be determined using `active_producer_count`) will
keep signalling the consumers until all consumers have finished (can be determined using `active_con
sumer_count`). }
**Consumer threads:**
 consumerFunction(){
// Each consumer dequeues items from the `production_buffer`?
 long item;
 Assignment 1
3/6
 long items_consumed = 0;
long value_consumed = 0; 
while(true) {
bool ret = production_buffer.dequeue(&item); // item includes value and type
if(ret == true) {
 if(production_buffer.itemCount() == production_buffer.getCapacity() - 1){
// The queue is no longer full
 // Signal all producers indicating queue is not full }
 value_consumed += item;
 items_consumed++;
 // also update value and count for consumed item type }
 else {
// production_buffer is empty, so block on conditional variable waiting for producer
to signal.
 // The thread can wake up because of 2 scenarios:
// Scenario 1: There are no more active producers (i.e., production is complete) and
the queue is empty. This is the exit condition for consumers, and at this point consumers should dec
rement `active_consumer_count`.
// Scenario 2: The queue is not empty and/or the producers are active. Continue cons
uming.

In addition, you are provided the following methods for the `ProducerConsumerProblem` class in
`solution.cpp` file. 
1. `ProducerConsumerProblem()` and `~ProducerConsumerProblem()` - Constructor and destructor for
the `ProducerConsumerProblem` class. You can initialize/destroy your mutexes and conditional variables
here.
2. `startProducers()` and `startConsumers()` - Methods where producer and consumer threads should be
created respectively (use `pthread_create()` here).
3. `joinProducers()` and `joinConsumers()` - Methods where producer and consumer threads should be
joined with the main thread.
4. `printStats()` - Method to print statistics of the producer and consumer threads (see output
requirements below).
**NOTE: You should not change the signature (i.e., function names, parameters, return types) of
these 7 methods. To evaluate your solutions, we will use different driver programs that will rely
on the provided function signatures. You can define additional structs, classes or functions in
solution.cpp or solution.h as necessary.**
The driver program (in `driver.cpp`) processes the following command-line parameters, and calls various
methods of the `ProducerConsumerProblem` class.
1. `--nItems`: the total number of items produced all producers.
2. `--nProducers`: the number of producer threads.
3. `--nConsumers`: the number of consumer threads.
4. `--bufferSize`: the size of the buffer used for `production_buffer`.
 Assignment 1
4/6
Your solution must satisfy the following:
1. The executable should support the 4 command-line parameters mentioned above. This is already
taken care by `driver.cpp` and make sure your solution is not dependent on any other command-line
parameters since the driver program used for evaluation may not be the same.
2. Output the following (sample output can be found in `sample_outputs/producer_consumer.output` file):
 1. The total number of producer and consumer threads.
 2. For each producer thread: the producer id, the number of items produced by the producer for type
0, the total value of the items produced for type 0, the number of items produced by the producer for
type 1, the total value of the items produced for type 1, the number of items produced by the producer
for type 2, the total value of the items produced for type 2, total value produced for all types, the time
taken by the producer thread (i.e. time between the start and end of the producer function).
 3. For each consumer thread: the consumer id, the number of items consumed by the consumer for
type 0, the total value of the items consumed for type 0, the number of items consumed by the consumer
for type 1, the total value of the items consumed for type 1, the number of items consumed by the
consumer for type 2, the total value of the items consumed for type 2, the time taken by the consumer
thread (i.e. time between the start and end of the consumer function), and the total value consumed from
each producer thread (thread 0, thread 1 etc.).
 4. The total number of items consumed by all consumers.
 5. The total time taken (this is already printed by the driver program).
Please note that the output format should strictly match the expected format (including "spaces" and
"commas"). You can test your code using the test script as follows:
 $ python test_scripts/solution_tester.pyc --execPath=<absolute path of producer_consumer executa
ble file> --scriptPath=./test_scripts/solution_evaluator.pyc
Submission Guidelines
Make sure that your solutions folder has the following files and sub-folders. Let's say your solutions
folder is called `my_assignment1_solutions`. It should contain:
 `core/` -- The folder containing all core files. It is already available in the assignment package. Do not
modify it or remove any files.
 `Makefile` -- Makefile for the project. This file should not be changed.
 `driver.cpp`-- Main program for the assignment. This file should not be changed.
 `solution.cpp`-- You should modify this file
 `solution.h`-- You should modify this file
To create the submission file, follow the steps below:
 1. In your solutions folder, remove all the object/temporary files.
 Assignment 1
5/6
  $ cd my_assignment1_solutions/
$ make clean 
 $rm *.gz
 2. Create the tar.gz file. 
 $ tar cvzf assignment1.tar.gz *
 which creates a compressed tar ball that contains the contents of the folder.
 3. Validate the tar ball using the `submission_validator.py` script.
 $ python test_scripts/submission_validator.pyc --tarPath=<full path to tarball>/assignment1. 4. Submit the tar.gz file via canvas before the deadline.