Recurrent Neural Networks

Problem 2: Recurrent Neural Networks

Recurrent neural networks (RNNs) are deep-learning architectures which are particularly

powerful in dealing with time-series (e.g., financial) and natural language data. In this problem,

you are going to write a program to simulate the basic forward propagation mechanism in an

RNN using simple 1D arrays. Note that you do not need to understand what an RNN is to

solve this problem; you will find all the necessary information that you need to solve this

problem in the following description.

An RNN cell at time t takes an input x t and a hidden state h t from the previous cell, and

computes a next state h t +1 and an output o t .

Specifically, each RNN cell computes o t and h t +1 using the following two functions: o t = sigmoid(0.1 x t + 1.5 h t )

h t +1 = tanh(0.5 x t − 2 h t )

where

sigmoid( x ) = 1/(1 + e − x )

tanh( x ) = 2 × sigmoid(2 x ) − 1

and e = 2.72 .

Multiple RNN cells can then be connected to form a network. The following figure depicts an

RNN of T cells that accepts an input sequence x 0 , x 1 ,..., x T −1 and an initial hidden state h_0

as inputs, and outputs a sequence o 0 , o 1 , ..., o T −1 .

Write a C++ program that implement the above simple RNN.

Input:

• the first line contains two numbers: an integer T denoting the recurrent times (

1 ≤ T ≤ 100 ) and a floating-point number h 0 denoting the initial hidden state;

and

• the second line contains the input sequences which are T floating-point numbers

x 0 , x 1 ,..., x T −1 .

Output:

• Your program should display the hidden state sequences h 1 , h 2 , ..., h T in the

first line and the output sequences o 0 , o 1 , ..., o T −1 in the second line. • Floating-point numbers are displayed with 10 decimal places. Use

setprecision(n) defined in the header <iomanip> to set the precision

parameter of the output stream.

Requirement:

• You will need to complete the following functions in the provided template

2.cpp . Read the code in the template carefully to see what have been provided

for you. You will find details of the function prototypes in the in-code comments

too.

◦ sigmoid() for sigmoid activation function

◦ tanh() for tanh activation function

◦ ComputeH() for computing the next hidden value in RNN cell

◦ ComputeO() for computing the output value at current time step

◦ PrintSeqs() for printing the values in a 1D array to screen

◦ main() for main function

• Use 1D arrays to store the sequences for x i , h i and o i .

• You can ONLY use the simple data types char , bool , int , double and

arrays. In other words, you are not allowed to use other data types or data

structures such as strings or STL containers (e.g., vectors), etc.

Sample Test Cases

User inputs are shown in blue .

2_1:

1 1.0

1.0

-0.9053193842

0.8321596401

2_2: 5 0.0

1.0 2.0 3.0 4.0 5.0

0.4623655914 0.0751742883 0.8741722520 0.2466235712 0.9645899021

0.5249949450 0.7097382221 0.6018123354 0.8471395064 0.7048466172

2_3:

3 1.0

0.0 0.0 0.0

-0.9641167571 0.9586890814 -0.9578009493

0.8177157977 0.1904499845 0.8082908051