NodeJS Notes

NodeJS

A web server is a computer hosting one or more websites. "Hosting" means that all the web pages and their supporting files are available on that computer. The web server will send any web page from the website it is hosting to any user's browser, per user request.

A static web server, or stack, consists of a computer (hardware) with an HTTP server (software). We call it "static" because the server sends its hosted files "as-is" to your browser.

When a user wants to navigate to a page, the browser sends an HTTP GET request specifying the URL of its HTML page. The server retrieves the requested document from its file system and returns an HTTP response containing the document and an HTTP Response status code of "200 OK" (indicating success). The server might return a different status code, for example "404 Not Found" if the file is not present on the server, or "301 Moved Permanently" if the file exists but has been redirected to a different location.

The server for a static site will only ever need to process GET requests, because the server doesn't store any modifiable data. It also doesn't change its responses based on HTTP Request data (e.g. URL parameters or cookies). 

 

A dynamic web server consists of a static web server plus extra software, most commonly an application server and a database. We call it "dynamic" because the application server updates the hosted files before sending them to your browser via the HTTP server.

After the coach submits the form with the team name and number of players, the sequence of operations is:

1. The web browser creates an HTTP GET request to the server using the base URL for the resource (/best) and encoding the team and player number either as URL parameters (e.g. /best?team=my_team_name&show=11) or as part of the URL pattern (e.g. /best/my_team_name/11/). A GET request is used because the request is only fetching data (not modifying data).
2. The Web Server detects that the request is "dynamic" and forwards it to the Web Application for processing (the web server determines how to handle different URLs based on pattern matching rules defined in its configuration).
3. The Web Application identifies that the intention of the request is to get the "best team list" based on the URL (/best/) and finds out the required team name and number of players from the URL. The Web Application then gets the required information from the database (using additional "internal" parameters to define which players are "best", and possibly also getting the identity of the logged in coach from a client-side cookie).
4. The Web Application dynamically creates an HTML page by putting the data (from the Database) into placeholders inside an HTML template.
5. The Web Application returns the generated HTML to the web browser (via the Web Server), along with an HTTP status code of 200 ("success"). If anything prevents the HTML from being returned then the Web Application will return another code — for example "404" to indicate that the team does not exist.
6. The Web Browser will then start to process the returned HTML, sending separate requests to get any other CSS or JavaScript files that it references (see step 7).
7. The Web Server loads static files from the file system and returns them to the browser directly (again, correct file handling is based on configuration rules and URL pattern matching).

 

NodeJS, javascript runtime, the server-side Library (backend)

 

Node basics

core modules

• http->launch a server, send requests
• https->launch a ssl server
• fs->file system module, can be used to write into a new file
• path
• os

root file name app.js

create a node server

// import module
// require("pathAndFileNameWithOutExtension")
const http = require('http');

// node will execute this function whenever request reach our server.
// event driven architecture( if x happen, do y)
const server = http.createServer((req,res)=>{
    console.log(req.url, req.method, req.headers);

    res.setHeader();
    res.write();
    res.end();
});

// start a process, listen for the incoming requests
server.listen(3000);

// quit the server
process.exit();

 Parse request body

 

 

npm (node package manager)

• npm init
  • create a new package.json

• package.json
  • hold meta data important, it contains the dependencies (packages requires)
• --save
  • into package.json file if we have one
• npm start
  • add "start": "node app.js" into "scripts"
  • or add "nodemon app.js"
• npm run xxx
  • add "xxx": "yyy" into "scripts"
• npm install -g nodemon
  • automatically restarting of your node server
  • -g global installation

 

Express

const express = require('express');
const app = express();

const bodyParser = require('body-parser');

//however it will not parse all kinds of bodies, but body sent through a form
app.use(bodyParser.urlencoded({extended:false}));

// next is a function, a function will be passed into this arrow function by express.
// executed to allow the request to travel on to the next middleware.
// app.use([path,] callback [,callback...])

app.use((req,res,next)=>{
    
    next();
}); 

app.use((req,res,next)=>{
    // req.body is an object contains all the data from the request body.
    // need to be parsed using body-parser package
    req.body
  

    //send response back
    res.send();
    res.redirect()
}); 

// listen function createServer and listen get called.
app.listen();

app.use() allows us to add new middleware functions

requests

• req.body is an object made from the fields inputted from a form and that form has a method POST with different inputs, all of those inputs will be parsed by body parser and placed in.

• req.query is an object made from the fields inputted via a query string (the string after the '?' in the url).
• req.params is an object made from the fields inputted via the url path.

Routes

listen for the requests and run the other code depending on the request received on the server.

Order of routes matters! first route matches the request will be the only route that run.

var express = require("express");
var app = express();

// "/" => "Hi there!"
// root path, call back function with two objects request and response
// request trigger this route
// reponse with the request
// only for get routes
// get,post will have exact path matching here ('/'), but use won't
app.get("/", (req,res,next) => {
    res.send("Hi there!");
})

// looking for a pattern using ":" route(path) variable, but work only inside one slash
app.get("/r/:name/comments/:id/:title", (req,res,next) => {
    // to get access to what is the content for the name, use
    var subpath = req.params.name
    res.send("Hi there!" + subpath);
})

// catch every other route than defined route
app.get("*", (req,res,next) => {
    res.send("Star!");
})

// adding a 404 error page, handle all http requests
app.use((req,res,next) => {
    res.status(404).send('<h1>Page not found </h1>');
});

// send new data to the server
app.post();


// tell express to listen for requests (start server)
// localhost:3000, fill in 3000, otherwise process.env.PORT
app.listen(3000, process.env.IP, () => {
    console.log("Server has started!");
});

Express Router

const router = express.Router();

router.use();
router.get();
router.post();

module.exports = router;

const adminRoutes = require('routeFilePath');

app.use(adminRoutes);

Filtering paths

// only routes start with /admin will go into admin routes
// and in adminRoutes, /admin part will be ignored.
app.use('/admin',adminRoutes);

Server HTML pages

// nodejs module
const path = require('path');
// __dirname is absolute path
res.sendFile(path.join(__dirname, '../', 'views', 'shop.html'));

Helper function for navigation

path to the root file

const path = require('path')

module.exports = path.dirname(process.mainModule.filename);

const rootDir = require('../util/path');

res.sendFile(path.join(rootDir, '../', 'views', 'shop.html'));

 

Serving files statically

not handled by express router or middleware, but directly forwarded to the file system.

basically a  folder which we grant read access to only.

tell express to serve the content in the directory

app.use(express.static(path.join(__dirname, "public")));

 

MVC (model, view, controller)

 

Sessions and Cookies

With cookies we can store data in the browser of a single user and store data in that browser which is customized to that user which does not affect all the other users but can be sent with requests.

 

res.setHeader('Set-Cookie','loggedIn=true');

npm install --save express-session

 

const session = require('express-session'); 

app.use(session({secret: 'my secret', resave: false, saveUninitialized: false, store: store}));

 store sessions in MongoDB

npm install --save connect-mongodb-session

const MongoDBStore =require('connect-mongodb-session')(session);
const store = new MongoDBStore({
    uri: 
    collection: 'sessions'
});

store login flag and user in  the session when login

req.session.isLoggedIn = true;
req.session.user = user;

clear session when logout

req.session.destroy();

 

Authentication

 encrypt the password

npm install --save bcryptjs

// asynchronous task
bcrypt.hash(password, 12);

// return it can chain it into a promise


// compare the password matches the one stored in the database
bcrypt.compare(password, user.password);

Route Protection

// as a middleware and add to each router
module.exports = (req,res,next) => {
    if(!req.session.isLoggedIn){
        return res.redirect('/login');
    }
    next();
}

CSRF attacks

session stolen

npm install --save csurf

generate csrf token, value embed in our forms. in the server, check the  token.

const csrf = require('csurf');
const csrfProtection = csrf();
app.use(csrfProtection);
req.csrfToken();

// use in middleware
app.use((req,res,next) => {
  // set local variables passed into the views
  res.locals.isAuthenticated = req.session.isLoggedIn;
  res.locals.csrfToken = req.csrfToken();
  next();
});

// attached to each form
<input type="hidden" name="_csrf" value="<%= csrfToken %>">

Error message

store error message in the session, after used, remove from the session.

npm install --save connect-flash

const flash = require('connect-flash');
app.use(flash());

req.flash('error', 'Invalid email or password');


res.render("auth/signup", {
    path: "/signup",
    pageTitle: "Signup",
    errorMessage: message
  });

% if(errorMessage){ %>
            <div class="user-message user-message--error"><%= errorMessage %></div>
        <% } %>

Send emails

 

SendGrid

npm install --save nodemailer nodemailer-sendgrid-transport

const nodemailer = require('nodemailer');
const sendgridTransport = require('nodemailer-sendgrid-transport');

const transporter = nodemailer.createTransport(sendgridTransport({
  auth: {
    api_key: ''
  }
}));

transporter.sendMail({
            to: email,
            from: "admin@random.com",
            sbuject: "Signup succeeded!",
            html: "<h1>You sucessfully signed up!</h1>"
          });

 

 

Validation

 

 express validator

npm install --save express-validator

Errors

const error = new Error("Validation failed, entered data is incorrect.");
error.statusCode = 422;
throw error;

try{

}catch () {

} 


// if in then block throw the error, the next catch block will catch the error and the error will be passed to the catch block.
.then()


.catch(err => {
    if(!err.statusCode){
      err.statusCode = 500;
    }
    // go to the next error handling middleware
    next(err);
  });

 error handling middleware

// error handling middleware
// will be called when next(error)
// where error = new Error(err); 
app.use((error, req, res, next) => {


});

In synchronous places,  outside of callbacks and promises, you throw an error and express will detect this and execute your next error handling middleware.

Inside of that, you have to use next with an error included.

next(new Error(err))

upload file

multer

multer is able to accept our incoming data, extract the files and store them for us and then store information about the file upload in this req.file object.

 

npm install --save multer

// contain mixed data
enctype="multipart/form-data"

app.use(multer().single('image'));

Streaming data

better for bigger files.

const file = fs.createReadStream(invoicePath);
      res.setHeader("Content-Type", "application/pdf");
      res.setHeader(
        "Content-Disposition",
        'attachment; filename="' + invoiceName + '"'
      );
      file.pipe(res);

Create PDF

npm install --save pdfkit

      // readable stream
      const pdfDoc = new PDFDocument();
      res.setHeader("Content-Type", "application/pdf");
      res.setHeader(
        "Content-Disposition",
        'attachment; filename="' + invoiceName + '"'
      );
      //pdf generate will store in the server
      pdfDoc.pipe(fs.createWriteStream(invoicePath));
      //res is writable readstream
      pdfDoc.pipe(res);
      pdfDoc.text();
      pdfDoc.end();

Pagination

 

Payment

 stripe

 

 

Async resquests

send async requests to the backend and handle those requests at the backend.

eg. delete product.

1. handle the request and redirect to the page
2. Or send and handling background requests and manipulate the DOM.

JSON stands for JavaScript Object Notation and a typically JSON data structure looks like this:

    {
        "name": "Your Name",
        "age": 29,
        "courses": [
            "angular-the-complete-guide",
            "react-the-complete-guide"
        ],
        "profile": {
            "joined": "2017-05-21",
            "courses": 2
        },
        "averageRating": 4.8,
        "active": true
    }

It looks a lot like a normal JavaScript object, but one important difference is that all key names are enclosed by double quotation marks (").

Besides that, you can store text (string), numeric (integers and floats) and boolean data as well as nested objects and arrays.

 

async/await

 asynchronous requests in a synchronous way (only by the way it looks, not by the way it behaves)

exports.getPosts = async (req, res, next) => {
  const currentPage = req.query.page || 1;
  const perPage = 2;
  let totalItems;

  try {
      const totalItems = await Post.find().countDocuments()
      const posts = await Post.find().skip((currentPage - 1) * perPage).limit(perPage);

     res.status(200).json({
          message: "fetched posts succeed",
          posts: posts,
          totalItems: totalItems
        });
 }catch (err) {
    if (!err.statusCode) {
        err.statusCode = 500;
      }
      // go to the next error handling middleware
      next(err);
    });
};

Websockets

real-time communication

for example in a chat app, userA send a new message but userB doesn't send a request hence doesn't know what is updated in the server. hence need a way to push to data from server to the client for any changes.

socket.io

use a different protocol -> web sockets

can also send data from the client to the server

npm install --save socket.io

share io instance across files in socket.js

let io;

module.exports = {
    init: httpServer => {
        io = require('socket.io')(httpServer);
        return io;
    },
    getIO: () => {
        if(!io){
            throw new Error('Socket.io not initialized!');
        }
        return io;
    }
};

push information from the server to the client using emit method provided by socket.io

share io instance across files in socket.js

    const server = app.listen(8080);
    // use http server to establish the websocket connection
    const io =require('./socket').init(server);
    // wait for new connections
    io.on('connection', socket => {
      console.log('Client connected');
    });

in the controller file

const io = require('../socket');

send message to all clients 

// name, data
io.getIO().emit('posts', {action: 'create', post: post});

2. establish a connection from the client

npm install --save socket.io-client

in front-end code

import openSocket from 'socket.io-client';

const socket = openSocket('http://localhost:8080');
socket.on('posts', data =>{

});

 

 

Deployment

 

 

Environment variables

process.env.

Secure response

helmet

compression assets

 

SSL server

 

Testing

 

 use mocha and chai, sinon

npm install --save mocha chai sinon

add folder 'test'

npm test

write unit test

like testing the middleware

use stub

testing database