CSE 154

Lecture 17: More Node.js with Express and File I/O

Summary: Starting a Node.js Project

There are a few steps to starting a Node.js application, but luckily most projects will follow the same structure.

When using Node.js, you will mostly be using the command line (e.g. php-ide-terminal you should have installed in Atom).

1. Start a new project directory (e.g. node-practice)
2. Inside the directory, run npm init to initialize a package.json configuration file (you can keep pressing Enter to use defaults)
3. Install any modules with npm install <package-name>
4. Write your Node.js file! (e.g. app.js)
5. Include any front-end files in a public directory within the project.

Along the way, a tool called npm will help install and manage packages that are useful in your Node app.

Terminology and Concepts So Far

Node.js

npm

Packages and Modules

API (Application Programming Interface)

Express.js

Route

Route Parameters: Path vs. Query

Package Management

We use npm (Node Package Manager) to install and manage packages (many of which are modules)

A package is any project with a package.json file

• To create your own package, use npm init
• To install module dependencies in an existing project containing a package.json, use npm install
• Each time you run npm install <module-name>, npm will automatically add the module as a dependency to the current package.json and add the module to node_modules

When sharing your project, you provide package.json but not node_modules

Application Program Interfaces (APIs)

An application programming interface (API) is a communication protocols that allows two pieces of software to communicate.

A Web API (or web service) is a set of pre-defined URLs with parameters that allow a user to get information from a web server. Can be used by many types of clients.

Source: https://happycoding.io/tutorials/java-server/rest-api

Node.js + Express: How we write our Web APIs

Node.js is the runtime environment to execute JavaScript on the server

We have access to HTTP functionality, the file system, database interaction, etc. that we wouldn't have in the browser (the client)

Express.js is one of the most popular Node frameworks used to implement APIs

Serves as an easy-to-use wrapper around Node's more complex core networking modules.

Provides simple functionality through "middleware" to listen and respond to HTTP requests from clients at different endpoints.

Creating an Express App: The Basic Template

"use strict";
// 1. Load required modules
const express = require("express");
const app = express();

// 2. Add routes and other middleware and functions here

// 3. Start the app on an open port!
const PORT = process.env.PORT || 8000;
app.listen(PORT);

JS

Remember to use nodemon app.js to run your app on localhost:8000 and "mon"-itor any changes in your file so you don't have to restart the server each time.

Basic Routing in Express

Routes are used to define endpoints in your web service

Express supports different HTTP requests - we will use GET and POST, but there are also PUT and DELETE

Express will try to match routes in the order they are defined in your code

Useful Request Properties/Methods

Request Parameters: Path Parameters

Act as wildcards in routes, letting a user pass in "variables" to an endpoint

Define a route parameter with :param

Route path: /states/:state/cities/:city
Request URL: http://localhost:8000/states/wa/cities/Seattle
req.params: { "state": "wa", "city": "Seattle" }

These are attached to the request object and can be accessed with req.params

app.get("/states/:state/cities/:city", function (req, res) {
  let state = req.params.state; // wa
  let city = req.params.city;   // Seattle
  // do something with variables in the response
});

JS

Request Parameters: Query Parameters

You can also use query parameters in Express using the req.query object, though they are more useful for optional parameters.

Route path: /cityInfo
Request URL: http://localhost:8000/cityInfo?state=wa&city=Seattle
req.query: { "state": "wa", "city": "Seattle" }

app.get("/cityInfo", function (req, res) {
  let state = req.query.state; // wa
  let city = req.query.city;   // Seattle
  // do something with variables in the response
});

JS

Unlike path parameters, these are not included in the path string (which are matched using Express routes) and we can't be certain that the accessed query key exists.

If the route requires the parameter but is missing, you should send an error to the client in the response.

Setting Errors

The Response object has a status function which takes a status code as an argument.

The 400 status code is what we'll use to send back an error indicating to the client that they made an invalid request.

A helpful message should always be sent with the error.

app.get("/cityInfo", function (req, res) {
  let state = req.query.state;
  let city = req.query.city;
  res.type("text");
  if (!(state && city)) {
    res.status(400).send("Error: Missing required city and state query parameters.");
  } else {
    res.send("You sent a request for " + city + ", " + state);
  }
});

JS

Useful Response Properties/Methods

Setting the Content Type

By default, the content type of a response is HTML - we will only be sending plain text or JSON responses though in our web services

To change the content type, you can use the res.set function, which is used to set response header information (e.g. content type).

You can alternatively use res.type("text") and res.type("json") which are equivalent to setting text/plain and application/json Content-Type headers, respectively.

app.get("/hello", function (req, res) {
  // res.set("Content-Type", "text/plain");
  res.type("text"); // same as above
  res.send('Hello World!');
});

Setting plain text response

app.get("/hello", function (req, res) {
  // res.type("json");
  // res.send({ "msg" : "Hello world!" });
  res.json({ "msg" : "Hello world!" }); // same as above
});

Setting JSON response content type

Middleware

Middleware is a term you'll see when working with Express - it refers to any function that works "in the middle" of communication between the client and Node.js.

For example, we will use the express.static middleware function to specify the default directory of "static" client-facing files (HTML, CSS, client-side JS, images, etc.)

Every middleware function has access to the request and response objects, and usually modifies the response to send.

You can add multiple middleware functions used in a single request-response cycle with app.use, but the last (usually app.get) must close the connection (e.g. with res.send) to prevent the connection to be left hanging.

This is a great resource covering middleware more in Express

Example of Middleware

const express = require("express");
const app = express();

// defining middleware
// one middleware
function logger(req, res, next) {
  let requestTime = new Date();
  console.log("You sent a request at " + requestTime + "!");
  next(); // continue
}

// a second middleware
function hello(req, res, next) {
  res.write("Hello \n"); // use write when you want to output, but don't want to end
  next();
}

function bye(req, res, next) {
  res.write("Bye \n");
  res.end();
}

// using middleware
app.use(logger);
app.get("/hello", hello, bye);

const PORT = process.env.PORT || 8000;
app.listen(PORT);

JS

Code: middleware-example.js

Express as a Routing Mechanism for Node.js

Image source

Summary of Building an Express App

1. Create a file (e.g. app.js)
2. Add required modules at the top (at minimum, require("express"))
3. Create an app instance: const app = express();
4. At the end of the file, listen to a port (e.g. 8000)
5. Add routes! Remember that "/" stands for the basic root route, which can be visited in your browser at localhost:8000/ when your app is running.

What if we want to fetch from our app from our HTML/CSS/JS files?

Serving Static Files

If a project is "full-stack" and contains both client-side "static" files along with your Node.js files, you can use the express.static middleware to specify the directory serving static files

The convention is to put your static files in a public directory, in the same location as your Node.js file

"use strict";
const express = require("express");
const app = express();

app.use(express.static("public"));
// now "/" points to "public/" so we can visit "localhost:8000/greeter.html"

const PORT = process.env.PORT || 8000;
app.listen(PORT);

JS

Demo'd "full-stack" example with our greeter API: greeter.zip

A Simple CSE154 TA API

Let's write an API that takes GET requests to get information about CSE154 TAs!

tas/: Send back JSON response with array of TA names

tas/:section: Send back plain text response with TA name for given section code (e.g. "AA" for "Tal")

sections/: Send back JSON response with array of section codes

sections/:ta: Send back plain text response with section code for given TA (e.g. "Tal" for "AA")

Solution: app.js

Motivating File I/O

The TA API had a constant SECTIONS to map sections to TA names

const SECTIONS = {
  "AA" : "Tal",
  "AB" : "Hudson",
  "AC" : "Sven",
  "AD" : "Manny-Theresa"
};

JS

What if we wanted to have multiple SECTION objects for different quarter offerings of CSE 154?

What if we wanted to use the same data across different JS programs?

Often, it's best to factor out data we use in web services using files or databases. We'll start with files.

File I/O in Node.js

Unlike the browser, we have access to the file system when running Node.js

We can read all kinds of files, as well as write new files

Next week, we will learn how to process directories as well

The fs Core Module

We saw http as our first core module in Node.js to create a basic HTTP server - this is what the npm module Express simplifies

Another useful Core Module we get in Node is fs (file system)

There are many functions in the fs module (with excellent documentation)

Most functions rely on error-first callbacks

Reading Files

fs.readFile(fileName, encodingType, callback)

• fileName (string) file name
• encodingType file encoding (usually "utf8")
• callback "error-first" function that takes two arguments: an Error object (undefined if no error) and the result file contents (e.g. string text)

fs.readFile("example.txt", "utf8", (err, result) => {
  if (err) {
    console.error("Something went wrong when reading the file...");
  } else {
    let lines = result.split("\n");
    console.log("First line contents: " + lines[0]);
  }
});

JS

Example code: file-reading.js

Reading and Parsing JSON

You can read any file, including JSON. To parse JSON file contents and use as a JS object, use JSON.parse (works similarly to .json() you've used in fetch).

// Example reading/parsing JSON
fs.readFile("package.json", "utf8", (err, result) => {
  if (err) {
    console.error(err);
  } else {
    let data = JSON.parse(result);
    console.log(data);
  }
});

JS

Writing Files

To write a file, use the fs.writeFile(fileName, data, callback) function.

This function's callback is also error-first, but only takes an error argument

fs.writeFile("new-file.txt", "Hello!", (err) => {
  if (err) {
    console.error("Something went wrong when writing the file...");
  } else {
    console.log("new-file.txt written to successfully!");
  }
});

JS

Example code: file-writing.js

Asynchronous Programming

One of the advantages of Node.js is its efficiency to handle many requests

This is possible through an "event loop" - the event loop is also what your browser uses to handle asynchronous functions like setTimeout

However, this can make it difficult to write code with asynchronous functions that are dependent on another asynchronous function

We'll learn some techniques to handle this easily on Monday!