Section 16: MySQL in Node

Section Goals

By the end of this section, you should know how to:

  • Create a SQL file for setting up a table.
  • Query databases from a Node API.
  • Insert values into a Table with SQL

SQL Files

On Tuesday, we imported SQL files into phpMyAdmin to add data to the database. Today we will write one of those files.

We will be using the cafe database from Tuesday, so create it if you haven't, and import the menu items with menu-data.csv

Common SQL Types

Type Argument Description
INT none An integer number
VARCHAR(n) The string can be at most n characters (max of 65,535). a text string
DATE none Stores dates: YYYY-MM-DD
DATETIME none Stores dates and times for precise time information: YYYY-MM-DD HH:MM:SS
DECIMAL(d,s) Has at most d digits, and s digits after the decimal place A decimal number. Rounds based on the provided precision.
TEXT none A potentially very large text string.

Prefer VARCHAR to TEXT, and with as small a max limit as you are comfortable setting. This limits the amount of data someone can dump into your table.

CREATE TABLE

We make tables with the CREATE TABLE keyword.

CREATE TABLE highscores(
  id INT,
  username VARCHAR(255),
  score INT
);

Basic SQL Example

First we provide the table name, in this case "highscores". We then give a comma-separated list of column names, with the type of each column coming after the name.

NOT NULL

Use the NOT NULL keyword to prevent null entries in a column.

CREATE TABLE highscores(
  id INT NOT NULL,
  username VARCHAR(255) NOT NULL,
  score INT NOT NULL
);

Basic SQL Example

A query that attempts to enter NULL into one of these columns will cause the query to fail.

When inserting data, you can pass NULL by not including that column in your query.

PRIMARY KEY, AUTO_INCREMENT

Every table should have a column which is used to uniquely identify each row. This improves efficiency and will prove very useful next week.

CREATE TABLE highscores(
  id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
  username VARCHAR(255) NOT NULL,
  score INT NOT NULL
);

Basic SQL Example

Adding PRIMARY KEY makes it so that the code will error if that column ever has duplicates. It will use that column to identify each row quickly. This is usually an integer id.

AUTO_INCREMENT will make it so that, if you provide no input for that column, it will pick the next unused value. Perfect for making it so you don't have to worry about what the next id is.

DROP TABLE IF EXISTS

We cannot create a table if it already exists. Usually, we want our SQL files to be able to overwrite old tables if we re-import them.

DROP TABLE IF EXISTS highscores;
CREATE TABLE highscores(
  id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
  username VARCHAR(255) NOT NULL,
  score INT NOT NULL
);

Basic SQL Example

SQL files will often look like this, perhaps with some data inserted into the table afterwards. We use DROP TABLE to delete the old table. However, you cannot delete a non-existent table, so we use IF EXISTS to check.

Exercise 1: Creating an Orders Table

We want to add the following orders table to the cafe database. The entry below is for reference, but we won't be adding it.

id phone_number email item_name qty total_price order_time
1 NULL student@uw.edu Tea Loaf 2 8.00 2019-08-13 03:03:00

On the slide below are specifications for each column.

The table should be named orders and should replace existing tables.

column name Requirements
id The primary key for the table. Should increment automatically.
phone_number Someone's phone number, as a string. Might be NULL.
email Someone's email. Might be NULL.
item_name The name of an item from the menu table. Required.
qty The number of an item ordered. Required.
total_price The price of an order, as a decimal number. Required.
order_time The date and time of the order. Required.

Code Quality note: SQL files, like any other, should have header comments. Use -- at the start of a comment line.

Once you finish the .sql file, import the table into your cafe database.

A solution

DROP TABLE IF EXISTS orders;
CREATE TABLE orders(
  id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
  phone_number VARCHAR(20),
  email VARCHAR(63),
  item_name VARCHAR(63) NOT NULL,
  qty INT DEFAULT 0,
  total_price DECIMAL(6,2) NOT NULL,
  order_time DATETIME DEFAULT NOW()
);

Getting Ready to Use SQL in Node

Like usual, we have some packages to import.

  • npm install promise-mysql
  • const mysql = require("promise-mysql");

Once you have mysql, add this function to your program, which we will use to create connections with the database hosted with MAMP.

async function getDB() {
  let database = await mysql.createConnection({
    host: "localhost",
    port: "8889", // You may need to change this port.
    user: "root",
    password: "root",
    database: "mydatabasename" // The name of the database you want to query.
  });
  return database;
}

Node.JS template

Using SQL in Node

When you want to use SQL, get a connection to the database, and then query it with a SQL query. If you make a SELECT query, it will return the result as an array of row data objects.

try {
  db = await getDB();
  let data = await db.query("SELECT * FROM table_name");
  return data;
} catch (err) {
  console.error(err);
}

Things can go wrong when establishing a connection and making the query, so be sure to use try/catch to handle errors.

Important: Ending the connection

async function makeQuery() {
  let db;
  try {
    db = await getDB();
    let data = await db.query("SELECT * FROM table_name");
    db.end();
    return data;
  } catch (err) {
    if (db) { // undefined if an error occurred in getDB()
      db.end();
    }
    throw err; // to bubble up an error to be caught in the function that called makeQuery().
  }
}

When you open a connection to a database, the database will be "left hanging" unless you explicitly end the connection. You must always end the connection, even if errors occur in your program. (In this example, it is important that we end the connection if it was started, but then an error happens during the query.)

Response Format

The query function will return an array of objects (called RowDataPackets) with the following structure:

let data = db.query("SELECT name, platform FROM Games;");

[
  {
    "name": "Pokemon Red/Blue",
    "platform: "GB"
  },
  ...
]

JSON response

let firstGame = data[0]["name"]; // "Pokemon Red/Blue"

You can read more about the exact kind of Object it is in the documentation, but you can just treat it as an array of JSON objects.

Conveniently, you can send the returned rows with res.json(rows)

INSERT

To insert a new record into a table, we use the INSERT INTO keyword:

INSERT INTO menu(name, category, subcategory, price, cost)
VALUES ('Cookie Cat', 'Frozen', 'Ice Cream Cookie', 1.50, 0.40),
       ('Sea Salt Ice Cream', 'Frozen', 'Ice Cream', 2.00, 0.75);

SQL

First provide the table name, then optionally the list of columns you want to set. (By default it sets all columns. Columns left out will be set to NULL, unless they have AUTO_INCREMENT set.) Then provide the values for each column. You can separate groups of values with commas to insert multiple rows at once.

Naturally, you can use this in node with await db.query("...");

Exercise 2: Cafe API

We are going to build an API for a theoretical cafe (note that this one has different data than the CSE154 cafe from lecture). There are three endpoints:

  • /menu: Returns all menu items, organized by category.
  • /menu/:category: Returns all menu items in a category.
  • /order: Submits an order to the server.

We will be using the cafe database we set up earlier, with the menu table and orders table.

Use Postman throughout the development process, and test often!

starter code

Implement a /menu GET endpoint. It should query the database for all of the items on the menu, returning them organized by category in the following format:

{
  "Bakery": [
    {
      "name": "Blueberry Scone",
      "subcategory": "Scones",
      "price": 3.50
    },
    ...
  ],
  ...
}

JSON response

Within each category, the results should be sorted alphabetically by name. Utilize the ORDER BY keyword to make this easier. Remember to db.end() any connections you make, and handle errors in async/await code.

An Note about Handling Parameter Values from Clients

For today, we're just focusing on using SQL with Node.js/Express

You can build a SQL query that uses GET/POST variables with string concatenation

However, in practice this leads to security vulnerabilities for malicious user input

We will learn the better approach with variable placeholders tomorrow!

Implement a /menu/:category GET endpoint. Given a category, it will respond with an array of items in that category, again sorted alphabetically by name:

[
  {
    "name": "Blueberry Scone",
    "subcategory": "Scones",
    "price": 3.50
  },
  ...
]

If there are no items for that category (the query result is empty) respond with a descriptive 400 status error.

Note that the format of the response is identical to the format returned by db.query("...");, so you can just send it back as JSON unaltered.

Implement a /order POST endpoint that inserts an order into the database. It has the following required body parameters:

  • phone_number: A string representing someone's phone number (e.g. "206-154-2019").
  • email: A string representing someone's email (e.g. "dubs@uw.edu").
  • item_name: The name of an item on the menu. e.g. "Blueberry Scone"
  • qty: The quantity of the item being bought. e.g. "2"
  • tip: The tip for the order. e.g. "3.50" for $3.50

If any of these parameters are missing, send a descriptive 400 text error message.

If there is no item in the menu corresponding to the passed in item_name, you should give a different descriptive 400 error message.

If all is well, respond with the plain text, "Your order has been processed!"

More details on expected behavior are provided on the next slide.

When adding an order, add a record to the orders table as follows:

  • phone_number: The same format as the passed phone number.
  • email: The same format as the passed email
  • item_name: The same format as the passed item name (make sure it exists in the menu table)
  • qty: The same format as the passed quantity
  • total_price: The price of the item (from the menu table) multiplied by the quantity, plus the tip. Remember to parse the POST parameters to calculate, as they are strings.
  • Note about order_time: Because our CREATE TABLE used the constraint order_time DATETIME DEFAULT NOW(), we don't need to insert this value explicitly - SQL will calculate this for us using the DATETIME when the record is inserted.
  • When solving this problem, you should only make two queries, and should only call getDB once. It is important not to make redundant queries or connections. Use the query for the item price to check if the item exists.

Let's make the POST endpoint more flexible. Perhaps we want to allow users to provide either the email, the phone number, or both. Change the endpoint so that the user only has to provide at least one of the two parameters. You will need to alter the INSERT query to pass NULL instead of the quoted email/phone_number if it is not provided.

SOLUTION