Introduction to Observable Framework

Observable Framework is a static site generator designed for creating interactive dashboards, data apps, and explorable explanations. It combines the simplicity of Markdown with the power of JavaScript, enhanced by reactive semantics, to make data visualization and communication easier and more engaging. In this class we will use Framework to organize your readings, exercises, and assignments within individual repositories.

Let’s take a look at some of the basic functionality Framework provides…

Preliminaries

You should have git, npm, and node.js (v20 or higher) installed and accessible via the command line. To develop locally, we recommend the VS Code editor.

Once your individual course repository has been assigned, you should use git clone ... to create a copy on your local machine. Upon first use, navigate to the root directory of the cloned repository in your terminal and run npm i to ensure all dependencies are loaded.

While in the repository’s root directory, run npm run dev to launch a local preview site with live updates as you modify documents and code. This will typically open your default browser to http://localhost:3000. To verify that Framework provides live updates, go to the source file, src/readings/framework/introduction.md and change a few words, save the change, then check out this page. You should see the changes reflected without needing to refresh.

To share and publish your results, use git:

• git status to see local file changes
• git add to stage changes for sharing
• git commit to commit staged changes to the local git history
• git push to push those changes to GitLab

Once changes are pushed to GitLab, your Framework site will be automatically built and published using GitLab Pages.

Markdown

Observable Framework leverages Markdown for its content structure, allowing you to write text, create headings, lists, and links just like in standard Markdown. Here are some example markdown commands paired with the rendered versions:

**Basic Markdown:**

Basic Markdown:

## This is a Heading

This is a Heading

- Item 1

• Item 1

[Link to UW](https://uw.edu/)

Link to UW

Front-Matter

You can include YAML front-matter at the beginning of your Markdown file to configure page-specific settings. This is enclosed by --- markers. For example:

---
title: Introduction to Observable Framework
---

Check out this page for more options supported by Observable Framework. As we will see later, the front matter is also helpful for setting up a SQL database (running in-browser) you want to query within a Framework page.

HTML

You can also write HTML in Observable Framework. For example, the code below

<details>
  <summary>Click me</summary>
  This text is not visible by default.
</details>

produces

Images can also be referenced using either Markdown ![alt text](./image-path) or HTML <img src="./image-path" alt="alt text" />. For example,

![My Image](../../favicon.ico)

or

<img src="../../favicon.ico" alt="My Image" />

Code Blocks

To go beyond a static page, you can embed executable JavaScript code in a code block. Observable Framework understands these blocks and evaluates the JavaScript within them. JavaScript fenced code blocks (```js) are typically used to display content such as charts and inputs. They can also be used to declare top-level variables, for example to load data or declare helper functions.

JavaScript blocks come in two forms: expression blocks and program blocks. An expression block contains a single statement and should look something like this (note the lack of semicolon):

1 + 2

An expression block should display the result automatically.

A program block looks like this (note the semicolon):

1 + 2;

A program block doesn’t display anything by default, but you can call the display function to display something. Think of it as similar to print() in some languages. For example,

display(1 + 2);

Code Block Options

You can add options to your code blocks to control their behavior. These options are placed after the ```js keyword, separated by spaces.

• echo: Enables the code itself to be displayed. Framework’s default is to hide the code.

  ```js echo
  display("This code will run and be displayed.")
  ```
  

  will produce

  display("This code will run and be displayed.")
  

• run=false: Prevents the code block from being executed automatically. Framework’s default is to execute all JavaScript code cells automatically in the order of top-level variable declaration (more on this below). This can be useful for providing code you do not want to run.

  ```js run=false
  // This code will not be executed automatically.
  display("Not executed.");
  ```
  

  will produce

  // This code will not be executed automatically.
  display("Not executed.");
  

Inline Expressions

JavaScript inline expressions ${…} interpolate values into Markdown. They are typically used to display numbers such as metrics, or to arrange visual elements such as charts into rich HTML layouts. For example,

The current time is .

The current time is ${new Date(now).toLocaleTimeString("en-US")}.

To give another example,

${html`<span style="color: hsl(${(now / 10) % 360}, 100%, 50%)">Rainbow text!</span>`}

Here, now is a built-in variable referencing the current time as a timestamp (milliseconds since the UNIX epoch). The html tag function provided by Observable takes the string inside the backticks, parses it as HTML, and returns a special kind of object that Observable Framework understands as a dynamically generated HTML element.

Note that inline expressions cannot declare top-level reactive variables. To declare a variable, use a code block instead.

Reactive JavaScript

If you are familiar with Jupyter Notebooks, you might wonder why the code above runs without you having to explicitly execute it.

Observable Framework uses a reactive programming model, which means that when the value of a variable changes, any code that depends on that variable is automatically re-evaluated. This is similar to how spreadsheets work, where changing a cell updates any formulas that reference that cell. In standard JavaScript, you need to manually update values. In Observable, changes propagate automatically. This brings many benefits, including easier interactivity because state is automatically kept in sync, but it can take some getting used to.

It is important to note that code blocks run in topological order determined by top-level variable references (a.k.a. dataflow), rather than in top-down document order. For example, we can reference x + y before the code block that defines x and y:

x + y

const x = 1;
const y = 1;

The view Function

• view(expression): A view is a user interface element that directly controls a value in the notebook. A view consists of two parts: the view (typically an interactive DOM element) and its output value (any JavaScript value). The view function takes the DOM element as input and returns the value with reative semantics.

The view function is often used to define an input. For example:

const clicks = view(Inputs.button("Click me"));

Click the button above, and observe the value of clicks (note that this is also an example of Observable’s reactivity!):

clicks

The above example uses Inputs.button, provided by Observable Inputs. You can also implement custom inputs using arbitrary HTML. For example, here is a range input that lets you choose an integer between 1 and 15 (inclusive):

const n = view(html`<input type=range step=1 min=1 max=15>`);

n is

Scope and IIFE

In Observable, each code block has a global scope by default. To avoid naming conflicts and keep your code organized, you can use Immediately Invoked Function Expressions (IIFEs) to create local scope within a cell. Variables declared within a function in this way are only accessible within that function.

Essentially, IIFEs give us a way to define a function and run it immediately. They are used to avoid “polluting” the global scope. You can also use private variables to accomplish a task within IIFEs.

The syntax for an IIFE involves defining an anonymous function and then immediately calling it: (function() { /* your code here */ })(); or (() => { /* your code here */ })();:

let statement;

(() => {
    const mySecret = "don't let the global scope know about this";
    statement = `Let me tell you a secret: "${mySecret}"`;
})()

display(statement);

// This would cause an error because mySecret is not defined in the global scope of this cell.
// display(mySecret);

In the example above, mySecret is only accessible within the anonymous function. Admittedly, the example above it a bit contrived. Instead, you will most often want to use IIFE to define a full code block that contains internal variables and multi-part computations, then returns a completed output value or element.

Immutability of Top-Level Variables

Variable assignments in Framework are a bit different from vanilla JavaScript. An assignment in Framework both declares the variable and assigns a value to it. You can change the value of it within the cell it is declared, provided you declared the variable with let or var. However, that value is immutable outside the cell it is assigned.

let a = 1;
// You can change a here
a = 2;
display(a);

// This code will throw an error since a is immutable when you are outside the cell it is declared
a = 3

Immutable variables might seem like a limitation, but they are necessary for Observable’s many useful features to work. And they encourage a more functional programming style of thinking that leads to cleaner and more reusable code. This includes methods like map(), filter(), etc., which operate on arrays, as well as the many useful utility functions built into D3 (which is available in Observable by default.)

In function scope, you can modify a variable’s value provided it is declared the variable with var or let.

(() => {
  let b = 1;
  b = 2;
  return b;
})()

Importing Packages and Files

Observable Framework allows you to extend its functionality by importing code from various sources.

Built-ins: Observable Standard Library

Observable Framework comes with a standard library that includes popular data visualization tools like D3.js. You can directly use these libraries without any explicit import statements.

Plot.lineY(aapl, {x: "Date", y: "Close"}).plot({y: {grid: true}})

The plot above uses Observable Plot, a built-in library, as well as aapl, a built-in dataset.

Importing Internal JavaScript Files

You can organize your code by creating separate JavaScript files within your project and importing functions or variables from them. Use relative paths for internal imports.

We have placed a file named ./utils.js with the following content:

export function greet(name) {
  return `Hello, ${name}!`;
}

We can import and use the greet function in Markdown as follows:

import { greet } from './utils.js';

display(greet("Observable User"));

Importing Node Modules

You can import from node_modules. This is useful for managing dependencies with a package manager such as npm or Yarn, for importing private packages from the npm registry, or for importing from a different package registry such as GitHub.

After installing a package (say with npm install or yarn add), you can import it like this (note that d3-array is in our node_modules):

import * as d3Array from "d3-array";

Importing External JavaScript Libraries

You can import packages from npm using the npm: prefix followed by the package name.

import confetti from "npm:canvas-confetti";

Inputs.button("Throw confetti! 🎉", {reduce: () => confetti()})

Referencing Files

Observable Framework handles files differently depending on how they are referenced.

• Files without explicit references are not included in the built site. This means if you have images or data files in your project that you don’t explicitly link to or use with FileAttachment, they won’t be part of the final published site.

• Data files can be referenced using the FileAttachment method. You can reference local files within a local module by using FileAttachment from the Observable standard library like so:

  const data = await FileAttachment("../../data/driving.csv").csv(); // note the use of relative path
  display(data.slice(0, 5)); // Display the first 5 rows
  

Advanced: Loading SQL Databases using DuckDB

Framework includes built-in support for client-side SQL powered by DuckDB. You can use SQL to query data from CSV, TSV, JSON, Apache Arrow, Apache Parquet, and DuckDB database files.

To use SQL, first register the desired tables in the page’s front matter using the sql option. Each key is a table name, and each value is the path to the corresponding data file. For example, to register a table named gaia from a local Parquet file (we don’t really have this file locally—this is just for the purpose of demonstration):

---
sql:
  gaia: ./lib/gaia-sample.parquet
---

To load externally-hosted data, use a full URL:

---
sql:
  quakes: https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.csv
---

To run SQL queries, create a SQL fenced code block (```sql). For example, to query the first 5 records from the quakes table:

SELECT * FROM quakes LIMIT 5;

Check out this tutorial for more advanced usage.

That’s it for now! Have fun in this class!

For more on Framework, please see the Observable Framework documentation.