Lab net: Network stack

This is one of the project options.

In this lab you will add networking support to xv6. We will be using lwIP, an open-source TCP/IP stack, so you won’t have to write a network stack from scratch. Your job is to write a driver for a network interface controller (NIC) and add sockets to xv6.

Your solution is acceptable if DHCP succeeds with your NIC driver. Implementing sockets is encouraged but not required for this lab.

Driver

QEMU emulates a virtio-net NIC. The first step is to write a driver for this device. You may want to read both the virtio specification and the source code of the virtio disk driver in xv6 (kernel/virtio_disk.c). Make sure you understand the initialization process and how the driver and the device communicate through virtqueues.

Here are some hints about how you might go about this part.

We provide necessary functions to work with lwIP in kernel/net.c. To compile the source code of lwIP and related files, uncomment the line below “uncomment for lab net” in Makefile. Call netinit() from main() (kernel/main.c). Create a new file kernel/virtio_net.c for your NIC driver code, which should implement the following three functions.

/* initialize the NIC and store the MAC address */
void virtio_net_init(void *mac);

/* send data; return 0 on success */
int virtio_net_send(const void *data, int len);

/* receive data; return the number of bytes received */
int virtio_net_recv(void *data, int len);

To communicate with a device through memory-mapped I/O (MMIO) , you need to add its physical address to kernel/memlayout.h for your driver and map the MMIO page in the kernel page table in kvminit() (kernel/vm.c). You may use Ctrl-a c to switch to the QEMU monitor and type info qtree to find out the address of each device. For example, the virtio disk ( virtio-blk-device) resides at 0x10001000, whereas the virtio NIC (virtio-net-device) should be at 0x10002000.
Implement virtio_net_init() to initialize the device and store the MAC address. Your driver should negotiate the VIRTIO_NET_F_MAC feature bit; it doesn’t need to negotiate any other feature bit for simplicity (see Section 5.1.3 “Feature bits” of the virtio specification). You should be able to find the MAC address in the device configuration space. Once this part is done, you should see the following in the output:

net: mac 52:54:00:12:34:56

Finish virtio_net_send() and virtio_net_recv(). If everything works correctly, DHCP should succeed and you should see the following in the output:

net: addr 10.0.2.15 netmask 255.255.255.0 gw 10.0.2.2

For simplicity, your driver may use polling and disable interrupts for the NIC. Feel free to add interrupt support for the NIC if you are interested. As an example, check how xv6 enables interrupts from the disk (kernel/trap.c and kernel/plic.c).

For debugging, you may turn on verbose output in kernel/lwip/lwipopts.h. For instance, lwIP will print details of the DHCP process if you set DHCP_DEBUG to LWIP_DBG_ON there. We also configure QEMU to dump all incoming and outgoing packets to en0.pcap in your lab directory. You may use tcpdump or Wireshark to inspect the file.

$ tcpdump -nr en0.pcap
reading from file en0.pcap, link-type EN10MB (Ethernet)
19:21:49.038877 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 52:54:00:12:34:56, length 308
19:21:49.038919 IP 10.0.2.2.67 > 255.255.255.255.68: BOOTP/DHCP, Reply, length 548
19:21:49.040188 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 52:54:00:12:34:56, length 308
19:21:49.040202 IP 10.0.2.2.67 > 255.255.255.255.68: BOOTP/DHCP, Reply, length 548
19:21:49.040654 ARP, Probe 10.0.2.15, length 28
19:21:49.536180 ARP, Probe 10.0.2.15, length 28
19:21:50.036122 ARP, Probe 10.0.2.15, length 28
19:21:50.036343 ARP, Announcement 10.0.2.15, length 28

Use tcpdump -XXnr en0.pcap to further print the data of each packet if needed.

Wireshark provides both graphical and terminal-based user interfaces. Here is an example of using the terminal version:

$ tshark -nr en0.pcap
    1   0.000000      0.0.0.0 → 255.255.255.255 DHCP 350 DHCP Discover - Transaction ID 0x5d39a
    2   0.000042     10.0.2.2 → 255.255.255.255 DHCP 590 DHCP Offer    - Transaction ID 0x5d39a
    3   0.001311      0.0.0.0 → 255.255.255.255 DHCP 350 DHCP Request  - Transaction ID 0x5d39a
    4   0.001325     10.0.2.2 → 255.255.255.255 DHCP 590 DHCP ACK      - Transaction ID 0x5d39a
    5   0.001777 52:54:00:12:34:56 → ff:ff:ff:ff:ff:ff ARP 42 Who has 10.0.2.15? Tell 0.0.0.0
    6   0.497303 52:54:00:12:34:56 → ff:ff:ff:ff:ff:ff ARP 42 Who has 10.0.2.15? Tell 0.0.0.0
    7   0.997245 52:54:00:12:34:56 → ff:ff:ff:ff:ff:ff ARP 42 Who has 10.0.2.15? Tell 0.0.0.0
    8   0.997466 52:54:00:12:34:56 → ff:ff:ff:ff:ff:ff ARP 42 Gratuitous ARP for 10.0.2.15 (Request)

Sockets

The next step is to implement socket system calls on top of lwIP’s “raw” API. You may design your own socket interface, though the BSD socket API is a good starting point. The socket interface should support at least TCP (with IPv4); it doesn’t have to support UDP or IPv6. You may find the documentation of lwIP’s TCP API useful.

Here are some hints about how you might go about this part.

Add a new file type (FD_SOCK) to struct file and modify existing system calls read, write, and close to dispatch to socket operations.
Add two new system calls socket and connect for creating sockets and initiating connections, respectively.
Implement socket operations on top of lwIP’s TCP API. Be careful about when socket operations should block. You also need to somehow feed incoming packets to lwIP and check timeouts (e.g., using a dedicated process or the timer interrupt). By this point your xv6 should be able to run applications using client-side sockets.
Add DNS resolution support to translate hostnames to IP addresses. You may either add a new system call to perform DNS resolution using lwIP in the kernel, or implement DNS resolution in user space. QEMU provides a DNS server at 10.0.2.3. You may also use public DNS servers (e.g., Google’s 8.8.8.8).
Add system calls bind, listen, and accept for server-side sockets. To test, implement a simple web server listening at port 80. Check QEMU’s documentation for the “hostfwd” option. For instance, hostfwd=tcp::25502-:80 means that one can connect to the local port 25502, which is forwarded to guest port 80.
Note that lwIP itself is configured to run in a single-threaded environment. A safe approach is to always grab a lock before invoking any lwIP function.

Implement some user programs to test your interface. Feel free to borrow code from Hyperkernel, such as:

a daytime client (daytime.c),
a weather client (wttr.c),
a web server (httpd.c), and
a VNC server (vncd.c).

Below is an example output:

xv6 kernel is booting

net: mac 52:54:00:12:34:56
net: addr 10.0.2.15 netmask 255.255.255.0 gw 10.0.2.2
hart 1 starting
hart 2 starting
init: starting sh
$ daytime

59483 21-09-26 07:58:36 50 0 0 223.0 UTC(NIST) * 
$ wttr
HTTP/1.1 200 OK
Server: nginx/1.10.3
Date: Sun, 26 Sep 2021 07:58:39 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 350
Connection: close
Access-Control-Allow-Origin: *

Weather report: Mountlake Terrace, Washington, United States

      \   /     Clear
       .-.      55 °F          
    ― (   ) ―   ↙ 5 mph        
       `-’      9 mi           
      /   \     0.0 in         

$

Optional challenges:

Implement a web-based system monitor (e.g., the JOS dashboard).
Port or implement TLS.
Replace hardcoded physical addresses for virtio drivers (e.g., through device tree or PCI).
Implement XDP.