Lecture: Virtual machines

A Comparison of Software and Hardware Techniques for x86 Virtualization, ASPLOS 2006
Dune: Safe User-level Access to Privileged CPU Features, OSDI 2012
Optional: gVisor’s Architecture Guide
Optional: IX: A Protected Dataplane Operating System for High Throughput and Low Latency, OSDI 2014

You may refer to VMware’s Virtualization Overview if needed.

Question

Since a Dune process is running in ring 0 and has full access to privileged instructions (e.g., modifying %cr3), can it corrupt or take control of the underlying Linux kernel? Briefly explain why or why not.

Question

Dune provides speedup for handling traps and garbage collection (Section 6.2/Table 3). Briefly explain why.

Question

Briefly describe how you would port Dune to xv6. Specifically, what kind of architectural support would be needed on RISC-V?

Question

Provide a list of questions you would like to discuss in class. Feel free to provide any comments on the papers and related topics (e.g., which parts you like and which parts you find confusing).

Dune source code

If you are interested, check out the source code of Dune on GitHub.

Write down your answers in a file named answers.txt, and upload it using Canvas.

administrivia

why reading these papers?
what’s systems research like?
what’s SOSP/OSDI?

OS virtualization recap

examples
- virtual memory
- virtual CPU/time
- virtual file system: file descriptor & name space
a key scheme: naming
- (pid, va) → pa
- (ino, offset) → disk address
- other naming examples: DNS, Linux namespace (Docker etc.)
why virtual machines: run multiple OSes
- 60-70s: better use of expensive machines
- 80s: mostly OSes
- 90s: desktop virtualization
- cloud, business model driven (another trend: containers/datacenter OSes), security monitors

overview

terminology
- host, guest
- host can be baremetal or an OS (type 1/2)
- VMM (virtual machine monitor), hypervisor
- tagged TLB: PCID, VPID
- Intel VMX (VT-x), AMD SVM (AMD-V)
goals
- fidelity: (almost) identical to execution on hw
- performance: low overhead
- safety: isolation, mutiplexing
strawman plans
- directly run guest kernels on hw: safety?
- emulation (e.g., QEMU): performance?
- run guest kernels in user mode: what can go wrong?
classically virtualizable ISA
- trap-and-emulate: recall changing an invalid memory read to return 42
- sensitive instructions ⊆ privileged instructions (Popek-Goldberg theorem)
- RISC-V: virtualizable
  - imagine running the xv6 kernel in U-mode
  - executing any instructions that modify CSRs will trap into S-mode (where we can emulate)
- x86 (before VMX/SVM)
  - not virtualizable: 17 sensitive, unprivileged instructions
  - example: pushf/popf/iret behave differently in user mode

x86 virtualization

virtualize x86 without VMX/SVM
- idea: modify the guest code to avoid the 17 instructions
- binary translation (e.g., VMware)
  - goal: replace the 17 instructions with trapping ones
  - challenge: how to do this efficiently
- paravirtualization (e.g., Xen)
  - the term was from the Denali isolation kernel
  - replace the 17 instructions in guest kernels with hypercalls
VMX/SVM extensions: make x86 virtualizable
- how to “fix” the ISA
  - add root/non-goot mode for existing rings
  - key data structure
    - VMCS/VMCB (control & state) - see the Intel’s manual, chapter 24
      - guest state
      - host state
      - control fields: VM-exectuion, VM-exit, VM-entry
      - VM-exit information
    - VMCS requires specific instructions (e.g., vmread/vmwrite) to access
- CPU: either trap or duplicate state
  - another way to look at this: make fewer sensitive instructions in non-root mode
  - or make more privileged instructions (that trap in root mode)
- memory: EPT/NPT
  - shadow page tables
    - simulate %cr3 in software
    - write-protect guest’s page table pages
    - downside: performance
  - extended page tables (EPT) / nested page tables (NPT): guest pa → host pa
    - figure 2 of the dune paper is simplified
    - see two-dimensional page walk
    - page faults vs EPT faults: both can go to either guest or host depending on configuration
- I/O devices
  - example: two guests, one NIC on the host
  - who emulates two NICs?
  - when a packet arrives at the host NIC, which guest to notify?
- transitions: draw KVM workflow (KVM, QEMU on host, guest OS)
  - VM entry: host to guest
  - VM exit: guest to host
  - hypercalls: vmcall/vmmcall cause VM exits
  - overhead: 4000 (older)–500(newer) cycles (syscalls: 60-70 cycles)
  - guess why the overhead and how Intel/AMD might improve
- nested virtualization
  - running a hypervisor in guest mode - why is this needed?
  - what’s the performance implication - see recursive virtual machines, Figure 3
- take a look at the lvisor exercises from CSE 481A if you are interested

Dune

why reading this paper
- IX (OSDI’14), Shinjuku (NSDI’19)
- see Google’s gvisor
- research paradigm
  - writing a new OS is hard (e.g., features, adoption)
  - hacking Linux is hard (e.g., upstream, complexity)
  - Dune provides a nice design point
- source code: kern, libdune
Dune’s goal
- talk
- process absraction for privileged instructions
  - sharing
  - performance
- Dune process mode
  - ring 0, non-root
  - use hypercalls as syscalls
  - performance pros: extra bits on the page table (e.g., hw dirty bits), fast exception-handling path
  - performance cons: hypercalls, VM exits, EPT overhead
  - what kinds of applications beneift from this?
applications
- exception handling
  - traditional flow, Figure 1
  - CPU delivers page faults to a Dune process directly
  - no context switches: handle page faults in the process
- sandbox
  - syscall filtering
  - compare to chroot/seccomp
- garbage collection
  - use dirty bits to track if memory has been touched
  - better page table management & TLB control