Lecture: Virtual machines

You may refer to VMware’s Virtualization Overview if needed.

Question

Since a Dune process is running in ring 0 and has full access to privileged instructions (e.g., modifying %cr3), can it corrupt or take control of the underlying Linux kernel? Briefly explain why or why not.

Question

Dune provides speedup for handling traps and garbage collection (Section 6.2/Table 3). Briefly explain why.

Dune source code

If you are interested, check out the source code of Dune on GitHub.

administrivia

why reading these papers? how to read papers?
what’s systems research like? what’s SOSP/OSDI?

OS virtualization recap

examples
- virtual memory
- virtual CPU/time
- virtual file system: file descriptor & name space
a key scheme: naming
- (pid, va) → pa
- (ino, offset) → disk address
- other naming examples: DNS, Linux namespaces
why virtual machines: run multiple OSes
- 60-70s: better use of expensive machines
- 80s: mostly OSes
- 90s: desktop virtualization
- cloud, business model driven (another trend: containers/datacenter OSes), security monitors

overview

terminology
- host, guest
- host can be baremetal or an OS (type 1/2)
- VMM (virtual machine monitor), hypervisor
- tagged TLB: PCID, VPID
- Intel VMX (VT-x), AMD SVM (AMD-V)
goals
- fidelity: (almost) identical to execution on hw
- performance: low overhead
- safety: isolation, mutiplexing
strawman plans
- directly run guest kernels on hw: safety?
- emulation (e.g., QEMU): performance?
- run guest kernels in user mode: what can go wrong?
classically virtualizable ISA
- trap-and-emulate: recall changing an invalid memory read to return 42
- sensitive instructions ⊆ privileged instructions (Popek-Goldberg theorem)
- RISC-V: virtualizable
  - imagine running the xv6 kernel in U-mode
  - executing instructions that modify CSRs will trap into S-mode (where we can emulate)
- x86 (before VMX/SVM)
  - not virtualizable: 17 sensitive, unprivileged instructions
  - example: pushf/popf/iret behave differently in user mode

x86 virtualization

virtualize x86 without VMX/SVM
- idea: modify the guest code to avoid the 17 instructions
- binary translation (e.g., VMware)
  - goal: replace the 17 instructions with trapping ones
  - challenge: how to do this efficiently
- paravirtualization (e.g., Xen): replace the 17 instructions in guest kernels with hypercalls
VMX/SVM extensions: make x86 virtualizable
- how to “fix” the ISA
  - add root/non-goot mode for existing rings - see Figure 23-1. Interaction of a Virtual-Machine Monitor and Guests of Intel’s manual
  - key data structure
    - VMCS/VMCB (control & state) - see chapter 24 of Intel’s manual
      - guest state
      - host state
      - control fields: VM-exectuion, VM-exit, VM-entry
      - VM-exit information
    - VMCS requires specific instructions (e.g., vmread/vmwrite) to access
- CPU: either trap or duplicate state
  - another way to look at this: make fewer sensitive instructions in non-root mode
  - or make more privileged instructions (that trap in root mode)
- memory: EPT/NPT
  - shadow page tables
    - simulate %cr3 in software
    - write-protect guest’s page table pages
    - downside: performance
  - extended page tables (EPT) / nested page tables (NPT): guest pa → host pa
    - figure 2 of the dune paper is simplified
    - see two-dimensional page walk
    - page faults vs EPT faults: both can go to either guest or host depending on configuration
- I/O devices
  - example: two guests, one NIC on the host
  - who emulates two NICs?
  - when a packet arrives at the host NIC, which guest to notify?
  - arguably the most interesting part in hypervisor
- transitions: KVM workflow
  - VM entry: host to guest
  - VM exit: guest to host
  - hypercalls: vmcall/vmmcall cause VM exits
  - overhead: 4000 (older)–500 (newer) cycles (as a comparison, syscalls: 60-70 cycles)
  - guess why the overhead and how Intel/AMD might improve
- nested virtualization
  - running a hypervisor in guest mode - why is this needed?
  - what’s the performance implication - see recursive virtual machines, Figure 3
- take a look at the lvisor exercises from CSE 481A if you are interested

Dune

why reading this paper
- IX (OSDI’14), Shinjuku (NSDI’19)
- see Google’s gvisor
- innovation paradigm
  - writing a new OS is hard (e.g., features, adoption)
  - hacking Linux is hard (e.g., upstream, complexity)
  - Dune provides a nice design point
- source code: kern, libdune
Dune’s goal
- talk
- process abstraction for privileged instructions
  - sharing
  - performance
- Dune process mode
  - ring 0, non-root
  - use hypercalls as syscalls
  - performance pros: extra bits on the page table (e.g., hw dirty bits), fast exception-handling path
  - performance cons: hypercalls, VM exits, EPT overhead
  - what kinds of applications beneift from this?
applications
- exception handling
  - traditional flow, Figure 1 (recall lab alarm)
  - CPU delivers page faults to a Dune process directly
  - no context switches: handle page faults in the process
- sandbox
  - syscall filtering
  - compare to chroot/seccomp
- garbage collection
  - use dirty bits to track if memory has been touched
  - better page table management & TLB control