Lecture: Testing and verification

• KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs
• A Differential Approach to Undefined Behavior Detection
• Optional: A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World

overview

• how to test your xv6 kernel
  • what’s considered correct (specification)?
  • complex input sources & state transitions
  • randomly generate sytem calls and interrupts?
• challenge: generate “useful” tests
  • what’s the chance of randomly generating (x, y) to trigger crash? 1/2⁶⁴
  • blackbox: infinite monkey theorem

void test_me(int x, int y) {
  int z = 2 * x;
  if (z == y) {
    if (y == x + 10) {
      crash();
    }
  }
}

bug-finding tools

• types of tools
  • static tools: analyze source code without running (a smart compiler)
  • dynamic tools: run the code (and can try to break it)
  • the line is blurred
  • false positives vs. false negatives
• examples
  • gcc/clang
  • sparse, annotations, and some history
  • fuzzing: afl-fuzz, libFuzzer
  • bounded model checking: cbmc
• try the following code example
  • static analysis: run the analyzer in Xcode
  • compile using gcc/clang’s -fsanitize=address

#include <stdio.h>
#include <stdlib.h>

int foo(int n)
{
        int *arr = malloc(n * sizeof(int));
        //arr[0] = 42;
        //free(arr);
        return arr[0];
}

int main(int argc, const char * argv[]) {
    printf("%d\n", foo(argc));
    return 0;
}

• try the following code example
  • clang -fsanitize=fuzzer -g -o test test.c
  • cbmc --function test_me --trace test.c
  • run this in KLEE yourself

#include <assert.h>
#include <stdint.h>
#include <string.h>

void test_me(int x, int y) {
  int z = 2 * x;
  if (z == y) {
    if (y == x + 10) {
      assert(0);
    }
  }
}

int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
  int arr[2];

  if (Size == sizeof(arr)) {
    memcpy(&arr, Data, sizeof(arr));
    test_me(arr[0], arr[1]);
  }
  return 0;
}

• example: STACK

other approaches

• better language/library design: e.g., kmalloc_array vs. kmalloc
• formal verification