Lecture: bugs and testing

preparation

• finish the undefined behavior exercise

bug-finding tools

• static tools: analyze source code without running (a smart compiler)
• dynamic tools: run the code (and can try to break it)
• the line is blurred
• what bug-finding tools have you used?
  • what are considered as “bugs”?
  • how to find such bugs
• examples
  • gcc/clang/llvm
  • sparse and some history
  • valgrind and 333 notes
• false positives vs. false negatives

#include <stdio.h>
#include <stdlib.h>

int foo(int n)
{
	int *arr = malloc(n * sizeof(int));
	//arr[0] = 42;
	//free(arr);
	return arr[0];
}

int main(int argc, const char * argv[]) {
    printf("%d\n", foo(argc));
    return 0;
}

testing systems code

• example: how to test your JOS kernel
  • what’s considered correct (specification)?
  • complex input sources & state transitions
  • randomly generate sytem calls?
• challenge: generate “useful” tests
  • what’s the chance of randomly generating (x, y) to trigger crash? 1/2⁶⁴
  • blackbox: infinite monkey theorem
• symbolic execution
  • whitebox: implementation knowledge
  • compare the search space to the space of input
  • we need an oracle that is able to efficiently solve the path conditions

SAT/SMT solver

• termininology
  • SAT: boolean satisfiability
  • SMT: satisfiability modulo theories
• breakthrough in SAT/SMT solving
• building block for modern tools: MS Office (FlashFill), Visual Studio (IntelliTest, StaticDV), …
• example: the STACK undefined behavior checker
• next lecture: verification
• take 507 if you are interested