let rec qsort(lst) =
match lst with
| [] -> []
Then I asked if anyone remembered how quicksort works. Someone mentioned that
the main steps are to:
I said that we'd keep things simple by using the first value in the list as our pivot. This isn't an ideal choice, especially if the list is already sorted, but it will work well for the randomized lists we want to work with. So I changed the variable names in our code to reflect this choice:
let rec qsort(lst) =
match lst with
| [] -> []
| pivot::rest ->
The first step in solving this is to partition the list, so I asked
people how to implement this in OCaml and people seemed to be stumped.
That's not surprising because we're just starting to learn OCaml and
this is a nontrivial computation. I said that you don't have to
abandon your programming instincts from procedural programming. So
how would you solve it in a language like Java if you were asked to
work with a linked list?By thinking through that, we developed the following pseudocode for partitioning the list:
start with an empty lst1
start with an empty lst2
while (more values from original list)
look at first element
if it is <= pivot, put it in lst1
else put it in lst2
}
finish up
I said that you can convert this iterative solution to a recursive
solution without much trouble. With a loop-based approach you use a
set of local variables to keep track of the current state of your
computation. Here there are three such local variables: the list that
we are partitioning, the first partition, and the second partition.
Local variables like these become parameters to a helper function.
Our helper function is supposed to split the list, so we decided to
call it "split". Since the loop involves three variables, two of
which are initialized to be empty lists, we write split as a helper
function of three parameters and we include empty lists for two of the
parameters in the initial call:
let rec qsort(lst) =
match lst with
| [] -> []
| pivot::xs ->
let rec split(rest, part1, part2) = ?
in split(xs, [], [])
Notice how the call after the word "in" exactly parallels the
situation before our loop begins executing. Our three state variables
are the list of values to split (which I have called "rest") and two
variables for storing the two partitions which are initially empty.In our pseudocode we continue until the overall list becomes empty, at which time we "finish up" the computation. We can include this as one of the cases for our helper function:
let rec qsort(lst) =
match lst with
| [] -> []
| pivot::xs ->
let rec split(rest, part1, part2) =
match rest with
| [] -> finish up
in split(xs, [], [])
Compare this initial case for split versus the initial call for split. We go
from having these three values for our computation:
(original list, [], [])
to having this set of values for our computation:
([], partition 1, partition 2)
In other words, we go from having all of the values stored in the original list
and having two empty partitions to having an empty list of values and two
partitions that have been filled in. Now we just have to describe how we go
from one to the other. In our pseudocode, each time through the loop we
handled one element of the original list, either moving it to partition 1 or
moving it to partition 2. We can do the same thing with our helper function.
So first we need to indicate that in the second case for split, we want to
process one value from the original list. We do so by including a
pattern for lst
with "x::xs":.
let rec qsort(lst) =
match lst with
| [] -> []
| pivot::xs ->
let rec split(rest, part1, part2) =
match rest with
| [] -> finish up
| y::ys -> ?
in split(rest, [], [])
We had an if/else in the loop and we can use an if/else here. I asked people
if they wanted to include values equal to the pivot in the first or second
partition and the first person to answer said to put them in the first
partition, so we expanded this to:
let rec qsort(lst) =
match lst with
| [] -> []
| pivot::xs ->
let rec split(rest, part1, part2) =
match rest with
| [] -> finish up
| y::ys -> if y <= pivot then (y goes in 1st partition)
else (y goes in 2nd partition)
in split(xs, [], [])
If y belongs in the first partition, then we want to go from having this set of
values:
(y::ys, part1, part2)
to having this set of values:
(ys, y::part1, part2)
In other words, we move y from the list of values to be processed into the
first partition. In the loop, we'd then come around the loop for the next
iteration. In a recursive solution, we simply make a recursive call with those
new values:
let rec qsort(lst) =
match lst with
| [] -> []
| pivot::xs ->
let rec split(rest, part1, part2) =
match rest with
| [] -> finish up
| y::ys -> if y <= pivot then split(ys, y::part1, part2)
else (y goes in 2nd partition)
in split(xs, [], [])
In the second case we do something similar, moving y into the second partition
and using a recursive call to continue the computation:
let rec qsort(lst) =
match lst with
| [] -> []
| pivot::xs ->
let rec split(rest, part1, part2) =
match rest with
| [] -> finish up
| y::ys -> if y <= pivot then split(ys, y::part1, part2)
else split(ys, part1, y::part2)
in split(xs, [], [])
The only thing we had left to fill in was the "finish up" part. We
often return an empty list when the list we are processing becomes
empty. While that might be an appropriate choice for a splitting
utility, it's not what we want here. Remember that reaching that
point in the code is like finishing the loop in our pseudocode. We
started out by saying that we need to quicksort the two partitions.
That needs to be part of what we do in the "finish up" part:
let rec qsort(lst) =
match lst with
| [] -> []
| pivot::xs ->
let rec split(rest, part1, part2) =
match rest with
| [] -> need to: qsort(part1), qsort(part2)
| y::ys -> if y <= pivot then split(ys, y::part1, part2)
else split(ys, part1, y::part2)
in split(xs, [], [])
The qsort function returns a list, so what do we do with the two sorted lists
that come back from these recursive calls? We glue them together with
append:
let rec qsort(lst) =
match lst with
| [] -> []
| pivot::xs ->
let rec split(rest, part1, part2) =
match rest with
| [] -> qsort(part1) @ qsort(part2)
| y::ys -> if y <= pivot then split(ys, y::part1, part2)
else split(ys, part1, y::part2)
in split(xs, [], [])
This is not quite right, though, because we haven't accounted for the pivot.
We didn't add it to either of our partitions. That's, in general, a good
thing, because it guarantees that each of the partitions we recursively pass to
qsort will be smaller than the original list. But it means we have to manually
place the pivot into the result. It belongs right in the middle of the two
partitions. We could use a cons operator ("::") to indicate this, but I think
it's a little clearer in this case to show that we are gluing three pieces
together, one of which is the pivot itself:
let rec qsort(lst) =
match lst with
| [] -> []
| pivot::xs ->
let rec split(rest, part1, part2) =
match rest with
| [] -> qsort(part1) @ [pivot] @ qsort(part2)
| y::ys -> if y <= pivot then split(ys, y::part1, part2)
else split(ys, part1, y::part2)
in split(xs, [], [])
At that point we were done. This is a working version of quicksort.
I loaded it in the OCaml interpreter and we managed to sort some short
lists. It is polymorphic, so it can be used to sort lists of any
type.I introduced a new topic. Programmers have a notion of what we call first class citizens in a programming language. First class citizens are values that can be stored in variables, passed as parameters, and used as the return values.
Everyone understands that numbers are first class citizens in most programming languages. You can store numbers, you can pass numbers as parameters to a function, and you can return numbers from a function. Novices find it a bit odd that boolean values are also first class citizens in most programming languages including Java, C, C++, Python, and so on. Novices often find it puzzling to think that a variable can store true or false and in intro Java courses students often find it challenging to write methods that have a boolean return type.
In functional languages, functions themselves are first class citizens. We have seen that they can be stored in variables in the sense that we can use a let definition or expression to bind a name to a function. We're about to explore passing functions as parameters and returning functions from a function. We call functions that take other functions as parameters or that return them as their result as higher order functions.
I said that I wanted to add a comparison function to the qsort function we wrote in the previous lecture. I showed it's current behavior using a short list of strings that I had defined:
# t;;
- : string list =
["to"; "be"; "and"; "not"; "to"; "be"; "that"; "is"; "the"; "question"]
# qsort(t);;
- : string list =
["and"; "be"; "be"; "is"; "not"; "question"; "that"; "the"; "to"; "to"]
As expected, it sorts it into alphabetical order. I asked people what
would have to change in order to have qsort take a comparison function
as a parameter. Clearly the header changed. And the two recursive
calls. And the test used in the if/else went from testing "x <=
pivot" to a call on the comparison function.
let rec qsort(f, lst) =
match lst with
| [] -> []
| pivot::rest ->
let rec split(lst, part1, part2) =
match lst with
| [] -> qsort(f, part1) @ [pivot] @ qsort(f, part2)
| x::xs -> if f(x, pivot) then split(xs, x::part1, part2)
else split(xs, part1, x::part2)
in split(rest, [], [])
With this version, we were able to sort the list by string length
instead of alphabetically:
- : string list =
["to"; "be"; "to"; "is"; "be"; "the"; "and"; "not"; "that"; "question"]
But now we weren't able to sort alphabetically. That's because we
have assumed that the comparison function will take a pair of values
and return a boolean value. The built-in operator less-than is of a
different from:
# shorter;;
- : string * string -> bool = <fun>
# (<);;
- : 'a -> 'a -> bool = <fun>
The less-than operator is what is known as a curried function. We'll
discuss that more in the next lecture, but for now we just need to
notice that it expects a different form for the two values to
compare. It is possible to convert a curried function into the
uncurried form:
let uncurry f(a, b) = f a b
With this function available to us, we were able to sort the list of
strings alphabetically and in reverse order:
# qsort(uncurry(<), t);;
- : string list =
["and"; "be"; "be"; "is"; "not"; "question"; "that"; "the"; "to"; "to"]
# qsort(uncurry(>), t);;
- : string list =
["to"; "to"; "the"; "that"; "question"; "not"; "is"; "be"; "be"; "and"]
I then introduced what can be thought of as the "big three" of
higher-order functions: map, filter, and reduce. I mentioned that I
am going to begin by showing versions of these functions that take a
pair (2-tuple) as parameters, like the other functions we've been
writing, but in OCaml we normally use a different version of these
functions that we'll discuss in future lectures.We first looked at map:
let rec map(f, lst) =
match lst with
| [] -> []
| x::xs -> f(x)::map(f, xs)
This function applies a function to every element of a list. I asked
people how to convert from int to float and someone said the function
is called float_of_int. We can map that over a list of int values
obtained using our -- operator:
# map(float_of_int, 1--10);;
- : float list = [1.; 2.; 3.; 4.; 5.; 6.; 7.; 8.; 9.; 10.]
I tried asking for the square root of a list of ints, but that
failed:
# map(sqrt, 1--10);;
Error: This expression has type int list
but an expression was expected of type float list
Type int is not compatible with type float
The sqrt function requires a parameter of type float, so this didn't
work. But we can fix it by first mapping float_of_int over the list
and them mapping sqrt over that:
# map(sqrt, map(float_of_int, 1--10));;
- : float list =
[1.; 1.41421356237309515; 1.73205080756887719; 2.; 2.23606797749979;
2.44948974278317788; 2.64575131106459072; 2.82842712474619029; 3.;
3.16227766016837952]
I pointed out that in a procedural style program we tend to write
command oriented functions like "convert this to double" and "compute
the square root." In a functional language we tend to compose
function calls, as in the example above (f(g(h(...)))).Then I showed the filter function, which takes a predicate function (a boolean test) and a list as arguments and that returns the list of values that satisfy the given predicate. We write it this way:
let rec filter(f, lst) =
match lst with
| [] -> []
| x::xs -> if f(x) then x::filter(f, xs)
else filter(f, xs)
There is a repeated computation of filter(f, xs) that we could factor
out by using a let expression to store the result in a local variable,
but this doesn't introduce the kind of inefficiency problems we saw
with the min function we wrote in an earlier lecture. Here the call
on filter is computed only once in each branch.Using this definition and the function is_prime we wrote in the previous lecture, I was able to ask for primes up to 100:
# filter(is_prime, 1--100);;
- : int list =
[2; 3; 5; 7; 11; 13; 17; 19; 23; 29; 31; 37; 41; 43; 47; 53; 59; 61; 67; 71; 73; 79; 83; 89; 97]
The third higher-order function we looked at was reduce. The idea of
reduce is that we often work with binary functions that take two
values and combine them into one. This is true of simple arithmetic
with addition, subtract, multiplication, etc. For example, given two
numbers, we can find their sum. The reduce function takes a list of
values and use a binary function to turn it into a single value, like
the sum of a list.We intially wrote reduce this way:
let rec reduce(f, lst) =
match lst with
| [x] -> x
| x::xs -> f(x, reduce(f, xs))
This produced the following warning:
Warning 8: this pattern-matching is not exhaustive.
Here is an example of a case that is not matched:
[]
An empty list poses a problem for us. What is the sum of an empty
list of numbers? It seems reasonable to say that it is 0. But what
is the product of an empty list of numbers? We probably want it to be
1 because 1 is the multiplicative identity (we start variables at 1
when we compute a cumulative product).There are several ways to deal with this and we will explore all of them. For now, I called the invalid_arg function.
let rec reduce(f, lst) =
match lst with
| [] -> invalid_arg("empty reduce")
| [x] -> x
| x::xs -> f(x, reduce(f, xs))
Using this we were able to add up the numbers 1 through 10:
# reduce(uncurry(+), 1--10);;
- : int = 55
The key to the reduce function is that it collapes a sequence of values to a
single value, as in the example above of adding them up to get the overall sum.
I asked people for other examples of collapsing operations and we came up with
a rather extensive list:
| function | 1st argument | 2nd argument | returns |
|---|---|---|---|
| map | function mapping 'a to 'b | list of n 'as | list of n 'bs |
| filter | predicate converting 'a to bool | list of n 'as | list of m 'as (m <= n) |
| reduce | function that collapses a tuple 'a * 'a into a single 'a |
list of n 'as | one 'a |
Then I mentioned that in the early 2000s Google created a system they called MapReduce for performing large scale computations. Operations like map and reduce are relatively easy to split up across multiple processors. So this is one way to do parallel processing without having to think a lot about the fact that you are running on different machines. There was an open source version of MapReduce called Hadoop and Google even gave UW funding for a cluster of machines and to allow us to teach a class on Hadoop programming.
Then I mentioned that Java added functional constructs starting with Java 8 and that they included all of map, filter, and reduce. I have given a lecture on Java 8 features in some of my 143 classes. One of the more interesting bits of that lecture has to do with computing whether various numbers are prime. I wrote this method for testing whether something is prime:
public static boolean isPrime(int n) {
return IntStream.range(1, n + 1)
.filter(x -> n % x == 0)
.count()
== 2;
}
It examines the numbers 1 through n, filtering for numbers that are factors of
n, and it sees whether the count of factors is equal to 2. This matches the
standard mathematical defintion of a prime as an integer that has exactly two
factors (1 and itself). This is not a very efficient version, but it is
interesting how simple it is.The more interesting bit of code involves adding up the sum of prime numbers between 1 and 20,000. I ran the code twice keeping track of how long it took to execute but for the second execution I added a notation ".parallel()" which said to Java that it was allowed to compute this on multiple processors. When I ran this on a machine called barb which is one of the research unix servers in the Allen School, it indicated these times for the two different runs:
n = 21171191, time = 0.743
n = 21171191, time = 0.048
It got the same answer, but the version that was allowed to run in parallel ran
much faster (a little over 15 times faster). That's because we allowed Java to
run it on multiple cores at the same time. I suggested to students that they
download the program and run it on their own machine to see what results they
get.The key point is that functional approaches like using map, filter, and reduce are attractive to software engineers because they allow you to write code that runs in parallel without having to think much about how the tasks are split up across different computers.