CSE143X Notes for Wednesday, 10/30/13
I began by talking about three different ways to traverse a list structure.
Remember that our standard template for traversing an array is:
for (int i = 0; i < arry.length; i++) {
// do something with list[i]
}
I had mentioned in an earlier lecture that there is a similar template for
traversing a List structure using the size and get methods. Applying that to
our variable called list, we can print the list as follows:
for (int i = 0; i < list.size(); i++) {
System.out.println(list.get(i));
}
But if all you want to do is to iterate over the values of a list one at a
time, you can use what is called a for-each loop:
for (String s : list) {
System.out.println(s);
}
We generally read the for loop header as, "For each String s in list...". The
choice of "s" is arbitrary. It defines a local variable for the loop. I could
just as easily have called it "x" or "foo" or "value". Notice that in the
for-each loop, I don't have to use any bracket notation. Instead, each time
through the loop Java sets the variable s to the next value from the list. I
also don't need to test for the size of the list. Java does that for you when
you use a for-each loop.
There are some limitations of for-each loops. You can't use them to change the
contents of the list. If you assign a value to the variable s, you are just
changing a local variable inside the loop. It has no effect on the list
itself.
You can also traverse a list using an iterator. Iterator objects implement an
interface known as Iterator<E>. So we can also traverse our list of strings
this way:
Iterator<String> i = list.iterator();
while (i.hasNext()) {
System.out.println(i.next());
}
The foreach loop uses an iterator, so really these represent two different
approaches. The first version works, but it relies on the "get" method being
able to quickly access any element of the structure. This property is known as
random access. We say that arrays and the ArrayList and ArrayIntList
classes that are built on top of them are random access structures because we
can quickly access any element of the structure. If you knew that for the rest
of your life, you'd always be working with arrays, then you'd have little use
for iterators. You'd just call the get method because with arrays you get fast
random access.
But many of the other data structures we will be looking at don't have this
kind of quick access. In particular, linked lists have what we call
sequential access. Think of how a DVD works, quickly jumping to any
scene, or how a CD works, quickly jumping to any track, versus how a VHS tape
works, requiring you to fast forward through every scene until you get to the
one you want. A linked list is more like a tape player that requires you to
sequentially access all of the values in a row.
Imagine, for example, that you're working with a list of ten thousand values
and think of what happens when you're halfway through the process. You will
ask the list to get the element at index 5000. For a linked list, this
requires starting at the beginning of the list and moving forward 4999 times.
And then after that, you'd come around the for loop and get the element at
index 5001. The linked list would start all over again at the beginning and
move forward 5000 times. This is extremly inefficient. In fact, it turns what
would be an O(n) operation into an O(n2) operation.
I also mentioned that the iterator is what we would call a lightweight
object. You can use the iterator to gain access to everything in the
structure, but it doesn't store the data itself. I gave the analogy that this
is like going to a pharmacy and you'd really like to just jump over the counter
and grab your prescription, but instead you have to talk to the guy behind the
counter. The guy behind the counter has access to everything in the pharmacy.
But he's not the pharmacy. He's a person with access to the pharmacy and you
(the client) talk to the guy behind the counter to get things done. That's how
an iterator works. It has full access to the underlying structure and it keeps
track of how much of the structure it has traversed, but that's not the same
thing as being the pharmacy.
Then I then spent a little time discussing the issue of primitive data versus
objects. Even though we can construct an ArrayList<E> for any class E, we
can't construct an ArrayList<int> because int is a primitive type, not a class.
To get around this problem, Java has a set of classes that are known as
"wrapper" classes that "wrap up" primitive values like ints to make them an
object. It's very much like taking a candy and putting a wrapper around it.
For the case of ints, there is a class known as Integer that can be used to
store an individual int. Each Integer object has a single data field: the int
that it wrapped up inside.
Java also has quite a bit of support that makes a lot of this invisible to
programmers. If you want to put int values into an ArrayList, you have to
remember to use the type ArrayList<Integer> rather than ArrayList<int>, but
otherwise Java does a lot of things for you. For example, you can construct
such a list and add simple int values to it:
List<Integer> numbers = new ArrayList<Integer>();
numbers.add(18);
numbers.add(34);
In the two calls on add, we are passing simple ints as arguments to something
that really requires an Integer. This is okay because Java will automatically
"box" the ints for us (i.e., wrap them up in Integer objects). We can also
refer to elements of this list and treat them as simple ints, as in:
int product = numbers.get(0) * numbers.get(1);
The calls on list.get return references to Integer objects and normally you
wouldn't be allowed to multiply two objects together. In this case Java
automatically "unboxes" the values for you, unwrapping the Integer objects and
giving you the ints that are contained inside.
Every primitive type has a corresponding wrapper class: Integer for int, Double
for double, Character for char, Boolean for boolean, and so on.
Then I mentioned that we will be looking at a kind of structure known as a
Set. There is an interface Set<E>. For now, all of the sets we will construct
all of our sets using the TreeSet<E> class. For example, I used an array of
data to initialize both a list and a set by adding values from the array to
each:
int[] data = {18, 4, 97, 3, 4, 18, 72, 4, 42, 42, -3};
List<Integer> numbers1 = new ArrayList<Integer>();
Set<Integer> numbers2 = new TreeSet<Integer>();
for (int n : data) {
numbers1.add(n);
numbers2.add(n);
}
System.out.println("numbers1 = " + numbers1);
System.out.println("numbers2 = " + numbers2);
This produced the following output:
numbers1 = [18, 4, 97, 3, 4, 18, 72, 4, 42, 42, -3]
numbers2 = [-3, 3, 4, 18, 42, 72, 97]
There are two major differences between a set and a list. Sets don't allow
duplicates. So the duplicate values like 42 and 4 in the array appear just
once in the set. Sets also don't allow the client to control the order of
elements. The TreeSet class keeps things in sorted order. So the numbers will
always be in that order. If you want to control the order, then you should use
a list instead.
Sets have many of the same methods that lists do. You can add to a set, get
its size, ask for an iterator, use it with a foreach loop. But it doesn't
have a notion of indexing. So you can't remove at an index. Instead you
remove a specific value. And you can't get at a specific index. Instead you
use an iterator or a foreach loop.
We looked at this short example of using an iterator to remove the multiples of
3 from the set:
Iterator<Integer> i = numbers2.iterator();
while (i.hasNext()) {
int next = i.next();
if (next % 3 == 0) {
i.remove();
}
}
System.out.println("now numbers2 = " + numbers2);
which produced the following output:
now numbers2 = [4, 97]
This is the approach you need to take when you want to both examine and remove
values from a set. Notice that we called the iterator's remove method and not
the set's remove method. You are not allowed to alter a set while you are
iterating over it. That means that you also can't alter it while performing a
foreach loop because the foreach loop is implemented using an iterator. If you
attempt to modify the structure while iterating over it, the iterator will
throw a ConcurrentModificationException. In terms of the pharmacy analogy, you
will end up confusing the guy behind the counter if you try to change what's in
the pharmacy while he's iterating over it.
Then we turned to a new example. We didn't have time to do either of the
normal 143 examples in detail, but I include here the full discussion as if we
had.
I began by asking how we could write a program that would count the number of
unique words in an input file. I had a copy of the text of Moby Dick
that we looked at to think about this. I showed some starter code that
constructs a Scanner object tied to a file:
import java.util.*;
import java.io.*;
public class WordCount {
public static void main(String[] args) throws FileNotFoundException {
Scanner console = new Scanner(System.in);
System.out.print("What is the name of the text file? ");
String fileName = console.nextLine();
Scanner input = new Scanner(new File(fileName));
while (input.hasNext()) {
String next = input.next();
// process next
}
}
}
Notice that in the loop we use input.next() to read individual words and we
have this in a while loop testing against input.hasNext().
So how do we count the words? Someone suggested that a set would be the
perfect structure to solve this problem. It eliminates duplicates, so it will
keep track of how many different words there are. So we changed the code to
be:
Set<String> words = new TreeSet<String>();
while (input.hasNext()) {
String next = input.next();
words.add(next);
}
System.out.println("Total words = " + words.size());
Here is a sample log of execution:
What is the name of the text file? moby.txt
Total words = 32019
One limitation of this version is that it pays attention to capitalization. So
it considers the words "whale" and "Whale" and "WHALE" to be different words.
To fix that, we modified the code to read in a word so that it converts it to
its lowercase equivalent:
String next = input.next().toLowerCase();
It didn't make much difference, as we saw from this execution:
What is the name of the text file? moby.txt
Total words = 30368
Someone pointed out that we still haven't dealt with punctuation. It is
considering "whale" and "whale." and "whale," to be different words. I didn't
want to deal with it in this program, but I mentioned that the chapter 10 case
study discusses this and shows you how to configure the Scanner so that it
ignores those punctuation characters.
This program counts the number of unique words, but not the counts of the
individual words. To keep track of word counts, we could use many different
structures. For example, we could have a List of words and a List of counts
where element "i" in one corresponds to element "i" in the other. This
approach is often described as "parallel arrays." It's not a very
object-oriented approach because we really want to associate the word with its
counts rather than have a structure that puts all the words together and
another that puts all the counts together. Or we could make a class for a
word/count combination and then have a List of that. But Java gives us a
better alternative. The collections framework provides a data abstraction
known as a map.
The idea behind a map is that it keeps track of key/value pairs. In our case,
we want to keep track of word/count pairs (what is the count for each different
word). We often store data this way. For example, in the US we often use a
person's social security number as a key to get information about them. I
would expect that if I talked to the university registrar, they probably have
the ability to look up students based on social security number to find their
transcript.
In a map, there is only one value for any given key. If you look up a social
security number and get three different student transcripts, that would be a
problem. With the Java map objects, if you already have an entry in your map
for a particular key, then any attempt to put a new key/value pair into the
map will overwrite the old mapping.
We looked at an interface in the Java class libraries called Map that is a
generic interface. That means that we have to supply type information. It's
formal description is Map<K, V>. This is different from the List and Set
interfaces in that it has two different types. That's because the map has to
know what type of keys you have and what type of values you have. In our case,
we have some words (Strings) that we want to associated with some counters
(ints). We can't actually use type int because it is a primitive type, but we
can use type Integer.
So our map would be of type Map<String, Integer>. In other words, it's a
a map that keeps track of String/Integer pairs (this String goes to this
Integer). Map is the name of the interface, but it's not an actual
implementation. The implementation we will use is TreeMap. So we can
construct a map called "count" to keep track of our counts by saying:
Map<String, Integer> count = new TreeMap<String, Integer>();
The most basic methods in the map interface are the ones that allow you to put
something into the map (an operation called put) and to ask the map for the
current value of something (an operation called get).
I asked what code we need to record a word in our map the first time we see it.
Someone suggested using the put method to assign it to a count of 1. So our
loop becomes:
Map<String, Integer> count = new TreeMap<String, Integer>();
while (input.hasNext()) {
String next = input.next().toLowerCase();
count.put(next, 1);
}
This doesn't quite work, but it's getting closer. Each time we encounter a
word, it adds it to our map, associating it with a count of 1. This will
figure out what the unique words are, but it won't have the right counts for
them.
I asked people to think about what to do if a word has been seen before. In
that case, we want to increase its count by 1. That means we have to get the
old value of the count and add 1 to it:
count.get(next) + 1
and make this the new value of the counter:
count.put(next, count.get(next) + 1);
So we have two different calls on put. We want to call the first one when the
word is first seen and call the second one if it's already been seen. Someone
suggested using an if/else for this. The only question is what test to use.
The Map includes a method called containsKey that tests whether or not a
certain value is a key stored in the map. Using this method, we modified our
code to be:
Map<String, Integer> count = new TreeMap<String, Integer>();
while (input.hasNext()) {
String next = input.next().toLowerCase();
if (!count.containsKey(next)) {
count.put(next, 1);
} else {
count.put(next, count.get(next) + 1);
}
}
The first time we see a word, we call the put method and say that the map
should associate the word with a count of 1. Later we call put again with a
higher count. And we keep calling put every time the count goes up. What
happens to the old values that we had put in the map previously? The way the
map works, each key is associated with only one value. So when you call put a
second or third time, you are wiping out the old association. The new
key/value pair replaces the old key/value pair in the map.
Then we talked about how to print the results. Clearly we need to iterate over
the entries in the map. One way to do this is to request what is known as the
"key set". The key set is the set of all keys contained in the map. The Java
documentation says that it will be of type Set. We don't have to really
worry about this if we use a for-each loop. Remember that a for-each loop
iterates over all of the values in a given collection. So we can say:
for (String word : count.keySet()) {
// process word
}
We would read this as, "for each String word that is in count.keySet()..."
To process the word, we simply print it out along with its count. How do we
get its count? By calling the get method of the map:
for (String word : count.keySet()) {
System.out.println(count.get(word) + "\t" + word);
}
I didn't try to print all of the words in Moby Dick because it would
have produced too much output. Instead, I had it show me the counts of words
in the program itself. Obviously for large files we want some mechanism to
limit the output. On the calendar I will put a version that includes some
extra code that asks for a minimum frequency to use. We ran that on Moby
Dick and saw this list of words that occur at least 500 times:
What is the name of the text file? moby.txt
Minimum number of occurrences for printing? 500
4571 a
1354 all
587 an
6182 and
563 are
1701 as
1289 at
973 be
1691 but
1133 by
1522 for
1067 from
754 had
741 have
1686 he
552 him
2459 his
1746 i
3992 in
512 into
1555 is
1754 it
562 like
578 my
1073 not
506 now
6408 of
933 on
775 one
675 or
882 so
599 some
2729 that
14092 the
602 their
506 there
627 they
1239 this
4448 to
551 upon
1567 was
644 were
500 whale
552 when
547 which
1672 with
774 you
One final point I made about the Map interface is that you can associate
just about anything with just about anything. In the word counting program, we
associated strings with integers. You could also associate strings with
strings. One thing you can't do is to have multiple associations in a single
map. For example, if you decide to associate strings with strings, then any
given string can be associated with just a single string. But there's no
reason that you can't have the second value be structured in some way. You can
associate strings with arrays or strings with Lists.
In the final version that I posted on the calendar, I made one minor change.
There is an interface known as SortedMap. When you know that you want to
require a map to have its keys sorted, it is more appropriate to use that
interface. Every SortedMap is a Map, but not every Map is a SortedMap. So the
program changes the declaration to:
SortedMap<String, Integer> count = new TreeMap<String, Integer>();
Then I mentioned that I wanted to explore a sample program that will constitute
a medium hint for the programming assignment. The sample program involves
keeping track of friendships. You could think of it as keeping track of
Facebook friends. One of the first questions that comes up is how do we
represent friendships? For example, are friendships bidirectional? If person
A is friends with person B, does that mean that person B is friends with person
A? For our purposes, we will assume the answer is yes. If we were trying to
represent something like "is attracted to", then we'd come to a different
conclusion, but for friends, just like on Facebook and other social networking
sites, friendship goes both ways.
I said that a good way to visualize friendships is to draw a graph in which
each person is represented with a node (an oval) and each friendship is
represented by an edge connecting two nodes (a line drawn between two ovals).
I am using a program called Graphviz, which is an open-source graph viewer.. For example,
here is a sample friendship graph:
This information is stored in a file with lines that list pairs of friendships,
as in:
graph {
Ashley -- Christopher
Ashley -- Emily
Ashley -- Joshua
Bart -- Lisa
Bart -- Matthew
Christopher -- Andrew
Emily -- Joshua
Jacob -- Christopher
Jessica -- Ashley
JorEl -- Zod
KalEl -- JorEl
Kyle -- Lex
Kyle -- Zod
Lisa -- Marge
Matthew -- Lisa
Michael -- Christopher
Michael -- Joshua
Michael -- Jessica
Samantha -- Matthew
Samantha -- Tyler
Sarah -- Andrew
Sarah -- Christopher
Sarah -- Emily
Tyler -- Kyle
Stuart -- Jacob
}
I asked people what kind of structure would be useful for keeping track of this
kind of data and someone said a map. But what kind of map? Someone suggested
that it would be good to keep track of the neighbors for each person. The
neighbors are the friends. For example, Ashley's friends are Christopher,
Emily, Jessica, and Joshua.
If we want a structure that keeps track of these kind of friendships, then we
want to use names as keys into the structure. We ask the structure, "Who are
the friends of Samantha?" or "Who are the friends of Ashley?". So a name, a
String, will be used as the key. But what should it return? If we map a
String to a String, then we can store only one friendship. We want to be able
to return more than one friendship. Someone suggested that we want to use a
set. That is exactly right.
The idea is that we want to have a map that converts a String into a Set of
String values. Given the name of a person, we can get a Set with the names of
that person's friends. For our sample file:
"Andrew" => maps to => [Christopher, Sarah]
"Andrew" => maps to => [Christopher, Sarah]
"Ashley" => maps to => [Christopher, Emily, Jessica, Joshua]
"Bart" => maps to => [Lisa, Matthew]
"Christopher" => maps to => [Andrew, Ashley, Jacob, Michael, Sarah]
"Emily" => maps to => [Ashley, Joshua, Sarah]
"Jacob" => maps to => [Christopher, Stuart]
"Jessica" => maps to => [Ashley, Michael]
"JorEl" => maps to => [KalEl, Zod]
"Joshua" => maps to => [Ashley, Emily, Michael]
"KalEl" => maps to => [JorEl]
"Kyle" => maps to => [Lex, Tyler, Zod]
"Lex" => maps to => [Kyle]
"Lisa" => maps to => [Bart, Marge, Matthew]
"Marge" => maps to => [Lisa]
"Matthew" => maps to => [Bart, Lisa, Samantha]
"Michael" => maps to => [Christopher, Jessica, Joshua]
"Samantha" => maps to => [Matthew, Tyler]
"Sarah" => maps to => [Andrew, Christopher, Emily]
"Stuart" => maps to => [Jacob]
"Tyler" => maps to => [Kyle, Samantha]
"Zod" => maps to => [JorEl, Kyle]
Our first challenge, then, is to write code to construct such a structure. If
it maps a String to a Set<String>, then it would be of this type:
Map<String, Set<String>>
To construct one, we have to ask for a new TreeMap of this type:
Map<String, Set<String>> friends = new TreeMap<String, Set<String>>();
That is a rather complex line of code, but the main complexity comes from what
we are putting inside the "<" and ">" characters.
I said that we would finish up this example in the next lecture.
Stuart Reges
Last modified: Fri Nov 1 13:46:39 PDT 2013