for (int i = 0; i < list.size(); i++) { System.out.println(list.get(i)); } for (String s : list) { System.out.println(s); } Iterator<String> i = list.iterator(); while (i.hasNext()) { System.out.println(i.next()); }The foreach loop uses an iterator, so really these represent two different approaches. The first version works, but it relies on the "get" method being able to quickly access any element of the structure. This property is known as random access. We say that arrays and the ArrayList and ArrayIntList classes that are built on top of them are random access structures because we can quickly access any element of the structure. If you knew that for the rest of your life, you'd always be working with arrays, then you'd have little use for iterators. You'd just call the get method because with arrays you get fast random access.
But many of the other data structures we will be looking at don't have this kind of quick access. In particular, linked lists have what we call sequential access. Think of how a DVD works, quickly jumping to any scene, or how a CD works, quickly jumping to any track, versus how a VHS tape works, requiring you to fast forward through every scene until you get to the one you want. A linked list is more like a tape player that requires you to sequentially access all of the values in a row.
Imagine, for example, that you're working with a list of ten thousand values and think of what happens when you're halfway through the process. You will ask the list to get the element at index 5000. For a linked list, this requires starting at the beginning of the list and moving forward 4999 times. And then after that, you'd come around the for loop and get the element at index 5001. The linked list would start all over again at the beginning and move forward 5000 times. This is extremly inefficient. In fact, it turns what would be an O(n) operation into an O(n2) operation.
I also mentioned that the iterator is what we would call a lightweight object. You can use the iterator to gain access to everything in the structure, but it doesn't store the data itself. I gave the analogy that this is like going to a pharmacy and you'd really like to just jump over the counter and grab your prescription, but instead you have to talk to the guy behind the counter. The guy behind the counter has access to everything in the pharmacy. But he's not the pharmacy. He's a person with access to the pharmacy and you (the client) talk to the guy behind the counter to get things done. That's how an iterator works. It has full access to the underlying structure and it keeps track of how much of the structure it has traversed, but that's not the same thing as being the pharmacy.
Then I then spent a little time discussing the issue of primitive data versus objects. Even though we can construct an ArrayList<E> for any class E, we can't construct an ArrayList<int> because int is a primitive type, not a class. To get around this problem, Java has a set of classes that are known as "wrapper" classes that "wrap up" primitive values like ints to make them an object. It's very much like taking a candy and putting a wrapper around it. For the case of ints, there is a class known as Integer that can be used to store an individual int. Each Integer object has a single data field: the int that it wrapped up inside.
Java also has quite a bit of support that makes a lot of this invisible to programmers. If you want to put int values into an ArrayList, you have to remember to use the type ArrayList<Integer> rather than ArrayList<int>, but otherwise Java does a lot of things for you. For example, you can construct such a list and add simple int values to it:
List<Integer> numbers = new ArrayList<Integer>(); numbers.add(18); numbers.add(34);In the two calls on add, we are passing simple ints as arguments to something that really requires an Integer. This is okay because Java will automatically "box" the ints for us (i.e., wrap them up in Integer objects). We can also refer to elements of this list and treat them as simple ints, as in:
int product = numbers.get(0) * numbers.get(1);The calls on list.get return references to Integer objects and normally you wouldn't be allowed to multiply two objects together. In this case Java automatically "unboxes" the values for you, unwrapping the Integer objects and giving you the ints that are contained inside.
Every primitive type has a corresponding wrapper class: Integer for int, Double for double, Character for char, Boolean for boolean, and so on.
Then I mentioned that we will be looking at a kind of structure known as a Set. There is an interface Set<E>. For now, all of the sets we will construct all of our sets using the TreeSet<E> class. For example, I used an array of data to initialize both a list and a set by adding values from the array to each:
int[] data = {18, 4, 97, 3, 4, 18, 72, 4, 42, 42, -3}; List<Integer> numbers1 = new ArrayList<Integer>(); Set<Integer> numbers2 = new TreeSet<Integer>(); for (int n : data) { numbers1.add(n); numbers2.add(n); } System.out.println("numbers1 = " + numbers1); System.out.println("numbers2 = " + numbers2);This produced the following output:
numbers1 = [18, 4, 97, 3, 4, 18, 72, 4, 42, 42, -3] numbers2 = [-3, 3, 4, 18, 42, 72, 97]There are two major differences between a set and a list. Sets don't allow duplicates. So the duplicate values like 42 and 4 in the array appear just once in the set. Sets also don't allow the client to control the order of elements. The TreeSet class keeps things in sorted order. So the numbers will always be in that order. If you want to control the order, then you should use a list instead.
Sets have many of the same methods that lists do. You can add to a set, get its size, ask for an iterator, use it with a foreach loop. But it doesn't have a notion of indexing. So you can't remove at an index. Instead you remove a specific value. And you can't get at a specific index. Instead you use an iterator or a foreach loop.
We looked at this short example of using an iterator to remove the multiples of 3 from the set:
Iterator<Integer> i = numbers2.iterator(); while (i.hasNext()) { int next = i.next(); if (next % 3 == 0) { i.remove(); } } System.out.println("now numbers2 = " + numbers2);which produced the following output:
now numbers2 = [4, 97]This is the approach you need to take when you want to both examine and remove values from a set. Notice that we called the iterator's remove method and not the set's remove method. You are not allowed to alter a set while you are iterating over it. That means that you also can't alter it while performing a foreach loop because the foreach loop is implemented using an iterator. If you attempt to modify the structure while iterating over it, the iterator will throw a ConcurrentModificationException. In terms of the pharmacy analogy, you will end up confusing the guy behind the counter if you try to change what's in the pharmacy while he's iterating over it.
Then we turned to a new example. We didn't have time to do either of the normal 143 examples in detail, but I include here the full discussion as if we had.
I began by asking how we could write a program that would count the number of unique words in an input file. I had a copy of the text of Moby Dick that we looked at to think about this. I showed some starter code that constructs a Scanner object tied to a file:
import java.util.*; import java.io.*; public class WordCount { public static void main(String[] args) throws FileNotFoundException { Scanner console = new Scanner(System.in); System.out.print("What is the name of the text file? "); String fileName = console.nextLine(); Scanner input = new Scanner(new File(fileName)); while (input.hasNext()) { String next = input.next(); // process next } } }Notice that in the loop we use input.next() to read individual words and we have this in a while loop testing against input.hasNext().
So how do we count the words? Someone suggested that a set would be the perfect structure to solve this problem. It eliminates duplicates, so it will keep track of how many different words there are. So we changed the code to be:
Set<String> words = new TreeSet<String>(); while (input.hasNext()) { String next = input.next(); words.add(next); } System.out.println("Total words = " + words.size());Here is a sample log of execution:
What is the name of the text file? moby.txt Total words = 32019One limitation of this version is that it pays attention to capitalization. So it considers the words "whale" and "Whale" and "WHALE" to be different words. To fix that, we modified the code to read in a word so that it converts it to its lowercase equivalent:
String next = input.next().toLowerCase();It didn't make much difference, as we saw from this execution:
What is the name of the text file? moby.txt Total words = 30368Someone pointed out that we still haven't dealt with punctuation. It is considering "whale" and "whale." and "whale," to be different words. I didn't want to deal with it in this program, but I mentioned that the chapter 10 case study discusses this and shows you how to configure the Scanner so that it ignores those punctuation characters.
This program counts the number of unique words, but not the counts of the individual words. To keep track of word counts, we could use many different structures. For example, we could have a List of words and a List of counts where element "i" in one corresponds to element "i" in the other. This approach is often described as "parallel arrays." It's not a very object-oriented approach because we really want to associate the word with its counts rather than have a structure that puts all the words together and another that puts all the counts together. Or we could make a class for a word/count combination and then have a List of that. But Java gives us a better alternative. The collections framework provides a data abstraction known as a map.
The idea behind a map is that it keeps track of key/value pairs. In our case, we want to keep track of word/count pairs (what is the count for each different word). We often store data this way. For example, in the US we often use a person's social security number as a key to get information about them. I would expect that if I talked to the university registrar, they probably have the ability to look up students based on social security number to find their transcript.
In a map, there is only one value for any given key. If you look up a social security number and get three different student transcripts, that would be a problem. With the Java map objects, if you already have an entry in your map for a particular key, then any attempt to put a new key/value pair into the map will overwrite the old mapping.
We looked at an interface in the Java class libraries called Map that is a generic interface. That means that we have to supply type information. It's formal description is Map<K, V>. This is different from the List and Set interfaces in that it has two different types. That's because the map has to know what type of keys you have and what type of values you have. In our case, we have some words (Strings) that we want to associated with some counters (ints). We can't actually use type int because it is a primitive type, but we can use type Integer.
So our map would be of type Map<String, Integer>. In other words, it's a a map that keeps track of String/Integer pairs (this String goes to this Integer). Map is the name of the interface, but it's not an actual implementation. The implementation we will use is TreeMap. So we can construct a map called "count" to keep track of our counts by saying:
Map<String, Integer> count = new TreeMap<String, Integer>();The most basic methods in the map interface are the ones that allow you to put something into the map (an operation called put) and to ask the map for the current value of something (an operation called get).
I asked what code we need to record a word in our map the first time we see it. Someone suggested using the put method to assign it to a count of 1. So our loop becomes:
Map<String, Integer> count = new TreeMap<String, Integer>(); while (input.hasNext()) { String next = input.next().toLowerCase(); count.put(next, 1); }This doesn't quite work, but it's getting closer. Each time we encounter a word, it adds it to our map, associating it with a count of 1. This will figure out what the unique words are, but it won't have the right counts for them.
I asked people to think about what to do if a word has been seen before. In that case, we want to increase its count by 1. That means we have to get the old value of the count and add 1 to it:
count.get(next) + 1and make this the new value of the counter:
count.put(next, count.get(next) + 1);So we have two different calls on put. We want to call the first one when the word is first seen and call the second one if it's already been seen. Someone suggested using an if/else for this. The only question is what test to use. The Map includes a method called containsKey that tests whether or not a certain value is a key stored in the map. Using this method, we modified our code to be:
Map<String, Integer> count = new TreeMap<String, Integer>(); while (input.hasNext()) { String next = input.next().toLowerCase(); if (!count.containsKey(next)) { count.put(next, 1); } else { count.put(next, count.get(next) + 1); } }The first time we see a word, we call the put method and say that the map should associate the word with a count of 1. Later we call put again with a higher count. And we keep calling put every time the count goes up. What happens to the old values that we had put in the map previously? The way the map works, each key is associated with only one value. So when you call put a second or third time, you are wiping out the old association. The new key/value pair replaces the old key/value pair in the map.
Then we talked about how to print the results. Clearly we need to iterate over
the entries in the map. One way to do this is to request what is known as the
"key set". The key set is the set of all keys contained in the map. The Java
documentation says that it will be of type Set
In the final version that I posted on the calendar, I made one minor change.
There is an interface known as SortedMap. When you know that you want to
require a map to have its keys sorted, it is more appropriate to use that
interface. Every SortedMap is a Map, but not every Map is a SortedMap. So the
program changes the declaration to:
The sample program involves keeping track of friendships. You could think of
it as keeping track of Facebook friends. One of the first questions that comes
up is how do we represent friendships? For example, are friendships
bidirectional? If person A is friends with person B, does that mean that
person B is friends with person A? For our purposes, we will assume the answer
is yes. If we were trying to represent something like "is attracted to", then
we'd come to a different conclusion, but for friends, just like on Facebook and
other social networking sites, friendship goes both ways.
I said that a good way to visualize friendships is to draw a graph in which
each person is represented with a node (an oval) and each friendship is
represented by an edge connecting two nodes (a line drawn between two ovals).
I am using a program called Graphviz, which is an open-source graph viewer.. For example,
here is a sample friendship graph:
This information is stored in a file with lines that list pairs of friendships,
as in:
For example, here is a sample execution using our data file for finding the
connection between Stuart and Ashley:
Here is a sample execution where the connection is not found, asking for a
connection between Stuart and Bart:
We looked at one more example that involved a fairly long chain:
If we want a structure that keeps track of these kind of friendships, then we
want to use names as keys into the structure. We ask the structure, "Who are
the friends of Samantha?" or "Who are the friends of Ashley?". So a name, a
String, will be used as the key. But what should it return? If we map a
String to a String, then we can store only one friendship. We want to be able
to return more than one friendship. Someone suggested that we want to use a
set. That is exactly right.
The idea is that we want to have a map that converts a String into a Set of
String values. Given the name of a person, we can get a Set with the names of
that person's friends. For our sample file:
To fill up this structure, we need to process the input file. Remember that
the input file has lines that have two names separated by a "--", as in:
How do we
update our friends map given a new friendship? Friendships are bidirectional,
so we have to be careful to add the friendship in both directions. If there is
an Ashley--Chritopher friendship, then we have to make sure that Ashley's set
of friends includes Christopher and we have to make sure that
Christopher's set of friends includes Ashley.
I mentioned that this is a good place to introduce an extra method because
we're going to do the same thing twice. So we replaced the comment above with
the following two lines of code:
Although we can write the code in this way as two lines of
code, most programmers would write this as one line of code. There is no need
to introduce the local variable called names. So we can instead write this
as:
The very first time we see a name, we want to put it into the map. When we do
that, we want to associate it with a brand new set that can be used to store
the names of that person's friends:
This code constructs the friends map. The challenge then is to use it to
explore friends at various distances. To solve this problem, we will end up
using several sets of names. At any given time, we will be exploring a new set
of friends that are at the next distance away. We we will continue searching
until we either find the target name or run out of people to search. So the
overall structure of the method is as follows:
The solution is to introduce yet another set to keep track of people who have
already been explored. Then when we form a new set of friends to consider, we
remove the names of people who have already been explored. And we'll have to
add the new people to the set of explored people so that we won't explore them
in the future. The code below includes the extra lines of code indicated in
bold face:
for (String word : count.keySet()) {
// process word
}
We would read this as, "for each String word that is in count.keySet()..."
To process the word, we simply print it out along with its count. How do we
get its count? By calling the get method of the map:
for (String word : count.keySet()) {
System.out.println(count.get(word) + "\t" + word);
}
I didn't try to print all of the words in Moby Dick because it would
have produced too much output. Instead, I had it show me the counts of words
in the program itself. Obviously for large files we want some mechanism to
limit the output. On the calendar I will put a version that includes some
extra code that asks for a minimum frequency to use. We ran that on Moby
Dick and saw this list of words that occur at least 500 times:
What is the name of the text file? moby.txt
Minimum number of occurrences for printing? 500
4571 a
1354 all
587 an
6182 and
563 are
1701 as
1289 at
973 be
1691 but
1133 by
1522 for
1067 from
754 had
741 have
1686 he
552 him
2459 his
1746 i
3992 in
512 into
1555 is
1754 it
562 like
578 my
1073 not
506 now
6408 of
933 on
775 one
675 or
882 so
599 some
2729 that
14092 the
602 their
506 there
627 they
1239 this
4448 to
551 upon
1567 was
644 were
500 whale
552 when
547 which
1672 with
774 you
One final point I made about the Map interface is that you can associate
just about anything with just about anything. In the word counting program, we
associated strings with integers. You could also associate strings with
strings. One thing you can't do is to have multiple associations in a single
map. For example, if you decide to associate strings with strings, then any
given string can be associated with just a single string. But there's no
reason that you can't have the second value be structured in some way. You can
associate strings with arrays or strings with Lists.
SortedMap<String, Integer> count = new TreeMap<String, Integer>();
Then I mentioned that I wanted to explore a sample program that will constitute
a medium hint for the programming assignment. We will begin looking at the
program in this lecture and finish it up in the next lecture.
graph {
Ashley -- Christopher
Ashley -- Emily
Ashley -- Joshua
Bart -- Lisa
Bart -- Matthew
Christopher -- Andrew
Emily -- Joshua
Jacob -- Christopher
Jessica -- Ashley
JorEl -- Zod
KalEl -- JorEl
Kyle -- Lex
Kyle -- Zod
Lisa -- Marge
Matthew -- Lisa
Michael -- Christopher
Michael -- Joshua
Michael -- Jessica
Samantha -- Matthew
Samantha -- Tyler
Sarah -- Andrew
Sarah -- Christopher
Sarah -- Emily
Tyler -- Kyle
Stuart -- Jacob
}
Then I demonstrated what the Friends program is supposed to do. It is supposed
to use this data to find how far one person is from another. So starting with
a given person, it finds that person's friends, then the friends of those
friends, then the friends of the friends of the friends, and so on. It reports
how far it has to go to find a connection and if it runs out of people, it
simply reports that the connection couldn't be found.
Welcome to the cse143 friend finder.
starting name? Stuart
target name? Ashley
Starting with Stuart
1 away: [Jacob]
2 away: [Christopher]
3 away: [Andrew, Ashley, Michael, Sarah]
found at a distance of 3
It finds that Stuart has one friend (Jacob). And that friend has one friend
(Christopher). And he has 4 friends, including Ashley. So the program reports
that it found Ashley is 3 away.
Welcome to the cse143 friend finder.
starting name? Stuart
target name? Bart
Starting with Stuart
1 away: [Jacob]
2 away: [Christopher]
3 away: [Andrew, Ashley, Michael, Sarah]
4 away: [Emily, Jessica, Joshua]
5 away: []
not found
The program goes two levels farther than it did before, finding that it runs
out of people when it gets 5 away from Stuart. At that point it knows that
there is no connection between Stuart and Bart.
Welcome to the cse143 friend finder.
starting name? Bart
target name? JorEl
Starting with Bart
1 away: [Lisa, Matthew]
2 away: [Marge, Samantha]
3 away: [Tyler]
4 away: [Kyle]
5 away: [Lex, Zod]
6 away: [JorEl]
found at a distance of 6
I asked people what kind of structure would be useful for keeping track of this
kind of data and someone said a map. But what kind of map? Someone suggested
that it would be good to keep track of the neighbors for each person. The
neighbors are the friends. For example, Ashley's friends are Christopher,
Emily, Jessica, and Joshua.
"Andrew" => maps to => [Christopher, Sarah]
"Andrew" => maps to => [Christopher, Sarah]
"Ashley" => maps to => [Christopher, Emily, Jessica, Joshua]
"Bart" => maps to => [Lisa, Matthew]
"Christopher" => maps to => [Andrew, Ashley, Jacob, Michael, Sarah]
"Emily" => maps to => [Ashley, Joshua, Sarah]
"Jacob" => maps to => [Christopher, Stuart]
"Jessica" => maps to => [Ashley, Michael]
"JorEl" => maps to => [KalEl, Zod]
"Joshua" => maps to => [Ashley, Emily, Michael]
"KalEl" => maps to => [JorEl]
"Kyle" => maps to => [Lex, Tyler, Zod]
"Lex" => maps to => [Kyle]
"Lisa" => maps to => [Bart, Marge, Matthew]
"Marge" => maps to => [Lisa]
"Matthew" => maps to => [Bart, Lisa, Samantha]
"Michael" => maps to => [Christopher, Jessica, Joshua]
"Samantha" => maps to => [Matthew, Tyler]
"Sarah" => maps to => [Andrew, Christopher, Emily]
"Stuart" => maps to => [Jacob]
"Tyler" => maps to => [Kyle, Samantha]
"Zod" => maps to => [JorEl, Kyle]
Our first challenge, then, is to write code to construct such a structure. If
it maps a String to a Set<String>, then it would be of this type:
Map<String, Set<String>>
To construct one, we have to ask for a new TreeSet of this type:
Map<String, Set<String>> friends = new TreeMap<String, Set<String>>();
That is a rather complex line of code, but the main complexity comes from what
we are putting inside the "<" and ">" characters.
Ashley -- Christopher
I showed the following code to read lines of input and find the ones that
contain names:
while (input.hasNextLine()) {
String line = input.nextLine();
if (line.contains("--")) {
Scanner lineData = new Scanner(line);
String name1 = lineData.next();
lineData.next(); // this skips the "--" token
String name2 = lineData.next();
// process name1 and name2
}
}
This was not the interesting part of the code because we saw file processing in
cse142. The interesting part is to think of how to process the two names. How
do we update our friends map given a new friendship? Friendships are
bidirectional, so we have to be careful to add the friendship in both
directions. If there is an Ashley--Christopher friendship, then we have to
make sure that Ashley's set of friends includes Christopher and we have
to make sure that Christopher's set of friends includes Ashley.
addTo(friends, name1, name2);
addTo(friends, name2, name1);
So then we turned to the task of writing the addTo method. It takes the map
and the two names as parameters, so it looks like this:
public static void addTo(Map<String, Set<String>> friends, String name1,
String name2) {
...
}
If we're trying to add name2 to the set for name1, then in general we want
to:
get the set for name1
add name2 to that set
Here is a first attempt:
Set<String> names = friends.get(name1);
names.add(name2);
This is a good start. Remember that the whole point of the map is to associate
a name with a set of names. So in the first line of code we ask the map to
give us the set of names associated with name1. In the second line, we add to
that set name2.
friends.get(name1).add(name2);
But there is a problem with this approach. It assumes that there is a set of
names associated with name1. Initially the map is empty. And if we call get
for a key that is not in the map, then we get the value null back. That would
cause a NullPointerException if we tried to treat it as a set that we can add
something to.
friends.put(name1, new TreeSet<String>());
But we only want to do this once. For example, if we did this every time we
went to add a friendship for this person, then we would always have a set with
just one name in it. The first time we see name1, we want to make this set.
Then every other time we simply want to add a new name to the existing set. So
we need to include a test that constructs the set only the first time we see
name1:
if (!friends.containsKey(name1)) {
friends.put(name1, new TreeSet<String>());
}
friends.get(name1).add(name2);
This is the complete code for the addTo method. It constructs a new set each
time it sees a name for the first time. And every time it executes, it adds
name2 to the set for name1.
Set<String> newFriends = new TreeSet<String>();
newFriends.add(name1);
int distance = 0;
while (!newFriends.contains(name2) && !newFriends.isEmpty()) {
distance++;
// find friends one further away
}
Inside the loop, we want to use the current set of newFriends to find the next
group of newFriends. We can do so simply by adding all of the friends of these
friends to a new set and then replacing newFriends with that new set:
Set<String> newNewFriends = new TreeSet<String>();
for (String friend : newFriends) {
newNewFriends.addAll(friends.get(friend));
}
newFriends = newNewFriends;
This provides a pretty good solution to the problem. If we throw in some
statements to print out what is happening, we end up with this solution:
Set<String> newFriends = new TreeSet<String>();
newFriends.add(name1);
int distance = 0;
System.out.println();
System.out.println("Starting with " + name1);
while (!newFriends.contains(name2) && !newFriends.isEmpty()) {
distance++;
Set<String> newNewFriends = new TreeSet<String>();
for (String friend : newFriends) {
newNewFriends.addAll(friends.get(friend));
}
newFriends = newNewFriends;
System.out.println(" " + distance + " away: " + newFriends);
}
if (newFriends.contains(name2)) {
System.out.println("found at a distance of " + distance);
} else {
System.out.println("not found");
}
But notice what happens when we run this version of the program:
Welcome to the cse143 friend finder.
starting name? Stuart
target name? Joshua
Starting with Stuart
1 away: [Jacob]
2 away: [Christopher, Stuart]
3 away: [Andrew, Ashley, Jacob, Michael, Sarah]
4 away: [Andrew, Christopher, Emily, Jessica, Joshua, Sarah, Stuart]
found at a distance of 4
It is getting the right answer, but the intermediate answers are not correct.
It indicates, for example, that Stuart is 2 away from Stuart. That's because
it is including the possibility of going from Stuart to Jacob and then from
Jacob back to Stuart. In a similar way, it is saying that Christopher is 2
away and Christopher is 4 away. In this case it came up with the right answer,
but allowing this kind of duplication makes the program run more slowly and it
leads to an infinite loop when there is no connection between people. That's
because when you allow duplicates, it just keeps finding more and more friends
when it looks 5 away, 6 away, 7 away, and so on.
Set<String> oldFriends = new TreeSet<String>();
Set<String> newFriends = new TreeSet<String>();
newFriends.add(name1);
int distance = 0;
System.out.println();
System.out.println("Starting with " + name1);
while (!newFriends.contains(name2) && !newFriends.isEmpty()) {
distance++;
oldFriends.addAll(newFriends);
Set<String> newNewFriends = new TreeSet<String>();
for (String friend : newFriends) {
newNewFriends.addAll(friends.get(friend));
}
newNewFriends.removeAll(oldFriends);
newFriends = newNewFriends;
System.out.println(" " + distance + " away: " + newFriends);
}
if (newFriends.contains(name2)) {
System.out.println("found at a distance of " + distance);
} else {
System.out.println("not found");
}
This completes the program.
Stuart Reges
Last modified: Sat Nov 3 08:41:02 PDT 2012