Assignment Overview

This project is about layering, framing, and encoding of data. The class material and text have addressed these, but mostly at the physical and link layers. We'll be looking at the some of the same issues, but at the application layer.

Starting with this project, we're implementing a set of network layers that make building networked applications easier. (After that, we'll build a network application, or two.) We could begin with either UDP or TCP, but because we're going to want better reliability characteristics than UDP provides, we choose TCP as the starting point. This project addresses two shortcomings of TCP, relative to our eventual needs:

1. TCP doesn't provide any kind of framing.
2. TCP sends a byte stream, but provides no help to the receiver understanding what the bytes represent

We fix the former by implementing a new networking layer (TCPMessageHandler) that defines a simple frame format. We fix the latter by adopting a convention at the application level that the payload is represented using JSON (Javascript Object Notation), a widely adopted data interchange format. The results in some data transfers that look like this:

length

JSON encoded payload

Each frame begins with a length field, giving the length of the payload, which is often a single application layer message encoded in JSON. (We provide methods for a client to send byte arrays, Strings, and other types as the payload; however, for most applications we will write, the payload will consist of JSON objects.) This format is described in more detail below in TCPMessageHandler.java.

When you're done, your network stack will look like this:

An application can choose to use TCP as a stream, bypassing the new layer, or it can choose to use TCPMessageHandler to create messages on top of TCP. In fact, an application could mix both modes of communication, if it wanted.

Note that TCPMessageHandler is a networking layer, not part of any application. Thus, its design and implementation must be independent of the specific applications we build in this project sequence.

Application Functionality Overview

We'll re-implement essentially the same applications we implemented in Project 1. What you should be watching for in this project, and subsequent ones, is how much easier it gets to write and use those two applications. In terms of performance, we hope that the added convenience doesn't make things run much slower.

ping

We reimplement ping to send messages in the new, framed format. We use EchoTCPMessageHandlerService, an implementation of Echo that understands the framed format, as the server. First, as in Project 1, we send a header message which establishes which service to invoke. This will ordinarily be the String defined in EchoServiceBase, but other headers must be supported for testing code. Because we're using messages, it's now possible for the client to send a length 0 message to the echo service, so the second message it sends is a length 0 message to be echoed. The response is a similar two-message reply: first, the response header, followed by the length 0 echoed message. This is described in more detail in PingTCPMeesageHandler.java below.

In this diagram, "<headerStr>" should be replaced by the header String that is passed into the ping method. Note that all of the TCPMessageHandler messages sent in the case have Strings as payloads (represented by green messages).

From the user's perspective, the only change is that the Project 2 ping runs on top of TCPMessageHandler (so, TCP) only; we no longer use or report results for UDP.

dataxfer

In Project 1, the number of characters transferred from the dataxfer server to the client was determined by the port to which the client connected. That's an unrealistic mechanism. In Project 2, we allow the client to specify the amount of data to send.

First, as in project 1, we send a header indicating the service we want to run (ordinarily, this will be as defined in DataXferServiceBase, but the interface is flexible as to the header message sent.

Next, we send a control message to the server. The control message contains a JSON encoded payload, in particular, a single JSONObject with one key-value pair. The key is transferSize, and the value is an integer representing the number of bytes to transfer. Having received that control message, the server returns transferSize bytes to the client, sending these as messages containing no more than 1000 bytes. Here's a picture of the protocol when 2500 bytes are transferred:

All data is sent as messages (i.e., using TCPMessageHandler frames.) Only the control message (in blue) carries a JSON encoded payload; the headers are Strings (green) and the data messages (black) carry raw bytes. Once again, "<headerStr>" will be replaced by the header String that is passed in to DataXfer.

(ConsoleApps) Implementation Details and Requirements

Here's a summary of the files you'll work on.

Source File / Interface File	Eclipse Project
TCPMessageHandler.java / TCPMessageHandlerInterface.java	Net (edu.uw.cs.cse461.Net)
PingTCPMessageHandler.java / PingInterface.java	ConsoleApps (edu.uw.cs.cse461.consoleapps.solution)
DataXferTCPMessageHandler.java / DataXferInterface.java	ConsoleApps (edu.uw.cs.cse461.consoleapps.solution)
DataXferTCPMessageHandlerService.java / None	Service (edu.uw.cs.cse461.service)
client.config.ini, server.config.ini / None	ConfigFiles

You should create files PingTCPMessageHandler.java, DataXferTCPMessageHandler.java, and DataXferTCPMessageHandlerService.java.

TCPMessageHandler.java

This class handles sending messages over a TCP stream. Its interface allows users to pass in data of various types. The data is converted to a byte[] payload, prefixed by the length of the byte[], and then sent over the underlying TCP socket:

length

payload

The receiver reads the length field, then the payload, and then converts the byte[] it has read into the type requested by the client as part of the read call.

The integer length prefix is sent in binary, as four bytes in little endian order.

All required methods of TCPMessageHandler are defined in the skeleton code, but some are missing implementations. Note that the methods for writing and reading the integer length field are inverses of each other: if you encode an integer using one and then decode it using the other, you should get the original integer back. The same relationship holds between send and read routines for a particular data type: the send routines convert into byte[], and the read convert from byte[] back to the original data type.

To interoperate, all our implementations must agree on how to translate between the types supported by the interface and byte[]. The schemes used are these. A String is converted to a byte[] using String.getBytes(). A JSONObject or JSONArray is converted to byte[] by first converting to a String and then applying the String transformation.

Finally, we're building code that uses the network, and so security should come to mind. In writing our projects, we're not worried about malicious remote users (those whose goal is to interfere with our goals), mainly because there's little incentive in that relative to the amount of work required. On the other hand, we are worried about what effect simple programming errors in code we're talking to might have on our code, in part because we're likely to talk to buggy code as we develop. For this project, you should worry about bad length prefixes. The length value might simply be preposterous (e.g., negative, or enormous). TCPMessageHand's interface provides a method to give the client some control over what "preposterous" means. Alternatively, the length field might be reasonable, but wrong. You should continue to guard against blocking forever on a read by always setting timeouts on sockets. The code relies on the creator of the Socket to set the timeout. The creator is not TCPMessageHandler (the socket is passed in as an argument to TCPMessageHandler's constructor), so the timeout is set in some other code. However, you should be sure it is always set. (You don't have to try to verify in your TCPMessageHandler code that there is a timeout. In most cases it's a bug in the client code if it hasn't been sent, but it's a decision the application makes. Your code should be prepared for timeouts, however.)

PingTCPMessageHandler.java

This is Ping console client code. It should send two messages to EchoTCPMessageHandlerService:

1. A header message to establish the 'protocol'
2. A length 0 message to be echoed

It should measure the elapsed time to receive a reply, which will also consist of two messages:

1. A header message confirming the protocol (see EchoServiceBase)
2. The echoed length 0 message

The is almost identical to your Project 1 ping client code, including the fact that you need to be flexible in the headers that you accept from the calling code (as they are passed as an argument to ping) but may be restrictive in the responses you accept (they should be as defined by EchoServiceBase). It is different from Project 1's ping in that we no longer use UDP, and you're sending TCPMessageHandler messages rather than sending a TCP byte stream.

Remember to keep separate the notion of 'headers' internal to the TCPMessageHandler protocol, which encode the length of the upcoming payload, and the 'headers' that you are sending as a part of the ping and dataXfer applications, which are full TCPMessageHandler messages themselves.

You need to create this file. Note that though the interface file is PingInterface, you need to implement the PingTCPMessageHandlerInterface contained within.

DataXferTCPMessageHandler.java

This is the data transfer application client. It too is very like its Project 1 cousin. The main change is the one described above: after sending the protocol header message, the client sends a control message to the server indicating how many bytes it would like to transfer, rather than that being determined by the port the client connects to. The response includes a header message followed by several messages containing the requested bytes as payload.

You need to create this file.

DataXferTCPMessageHandlerService.java

The server side of the message based data transfer application. It should read the header message and control message sent by the client, then respond by sending its own header message followed by the amount of data the client requested as a sequence of one or more messages.

You need to create this file.

*.config.ini

You're creating new applications and services. These must be entered into the corresponding lists of components to be loaded that are given near the top of the config file: net.services for services, and console.apps for apps. This also goes for the new testing apps, which is described below in the testing section.

Testing

Testing jar
There is an updated Tester.jar test library, which is available from /cse/courses/cse461/13wi/Tester.jar. You should copy this into your Lib/ directory to replace the old Tester.jar. If you are unsure of which jar you are dealing with simply run the jar with java -jar Tester.jar and it will print its version number. The version for this project is: 1.1.4 2/11/2013.

Because we have new testing apps, you'll need to update the config files. To your client.config.ini file, you'll want to add the following to test.driver.console.apps:

• edu.uw.cs.cse461.consoleapps.grading.PingTCPMessageHandlerTester
• edu.uw.cs.cse461.consoleapps.grading.DataXferTCPMessageHandlerTester

(Your server.config.ini file should already have the necessary services listed.)

Note that, again, the tests are sanity checks rather than comprehensive evaluations, and that testing your service is up to you.

Solution jar
There is also an updated solution jar, which is available from /cse/courses/cse461/13wi/461solution/461solutionP2.jar. You can run this exactly the way you ran the solution jar for Project 1.

What to Hand In

Submit the code files you created or modified, following the same scheme as in Project 1 (described in section "What to Turn In" in the Project 1 assignment page).