Project 3 full - File System Implementation

Out: Friday, May 29th
Due: Friday, June 5th, at 11:59pm. The submission is entirely electronic.

Overview

The starting point for this assignment is a simplified file system, cse451fs, the design of which imposes strict limits on both the number of files that can be stored and the maximum size of any one file. In particular, no matter how big a disk you might have, this file system can hold only about 8,000 distinct files, no file can be larger than 10KB, and file names cannot be longer than 30 characters.  These restrictions result from the choice of on-disk data structures used to find files and the data blocks of a given file, that is, the superblock, inode, and directory entry representations.

Here are the major steps involved in this part of the assignment:

1. Make sure that you, individually, understand the mechanical aspects of the "development environment" you'll be working in. These include how to build the file system code, how to format and mount the partition we're devoting to your file system, and how to run and test. A description of these mechanical aspects can be found here.  

2. Improve each of the three limitations cited above:
   
   • Increase the maximum size of files (10 points).
   • Allow for longer file names (10 points).
   • Extra credit: allow for more files (5 points).
   Design how you want to implement these file system modifications on disk: how you will represent your directories on disk, how file data is indexed, etc. There can still be a limit on any of these properties, but your improvement needs to be more than simply altering a program constant. For full credit, you must support a maximum file size of at least 256KB, which can be achieved with a single-indirect strategy as discussed in lecture.

   If you encounter any inexplicable behavior while testing your file system, keep in mind that it might be caused by a limitation of the testing program and not your file system. For example, ls only supports 256 character file names, so if you create a longer name with your file system, ls will not show it properly. In general, a good sanity check is to make sure your tests work on regular file systems (e.g., on the parts of the VMWare machine's file system that is not running your file system, or on forkbomb) before running them on your cse451fs.

3. Alter the skeleton code to implement your file system. There are two major components to this. One is that the user level program mk451fs must be changed to initialize the raw disk device with a valid, empty file system using your new on-disk data structures. The other is to change the file system source itself.

Details

• While real file systems are very concerned with performance, in your implementation you can largely ignore it. That is, do not spend a great deal of effort to produce a faster implementation. (For one thing, because we're running on virtual machines on top of Windows, it's unlikely you'd be able to measure much difference.)
• A description of the skeleton version of the cse451fs file system is here.
• A description of the ext2 file system and vfs (Virtual File System) is here. Note that vfs, in particular, is fluid, and the current implementation may have evolved beyond this documentation.
• Potentially more up-to-date documentation is distributed with the linux source. For vfs, have a look at linux.../Documentation/filesystems/vfs.txt.
• A description of how dynamically loaded modules are handled in Linux is here.

Hints/Starting Points

Large Files:

• An important function for creating and accessing the blocks of a file (that you will almost certainly need to modify) is get_block() in super.c.
• Look at cse451_truncate() in file.c in order to handle file truncations (e.g., deleting a file uses this).  You don't need to worry about this until you are sure that you can create and access your files.

Long File Names:

• All the functions that you will need to modify for your kernel module are in dir.c.
• Start with cse451_add_entry() and cse451_readdir() to create and read directory entries with long file names.  If listing with ls doesn't appear correct, check how the directory structure looks on disk (using hexdump or xxd).

General:

• If you modify any of the data structures in cse451fs.h, you will probably need to modify mk451fs accordingly.
• As well as formating (initializing) a cse451fs partition, mk451fs can be used to dump readable forms of the raw data on the disk, as a debugging aid. Try ./mk451fs --help for more information. (Note that because vfs performs delayed write-backs of dirty blocks, and mk451fs is going around the file system (directly accessing the hardware device), there can be periods of up to a few seconds when you might see un-updated blocks using the mk451fs dump facilities.)
• You might find the following UNIX utilities useful in your testing: dd, df, du, diff, head/tail, hexdump.

Writeup

Please turn in a file called writeup.txt or writeup.htm that addresses the following:

1. Describe the design for your file system modifications.  Include enough details so that we can understand your code based on this description. This part might also include a discussion of other approaches you considered but rejected.
2. What concurrency-related issues does a file system have to deal with? You probably didn't deal with any of it directly when implementing your extentions, but what did you notice when looking at the rest of the code?
3. What methodology did you follow in order to test your file system (for functionality)?
4. Does your implementation work? If not, what parts work and what parts don't? How would you fix it if you had more time?
5. What do you like best about your design?  What do you like least about it?  How would you improve your design?

Aim to fit this report into about 2 pages. The file you submit should be in some easy-to-read format, e.g., plain (ASCII) text or HTML or pdf. It should contain the names of all the people who worked on the project as well as your group name.

Your writeup will be turned in electronically.

Turnin

You will be turning in 4 files:

• A tar.gz file containing all of your modified sources
• Your compiled mk451fs program
• Your compiled fs module, cse451fs.ko
• A single write-up file, writeup

You must compile your mk451fs and cse451fs.ko files on forkbomb. Do not use a different machine or compiler for this.

To create the source archive file, do a "/cse451/projects/makedist" from your top level project directory.

Use the turnin(1L) program under project name project3. Note: turnin will not work on forkbomb, so you'll need to use attu.

turnin -c cse451 -v -p project3 roxana-project3

where roxana-project3 is the directory containing all the files for the project.