Using Linux in 351
Overview
This course explores low-level topics in machine representation of programs and data. For our reference system, we're using Rocky Linux. Other Linux and Mac OS X systems are very similar, and Windows machines also use the same underlying processors and memory organization. But for assignments in this course, your submissions need to work on, and will be graded on, our reference system.
This is a quick little tutorial on how to get yourself up and running on Linux. If you are a CSE major, then you have access to departmental servers collectively called attu (actually reached at attu.cs.washington.edu). If you are not a CSE major, then you have access to a similar server called calgary (actually reached at calgary.cs.washington.edu), which is running the same Linux image.
Terminal and Shell Access
Terminology:
- A shell is a program that processes (textual) commands through the operating system and returns the output. This is the piece that "does the work".
- A terminal is a wrapper program that runs a shell. It is primarily responsible for handling the user text input and displaying the output from the shell. This is one example of a command-line interface (CLI).
We use a terminal (e.g., Terminal) to connect to one of many shells (e.g., bash) installed on a Linux machine. You may hear these terms used somewhat interchangeably even though there is a difference; the distinction is irrelevant in 351. The shell is the bread and butter of Linux and brings you a lot of power with not very many key presses.
Terminal on Windows
using any of the following methods:
- PowerShell: Windows 7 and up come installed with
PowerShell, which is a hybrid between Command Prompt and a Linux
terminal.
It does, however, recognize the
ssh
andscp
commands, which is all that you will need for this course. - bash: If you are running a 64-bit version of Windows 10 and have installed the "Anniversary Update" (released Aug. 2016) then you can enable a bash terminal (command-line interface) directly within Windows! Newer machines can directly from the Microsoft store, otherwise, here is one useful link for installing bash.
- PuTTY: A lightweight SSH client that lets you interact with another computer over the Internet (like opening a shell). You can directly download putty.exe onto your computer.
Terminal on Mac OS X
Mac OS X comes with a pre-installed terminal (aptly called Terminal) with which you can :
- Click the Spotlight Search button in the upper-right, type "terminal", and press return.
Terminal on Linux
Linux comes with a pre-installed terminal (aptly called Terminal):
- Click Applications at the top, mouse to Accessories, and click Terminal.
If you are using one of the CSE lab machines, then you can directly use the Terminal window. If you are using your own Linux machine, it is still recommended that you to ensure that you have all of the tools needed and are using the same environment as everyone else.
Logging onto attu or calgary
attu and calgary are powerful clusters of computers sitting in the CSE building that are ready to run your work, except that they have no screens! They run Linux and can only be accessed over the network. But they have all of the commands that you will need for the 351 labs and are built to be fast, so more often than not it's most convenient to do your work directly there.
calgary is accessed via UW NetID login (i.e., the netid and password that you would use for ).
From Windows (PuTTY)
- Open
putty.exe
from wherever you downloaded it. - For the host name, type either
<your-csenetid>@attu.cs.washington.edu
or<your-uwnetid>@calgary.cs.washington.edu
. - Click Open at the bottom.
- If you're prompted about an RSA key, click Yes.
- Put in your CSENetID (for attu) or UWNetID (for calgary) password when asked for a password. It will look like you're not typing anything – that's done on purpose so people looking over your shoulder won't know your password.
From Windows (bash), OS X, or Linux
- Open the terminal on your computer (see above).
- Type either
ssh <your-csenetid>@attu.cs.washington.edu
orssh <your-uwnetid>@calgary.cs.washington.edu
at the command prompt and press [Enter]. - If prompted about a server key, type
yes
and press [Enter]. - When prompted, enter your CSENetID (for attu) or UWNetID (for calgary) password and press [Enter]. It will look like you're not typing anything – that's done on purpose so people looking over your shoulder won't know your password.
Troubleshooting your connection
Computers are prone to fail every now and then, especially servers since they are constantly available and often handling multiple connected users at a time. Throughout the quarter, you will likely encounter some instances where you are rejected by attu or calgary or your files are missing when you do connect. This is likely because at least one of the servers is having a problem/issue and you were unfortunately routed to it. The quickest solution is to try to connect to a different individual cluster machine.
- For attu, append a number from 1-8 after "attu"
(e.g.,
ssh <netid>@attu1.cs.washington.edu
. - For calgary, contact the course staff or email support@cs.washington.edu directly.
Changing your password
If you would like to change your password, type passwd
into the command prompt once you're logged in and press [Enter], then
follow the instructions given.
Shell Basics
Linux Files and Directories
Terminology:
- A directory is a file system organizational tool for grouping (related) files, including subdirectories, together.
- A is a fancy word that Microsoft invented to make computers seem more human and refers to the grouping of files in the graphical user interface, which may not correspond to the underlying file system structure.
- A path is a textual reference to a location in a
directory system (e.g.,
/homes/iws/jhsia/myfile.txt
).
"Directory" and "folder" are generally used interchangeably, however, "directory" is more appropriate here since we are dealing with the file system via the command line.
All of the files we will create, edit, and use in this class are
stored in a directory (of your choosing) in the Linux file system.
The most important part for 351 is understanding the directory
structure layout so that you can properly locate your files.
File paths will be more important in future courses for specifying
file locations within your code or in shell commands.
Do note that paths in Linux are case-sensitive
(e.g., /home/test.txt
is not the same as
/home/Test.txt
).
The root directory (/
)
The Linux file system consists of a single, hierarchical directory
structure that starts from the root directory, represented by
'/
', and then expands into subdirectories.
Where DOS/Windows had various drives (e.g., C:
,
D:
) with directories under those partitions, Linux
places all the drives under the root directory by 'mounting' them
under specific directories.
The closest parallel to the root directory in Windows would probably
be C:
.
Note that Linux uses the frontslash '/
' as the directory
separator in paths instead of the backslash '\
' as in
Windows/DOS.
So C:\windows\system
in Windows would be
/c/windows/system
in Linux.
Your home directory (~
)
To keep people (users) from stepping on each other's toes, everyone
has one personal directory that they can freely write and maker
changes to.
This is called your "home directory."
On any Unix-like system, you can refer to the home directory with a
tilde, so a directory named frogs
inside of your home
directory could be referenced via the path: ~/frogs/
.
When you run a command, your shell will replace ~
with
the full path to your home directory.
If you want to know what that is or how to manipulate the shell,
read the next section on .
Other important directories (.
and ..
)
The shell also allows you to use/create paths relative to your current working directory.
- '
.
' represents the current/present working directory. - '
..
' refers to the direct parent of the current/present working directory (i.e., the directory that contains the current working directory). - Every additional '
.
' refers to the directory one level higher.
These are very convenient in working your way around your directory
structure.
For example, if you are in the folder lab1
, a sibling
directory lab2
would be referred to as:
../lab2
.
Basic Unix Commands
To move around through the directory structure in your terminal, you'll need to know a few basic Unix commands. Note that you always start in your home directory when you first open a terminal or SSH into a server.
man
and info
commands.
For example, if you want to know how to use ls
, pass the
man ls
or info ls
commands to your shell.
Command | Function | Example | Explanation | Notes |
---|---|---|---|---|
mkdir |
Creates a new directory with the given name in the current working directory. | mkdir lab1 |
This will create a new directory called "lab1 ". |
|
ls |
Lists all directories and files in the current directory. | ls -A |
This will list all sub-directories and files. The -A flag means that hidden directories and files will also be printed to the console. | Check the manual page for ls to find out various flags to show directories and files in different forms. |
cd |
Navigates to the specified directory, given its relative path. | cd lab1 |
This will navigate to the lab1 directory inside the current directory. |
This is a common place where . and .. are used.Also, to use a directory's absolute path, start the directory name with either / or ~ .For example: cd ~/cse351/lab2 |
pwd |
Prints the current working directory path. | pwd |
This will print the current working directory's absolute path to the console. | |
exit |
Exits the console, or logs out of the current SSH session. | exit |
This will terminate the current terminal window. If you are SSH'd into attu, this will terminate your session. |
Below is a sample execution of some Unix commands.
The [attu]$
is just the prompt telling you that you're
logged into attu and will differ on your terminal.
Lines that don't start with this prompt are output returned by the
shell in response to the previous command.
[attu]$ mkdir mydir [attu]$ ls mail mydir [attu]$ cd mydir [attu]$ pwd /homes/iws/jhsia/mydir [attu]$ exit
Transferring Files Remotely
You will frequently want/need to transfer files from one computer to another (e.g., between your local machine and attu). Some use cases include downloading files from attu to your local machine to submit to Gradescope or getting a copy of a starter file from the course website.
File transfer
The following are suggested methods for transferring files between your local machine and attu (i.e., either from your machine to attu or from attu to your machine):
scp
: A shell command of the formscp <source> <destination>
used in your terminal. A quick overview can be found on .-
Your local shell and the attu shell can likely both run this command, so it's important to remember which one you're currently using/viewing in your terminal. It is recommended that you always use
scp
with your local shell (i.e., disconnect from attu first) to avoid differences between file systems.
-
- :
GUI version of
scp
that allows for drag-and-drop and limited remote file system manipulation (e.g., file renaming, file deletion, file permissions, directory creation). Works on all platforms (Windows, MacOS, Linux). - :
GUI version of
scp
that allows for drag-and-drop and limited remote file system manipulation (e.g., file renaming, file deletion, file permissions, directory creation). Windows only!
File download
If the source file is accessible via a URL (e.g., files
hosted on the 351 course website), then you can use the shell command
wget <URL>
to download the file to your
current/present working directory.
More information can be found in the
.
Changing your Shell
There are different shell programs, which are all similar but have
different rules and features.
For sake of uniformity, we will use a shell called "bash,"
and your account should use this by default.
You can check using the following command: echo $0
.
In case the output isn't bash
or -bash
(or you're updating your personal Linux machine), you can change the
shell temporarily or permanently but we strongly
recommend the latter.
Temporarily
Issue the command bash
.
Now you may have a different-looking prompt (such as
[bash-3.00]$
), but otherwise at this point you will not
notice any differences.
Note that instead of changing your shell, you've actually
opened up a new shell (we'll discuss processes in this
course), so when you issue the exit
command, you'll
actually return to the original shell (i.e., the one you
typed bash
into).
Permanently
Issue the command chsh -s /bin/bash
.
This runs the "change shell" program
and tells the operating system that the "first shell" for your
account for every terminal should be the one found at
/bin/bash
.
- To be sure you have the right to do this, you'll be prompted for your password.
- Before that, you'll be warned that typing passwords can be dangerous. Say "y".
- As the message says, the change may take a few hours. In the meantime, see the "Temporarily" instructions above.
If your shell already is bash, chsh
will just
say "Shell not changed."
Editing
Once you get comfortable accessing a Linux shell and navigating files, you're going to need the ability to view and edit files, which is where come in. While we are generally most comfortable using text editors with nice graphical user interfaces (e.g., Microsoft Word) or integrated development environments (e.g., jGRASP), there is a lot of overhead involved in sending the graphics data over the Internet when you are connected to a remote server. So instead, we would encourage you to try out one of the two most popular editors: Emacs or vim.
There is a TON of debate between these two editors (see , , and ). Most of the course staff are more familiar with vim, but don't let that dissuade you from trying Emacs if you'd prefer to. Both editors have a steep learning curve and are greatly customizable, so we've provided an entirely separate to help you get started.
Compiling
We will be using a C
called gcc
in this course, so once you've made edits to
your source file and want to test your program, you will need to run
a command to produce the executable that will run on your computer
(or other desired build product).
For the most part, the exact compilation command that you will need to use will be given to you in the assignment specs, so you can just copy-and-paste it into your terminal. We'll learn about some of the different options available for the compilation command in this course, and more is covered in the follow-on course, CSE 333.
Makefiles
To make compilation even simpler for you, many of the lab assignments
provide you with a
,
which contains the correct compilation command so you never have to
type it out yourself!
You do NOT need to know how these work or how to read these files,
but those who are curious should take the follow-on course, CSE 333.
All you need to know for 351 is that you only need to run the command
make
while you are in the directory that contains that
lab's Makefile
file and the compilation will be done
for you!
make
, first run
make clean
before make
again.
For the curious, the likely culprit in these situations is that
make
and gcc
have cached computations
required to build your software which have either become corrupt or
interfere with the new changes you've made.
The make clean
command will remove intermediate build
products so that the whole compilation process starts over from
scratch.
Executing
We're not using a graphical interface, so there's no option to double-click an executable to run it anymore; everything must be specified in the command line!
All of the are the names of executables that come pre-installed with either the operating system or the shell that you are using. The trick is that your shell has a list of places that it knows to look for an exectuble that matches the name of the command that you gave it. However, the shell won't know about an executable that you've built yourself unless you explicitly tell it where to look. In Linux, what this means is that in order to run your locally compiled executable, you will need to specify a path to the executable instead of just using the name of the executable.
For example, if you are remotely connected to attu,
currently in your home directory, and have compiled an executable
named hello
there that outputs the text
"Hello, world!", the following transcript shows:
(1) an incorrect call to the executable,
(2) a correct call using the relative path (recall that
.
represents the current/present directory),
(3) a correct call using the absolute path (for the user
jhsia).
[attu]$ hello -bash: hello: command not found [attu]$ ./hello Hello, world! [attu]$ /homes/iws/jhsia/hello Hello, world!
./
to your executable name) is by
far the most common usage.
Execution Output
Some executables, particularly some of our lab test suites, output many lines of text and numbers that will be useful for understanding your current progress and debugging. However, your terminal can only display so many of those lines, based on the window size, the text font size, and your terminal's line buffering (this can sometimes be changed in settings, but will have a defined maximum value). Try scrolling upward with your mouse first, but if you can't see all of the output, you can use one of the techniques below to do so:
Save output to a file
Append a greater-than character and file name to the end of your
command to save the execution output to a file instead of printing
it to the terminal.
For example, to save the output of the executable ptest
to the file output.txt
, you would run:
[attu]$ ./ptest > output.txt [attu]$
- Notice that the output doesn't show up in the terminal anymore
because it's been redirected to the file.
Use your favorite text editor to open
output.txt
to view the output. - If the file
output.txt
already exists in the current directory, it will overwrite/replace the contents of that file, so be careful!
View output screen-by-screen
Append a vertical bar character and the command less
to
the end of your command to show output "page-by-page."
For example, for the executable ptest
, you would run:
[attu]$ ./ptest | less
- You can navigate the output using the Up (up one line), Down (down one line), and Space (down one screen) keys.
- You can exit from this view by either reaching the end of the output or pressing the 'Q' key.
- Note that this method does not store the output anywhere so once you exit this view, you won't be able to view the output unless you run the whole command again.
Debugging
Debugging is a critical skill for computing, but one that you are not expected to be very good at yet. An explicit goal of the course is to help you improve your debugging skills, though debugging in 351 will likely feel very different than what you may have experienced previously for a number of reasons:
- We will use new languages in C and x86-64 assembly.
- We will use a text user interface debugger called GDB.
- The labs will focus less on composing programming language constructs properly and more on using our conceptual knowledge of the underlying computer organization to solve tasks/puzzles.
Because of the importance of this skill and the newness of the 351 environment, we've provided an entirely separate to help you throughout the course.
Written by Andrew Reusch (areusch@gmail.com). Updated by Sarang Joshi (sarangj@cs.uw.edu), Andrew Hu (andrewhu@uw.edu), and Justin Hsia (jhsia@cs.uw.edu).