CSE 466 Lab 5: Introduction to FriendlyARM and Qtopia Linux

Introduction

Now that you've worked with a simple 8-bit microcontroller, we'll be exploring more powerful embedded devices. For the rest of the quarter, we'll be using the FriendlyARM board, which uses a 400 MHz Samsung S3C2440 ARM920 Microprocessor. This lab will introduce you to the FriendlyARM platform, and in using Linux as an embedded operating system.

Objectives

In this lab, you will learn:

• How to write user-level applications and load them onto the FriendlyARM
• How applications communicate with devices in Linux
• The basics of the Linux character device model
• How to write programs that use the ALSA audio libraries and devices
• How to read and play uncompressed digital audio files
• How to generate sounds from a sine table
• How to apply ADSR envelopes
• Optionally, how to use frequency modulation to generate more complex sounds

Suggested Reading and Resources

WARNING: There is some important and really useful information in this section. Don't skip over it!

• Chapters 1-4 and 10 from the Linux Device Drivers book. This is a lot of reading to do, but I suggest that you sit down with the book and at least skim through it. This book is one of the best resources on the Linux kernel. It's also available on Safari Books (free access on-campus thanks to the Engineering Library, and you can use the library proxy from off-campus) if you like the web interface better than the individual chapter PDF files. It's also a good reference to have in dead tree form if you're interested in doing more Linux development in the future.
• It's not required reading for any of the labs, but Building Embedded Linux Systems (link to Safari Books) provides a good overview of a lot of the lower-level details of an embedded Linux system (and Linux in general). It discusses the purpose of all of the files on the root filesystem, how the boot process is handled, and many other topics pertaining to embedded Linux.

Part 1: Getting acquainted with the platform

NOTE: As they were initially constructed, the FriendlyARM screen covers most of the connections and buttons you will be playing with. Before you do anything, you should unscrew the bolts on the bottom of the board, and mount the screen on the other side. Please ask a TA for help if you're not sure exactly what to do to remount the screen.

Before starting, you should make sure that you have the FriendlyARM platform, a stylus, a USB cable, an SD card, and an SD card reader. Qtopia linux should already be installed. Spend some time opening applications, pressing buttons, and watching the LCDs. There's also a really annoying buzzer application which I won't help you find, but if you do, you can probably drive out everyone in the lab.

NOTE: To change the language to english, click on the 'bubbles' icon at the top of the screen, then the 'flag' icon

To get programs onto the FriendlyARM, we are compiling C programs using an arm version of gcc. The way we'll do this is to compile our C program, load it into an SD card, then put the SD card into the FriendlyARM and run the program from there.

You'll probably be doing a good amount of shell manipulation on the FriendlyARM platform. If you connect a keyboard or mouse to the platform, it should automatically connect.

First we need to setup the arm cross compiler. To do this, log into attu, and add this command to your .bashrc:

    export PATH=/cse/courses/cse466/Instr/arm_gcc/4.3.2/bin/:$PATH

If you are using a different shell, Google, or the man page for your shell, can probably tell you how to add things to your profile. You can also always specify the full path to the compiler every time you invoke it, if you really want to do all of that extra typing.

Download the hello world code to your attu account, and run 'make'. Take the resulting hello application, move it to your SD card, and put it on the FriendlyARM. Open the terminal and run the 'hello' application, making sure it prints out hello. It might be a good idea to pay attention to what the makefile is doing, since you're gonna have to create your own from now on (it's really not complicated).

Next, try running the LED code, using the same steps as above. This program shows an example of how to open drivers and how to communicate with these drivers. Make sure you understand what this code does and play around with different input arguments to see the effects on the LED buttons.

NOTE: If you'd prefer not to type on the tiny FriendlyARM screen, you can either directly plug in a usb keyboard/mouse, or telnet into the unit. To do this, find the networking tool on the friendlyARM, connect an ethernet cable between it and your computer, and run ifconfig to set up the proper ip address and netmask on your computer. Then just use the command "telnet x.x.x.x", where x.x.x.x represents the ip of the friendlyARM (as set in the networking tool).

Part 2: Sound on the FriendlyARM

Suggested Reading and Resources

• The ALSA project web site — includes documentation
• Documentation for ALSA Digital Audio (PCM)
• ALSA API Tutorial — a good starting point
• Sun .au file format specification
• FM Synthesis Examples
• Open Sound System v4.0 API Reference (deprecated - for reference only)

For this part of the lab, you will implement playback of sampled audio files in .au format. .au is a good format for us to use because the file format is very simple and there are few variations. (WAV is straightforward, but there are many different variations and implementing them all can be tedious.)

First let's get ALSA working on the FriendlyARM. Download the alsa library file to your attu account. CD into the directory and compile with these commands:

CC=arm-linux-gcc ./configure --target=arm-linux --host=i686-pc-linux-gnu 

make install DESTDIR=~/fileDirectory

This will cross compile the alsa library source and put it in a folder in your home directory that you can then move over to the friendlyARM. Copy the usr folder in this directory to your friendlyARM's /usr folder. Also, download the devices setup file, move this to the FriendlyARM, and run the command. This will set up the /dev devices that alsa can use.

UPDATE: If you're having trouble compiling alsa lib, just download the compiled libraries here: compiled alsa library files

Finally, when we need to compile our alsa programs, we can use the command:

 arm-linux-gcc auplay.c -L ~/fileDirectory/usr/lib/ -I ~/fileDirectory/usr/include/ -lasound

1. Download the auplay starter code. The zip file contains:
   • auplay.c, a skeleton that contains a struct that defines the format of an .au file header and a few useful constants
   • several sample .au files, with different encodings, sample rates, and numbers of channels, which you can use to test your application.
2. Add code to auplay.c that accepts the name of a .au file as an argument, reads the header from the audio file, and prints out the encoding (format), sample rate, and number of channels for the recording.
   • Use the sample files provided to test that your program is able to accurately recognize different types of .au files. Your program should be able to handle 8- and 16-bit signed linear PCM files, in addition to 8-bit µlaw. It should also work for any sample rate that the codec will support, with either 1 or 2 channels.
   • Note that all of the data fields in the .au header, in addition to the sample data, are stored in network (big-endian) byte order. The system may be running in little-endian mode. You will likely find the ntohl() (convert from network to host order, long (i.e. 32-bit integer)) function in arpa/inet.h to be useful in reading the header.
3. Once your program has successfully opened the audio device and set the appropriate format parameters, read the rest of the data in the audio file and call the appropriate ALSA functions to play back the samples.
4. The ALSA tutorial linked in the suggested reading and resources section provides much of the information you need to know to get sound playback working.

How to use ALSA for simple playback, in a nutshell

Create a pointer to a snd_pcm_t, called pcm_handle, which all further functions will take as a parameter.

Open the audio device with the snd_pcm_open function. We want to open for playback (SND_PCM_STREAM_PLAYBACK), with no special mode flags. The device name is a string; you can explicitly open the first interface on the first soundcard with "plughw:0,0", or you can just use the default interface with "default"

Set up all of the hardware and software parameters. You can do this in two ways: you can create a snd_pcm_hw_params_t struct, and call the functions to allocate the space for it, initialize it, and set all of the parameters individually. Or, you can call snd_pcm_set_params, which allows you to set all of the parameters you will need for this lab in one shot.

snd_pcm_set_params takes a bunch of arguments. Format, channels, and rate should be obvious (you're reading these from the .au file.) For access, use SND_PCM_ACCESS_RW_INTERLEAVED. This will allow us to write to the device (and read if we wanted to do capture) and says that stereo samples will be interleaved (alternating between left and right, instead of having the samples for the two different channels in two separate buffers.) Set 1 for resample; this will allow ALSA to convert sample rates in software if you're playing back at a rate that the hardware doesn't support (the hardware supports all of the rates you will be implementing, however.) Set the latency to 500000; we're not trying to synchronize audio so we'll allow it to be as much as half a second late. Setting it lower will result in less latency (time between writing samples and actually having them played) but higher risk of buffer underruns that will result in dropped samples and having to recover the audio device.

Then, call snd_pcm_writei in a loop, giving it buffers of data from the file. Note that the size argument in writei is the number of frames that you are writing; a frame is defined as a complete set of samples for all channels. In other words, in mono, a frame is one sample. In stereo, a frame is two samples. Make sure that you don't split frames between calls to writei; you must write a whole number of frames each time!

Finally, when your program exits, you should call snd_pcm_close to clean up.

Part 3: Sine table synthesis

In this part of the lab, you will design a synthesizer that uses a table-based oscillator.

1. The sin() function in math.h implements the sine function for angles in radians with floating-point arithmetic. Since the support for floating point numbers is limited on the FriendlyARM, you should precalculate the sine table on attu and compile it into your synthesizer. You should write a C program that you can run to produce the table for you.
   • Your program should produce as output a valid .c file that defines your table as an array of type int16_t that you can include in your synthesizer program.
   • You can think of each sample as a 16-bit signed fixed-point number, where an integer value of 32767 represents the maximum value of 1, and -32767 represents -1. If you exceed this range, the numbers will "wrap around" and create crackly artifacts in the sound.
   • It is up to you to determine the size of your table. You may want to make the table size an argument to your program so you can easily create tables of different sizes as you are designing your synthesizer.
2. Now that you have your fixed-point sine table, write a program that uses it to generate continuous tones.
   • Set up ALSA to accept samples for playback like you did with your playback application. You should configure it to use 16-bit samples at 44100 Hz.
   • Your program should use a fixed-point counter that is incremented on each sample to track the current position in your sine table. To attain higher frequencies, you increment by a larger amount on each sample, and to attain lower frequencies, you increment by a smaller amount.
   • You should not write each sample to the audio device immediately as it is ready; rather, your program should copy samples to a buffer of about the same size you used for your au playback program, and then when the buffer is full, you write it to the audio device.
3. Next, write another program to run on attu that will generate another table containing an amplitude envelope. Like your sine table generator, this program should produce a valid .c file containing the table in an array that you can include in your synthesizer.
   • Once again, the size of the table is up to you. It doesn't necessarily have to be the same size as your sine table; you'll be incrementing through it with a different counter at a different speed. Whether you use 8- or 16-bit fixed-point numbers is up to you as well.
   • Your program should take arguments that let you customize the table size, as well as the attack, sustain, decay, and release parameters of the envelope. (Refer back to the lecture notes on sound for definitions of these parameters.)
4. Modify your synthesizer to apply your amplitude envelope to the generated sound. You should use a separate counter for the envelope table, and make sure that you stop producing sound when you reach the end of your envelope.
   • Remember the rules for multiplying two numbers with a fixed number of decimal places. The same rules apply to binary fixed-point numbers. If you multiply a number with 16 bits of fraction by a number with 8 bits of fraction, the resulting value will have 24 bits of fraction. Use the right shift operation to move the "binary point" back where you want it to be.
   • Be careful not to exceed the maximum value that you can represent in a 16-bit signed number when writing samples to the audio codec. If you do, you'll hear it in the form of pops and clicks in the audio.

Part 4: Do something interesting with your synthesizer

At this point, you should have a working synthesizer that generates sounds from a sine table and applies an amplitude envelope. Now is your chance to be creative and extend your synthesizer program to do something with the sounds it can generate.

• One option is to create a simple sequencer and have your synthesizer play a song. Calculate the increment values for your table that produce different notes on the chromatic scale, and add constants to your program that define the names of the notes ('C4', etc) as their numeric increment values. Write a function that takes the increment value and a duration as arguments, then call this function with a sequence of notes that play a song.
  • You can obtain the frequencies of the chromatic scale by taking the frequency of your base note and multiplying by the twelfth root of two. Each time you do this, you increase the pitch by a half step. You can go up an octave at a time by doubling the frequency.
  • You can also try something like playing back MIDI files with your synthesizer.
  • Instead of building tables with sine waves, you can also place a sampled audio file into the table and play it back at different pitches and with different envelopes. If you do this, your program should load the sound files at runtime; don't try and compile them into your C program.
• Alternately, come up with a different project idea for your synthesizer. There are several sensors on the boards available to you that could make interesting control inputs for a synthesizer. Run your idea by a T.A. prior to starting.
• Make sure you make a back-up copy of your working synthesizer code from Part 3 before modifying it.
• Extra credit may be awarded for particularly creative and well-implemented ideas.

Extra Credit: Implement FM synthesis

For extra credit, you can add a second oscillator and generate sounds via frequency modulation synthesis. See this PDF for some examples.

Lab 5, Second half: Get the AVR and the FriendlyARM to communicate

In this part of the lab, our goal is to set up I2C between your FriendlyARM and your Atmel setup. We want you to use this connection to create a musical instrument that involves both the FriendlyARM and the Atmel. An example of this could be to make a 'theremin', which would use the electric field as data input to the Atmel, which is then sent to the FriendlyARM via I2C, and then affects the pitch of the output sound of the FriendlyARM. You are free to use any combination of sensors/buttons/switches that you want for your instrument, as long as it is the Atmel that is reading the data and sending it off to the FriendlyARM.

This lab is intentionally left open ended, so we will give you some code that should allow you to get I2C started, and it will be up to you to create a working protocol between the ARM and the AVR (and to set up the instrument, of course).

The starting code you will use is available here

Finally, here are some tips to get you started:

• The code for the Atmel includes what appears to be a full state machine for the four possible states, Master Transmitter, Master Receiver, Slave Transmitter, and Slave Receiver. Apparently, the Mini2440 only works in Master mode, so the Atmel will need to run in pure Slave mode. You will need to find the correct code for the Atmel slave mode, and can likely get rid of the other states.
• As originally written, the I2C ARM code uses the EEPROM to interface with I2C. For this example, we've essentially hacked together code that bypasses the EEPROM and uses the GPIO pins for I2C. This means you will use the SDA and SCL pins to interface with the AVR (refer to the Mini2440 schematic to find these pins). You'll need a ribbon cable to connect these pins to the AVR, so ask a TA for this cable.
• The AVR uses pins 22 and 23 for SCL and SDA.

  Try and keep your instrument creative, we'll be giving extra points to the best ones!

  Deliverables

  Turn in all required documents via an e-mail attachment to cse466-tas@cs. For all files turned in, the comments at the top of the file should contain:

  • • Both partners' full name and Group Number.
    • The lab number and the part of the lab (e.g. "Lab4, Part2").
  1. Demonstrate your .au player, your synthesizer application, and your I2C instrument to a T.A.
  2. You should turn in source files for:
     • your .au player,
     • your table generation utilities, and
     • your synthesizer application.
     • your instrument, using I2C.
     You should create a .zip or .tar archive of all of the files you are turning in.