CSE 466 Lab 6:
Sound on the iMote 2 and SuperBird

Introduction

In this lab, you will explore the sound generation capabilities of the SuperBord board. You will begin by writing a program to read and play back sampled audio files. Then, you will implement a simple synthesizer using a sine table and ADSR amplitude envelopes.

The last part of this lab is left open-ended to allow you to come up with a creative application for your synthesizer on the SuperBird.

WARNING: NEVER attach or detach the iMote2 from the debug board or the sensor board when power is attached. ALWAYS make sure you've unplugged ALL of the USB cables when connecting or disconnecting the sensor board, debug board, or SuperBird.

Objectives

In this lab, you will learn:

• How to write programs that use the ALSA audio libraries and devices
• How to read and play uncompressed digital audio files
• How to generate sounds from a sine table
• How to apply ADSR envelopes
• Optionally, how to use frequency modulation to generate more complex sounds

Suggested Reading and Resources

• The ALSA project web site — includes documentation
• Documentation for ALSA Digital Audio (PCM)
• ALSA API Tutorial — a good starting point
• Lecture Notes on Sound and FM Synthesis
• Sun .au file format specification
• SuperBird Audio driver information and source code — binary modules are provided below for you to install, but the driver code is available for you to examine if you are interested.
• AK4642 Audio codec/speaker amp datasheet (for those interested in how the hardware works)
• SuperBird Schematic
• FM Synthesis Examples
• Open Sound System v4.0 API Reference (deprecated - for reference only)

Part 1: Getting started with the Superbird

You will obtain a SuperBird board from a TA before starting this lab. If we are able to get enough boards working, your group will also receive a second SuperBird board before the final demonstration in the Atrium. You will also obtain a portable speaker set that can be optionally plugged into your SuperBird for better sound quality.

Take a moment to familiarize yourself with the SuperBird hardware. The SuperBird connects to the iMote2 on the advanced connector side, and provides numerous additional peripherals. These include:

• Power supply to the iMote2, via an AC adapter, or batteries (with chargers included on the board; these will run when both the AC adapter and batteries are connected.) The SuperBird may run with just the AC adapter, with one battery, or with two for extended runtime.
• Two pushbutton switches and a jog dial for navigating menus
• A small cell-phone-sized color LCD screen
• An I2S audio codec, speaker, microphone, and input and output jacks
• An RGB LED identical to the one on the iMote2
• A light sensor and LED on the bottom of the board (labeled "touch here" that can be used as a heart-rate sensor)
• A high-resolution barometer (these were soldered on backwards during PCA and may have been destroyed by hot-air rework to turn them around)
• Connectors for a cell-phone style VGA camera, which are unfortunately miswired
• a USB host connector

The functions of the iMote2 are coordinated by an ATmega16 on the bottom of the board, which communicates with the iMote2 via an I2C interface. The firmware for the SuperBird has been preloaded onto this chip.

This quarter, we will be primarily using the audio codec, LCD, and switches. In this lab, we will focus on using the audio hardware to generate sounds.

Updating the Kernel

A few kernel configurations had to be changed from the one that was distributed on the iMote2s at the beginning of lab 5. Before you will be able to load the audio modules, you will need to update the Linux kernel on your iMote2.

The flash memory on the iMote2's PXA271 is partitioned into three partitions. The first contains the bootloader which the iMote2 begins executing when it boots. The second contains the kernel image, and the third contains the root filesystem. These are accessible as /dev/mtdblock0, /dev/mtdblock1, and /dev/mtdblock2, respectively. To update the kernel, we will replace the contents of /dev/mtdblock1 with the new image.

1. Download kernel.zip and extract zImage from it.
2. Copy zImage to the /tmp directory on your iMote2.
3. Execute the following command (check for typos!):

   	dd if=/tmp/zImage of=/dev/mtdblock1

4. Reboot your iMote2.

If something goes wrong and your iMote2 does not boot after updating, it will need to be re-flashed via JTAG. Consult a TA for assistance.

SuperBird Driver Installation

Before you can use the SuperBird, there are several drivers which you will need to install. These drivers have been precompiled and packaged in the ipkg format. ipkg is similar to package managers such as dpkg on Debian and rpm on Red Hat, but it has been designed with only the essentials for a small embedded system.

Instructions for installing ipk packages (and several other optional precompiled packages specific to our iMote2 setup) are available here, on my iMote2 wiki. The ipkg documentation on the Familiar Linux project pages may also be a useful reference.

NOTE: The /tmp directory is recommended for copying over the package files so that you can install them. /tmp is mounted as a filesystem in RAM, and will be erased when you reboot. Since you don't need to keep the package files around once they're installed, using /tmp is fine.

Install the following packages on your iMote2 by copying them over to your iMote2 with scp or sftp, running ipkg install <package>.ipk, and then removing the package files to free space. Installing them in the listed order will ensure that there are no package dependency issues.

• ncurses — user interface library required by alsamixer
• superbird — driver for the power management and switches on the SuperBird; interfaces with the ATmega16 on the SuperBird via I2C
• snd-soc-superbird — ALSA driver for the SuperBird's audio codec
• alsa-lib — ALSA userspace libraries
• alsa-util — ALSA userspace utility programs
• libmad — mp3 playback libraries and programs (optional)

The latest versions of these packages are, and always will be, available in this directory.

Using the SuperBird

Make sure that both the iMote2 and SuperBird are powered off and that no cables are connected, then connect the SuperBird to your iMote2's advanced connector side. Connect power to the SuperBird, and push the power button (on the left side of the SuperBird, near the battery connectors.) This will power on both the SuperBird and the iMote2. Then, you can connect the USB cable from the iMote2's USB connector to your computer. This is the recommended order for connecting and powering on the system; if you connect the USB cable before the SuperBird is powered on, the iMote2 will begin starting up without the SuperBird.

NOTE: While you will likely be using your SuperBird with wall power while you are developing in the lab, it is recommended that you connect the battery too so that it will be charged when you need it.

The sbstatus program (installed as part of the superbird driver package) communicates with the ATmega16 on the SuperBird and reads various status information. You should run sbstatus and verify that the jog dial and the two pushbutton switches on either side are functional. If any of the switches have no effect, exchange your SuperBird with a TA for repair. sbstatus also provides useful information about the power system on the SuperBird.

Try playing back a WAV or .au file (there are several in the auplay starter code zip file) with aplay, or an MP3 file with madplay to verify that the audio hardware on the SuperBird is working. You may also wish to experiment with the alsamixer program. This is the same interface that is used on desktop Linux machines to control audio volume and other settings. On the SuperBird, you can use alsamixer to enable and disable the various audio outputs, which you will need to do if you want to use external speakers or headphones.

Part 2: Sampled audio playback

For this part of the lab, you will implement playback of sampled audio files in .au format. .au is a good format for us to use because the file format is very simple and there are few variations. (WAV is straightforward, but there are many different variations and implementing them all can be tedious.)

1. Download the auplay starter code. The zip file contains:
   • auplay.c, a skeleton that contains a struct that defines the format of an .au file header and a few useful constants
   • several sample .au files, with different encodings, sample rates, and numbers of channels, which you can use to test your application.
2. Add code to auplay.c that accepts the name of a .au file as an argument, reads the header from the audio file, and prints out the encoding (format), sample rate, and number of channels for the recording.
   • Use the sample files provided to test that your program is able to accurately recognize different types of .au files. Your program should be able to handle 8- and 16-bit signed linear PCM files, in addition to 8-bit µlaw. It should also work for any sample rate that the codec will support, with either 1 or 2 channels.
   • Note that all of the data fields in the .au header, in addition to the sample data, are stored in network (big-endian) byte order. The PXA271 is running in little-endian mode. You will likely find the ntohl() (convert from network to host order, long (i.e. 32-bit integer)) function in arpa/inet.h to be useful in reading the header.
3. Once your program has successfully opened the audio device and set the appropriate format parameters, read the rest of the data in the audio file and call the appropriate ALSA functions to play back the samples.
4. The ALSA tutorial linked in the suggested reading and resources section provides much of the information you need to know to get sound playback working.

How to use ALSA for simple playback, in a nutshell

Create a pointer to a snd_pcm_t, called pcm_handle, which all further functions will take as a parameter.

Open the audio device with the snd_pcm_open function. We want to open for playback (SND_PCM_STREAM_PLAYBACK), with no special mode flags. The device name is a string; you can explicitly open the first interface on the first soundcard with "plughw:0,0", or you can just use the default interface with "default"

Set up all of the hardware and software parameters. You can do this in two ways: you can create a snd_pcm_hw_params_t struct, and call the functions to allocate the space for it, initialize it, and set all of the parameters individually. Or, you can call snd_pcm_set_params, which allows you to set all of the parameters you will need for this lab in one shot.

snd_pcm_set_params takes a bunch of arguments. Format, channels, and rate should be obvious (you're reading these from the .au file.) For access, use SND_PCM_ACCESS_RW_INTERLEAVED. This will allow us to write to the device (and read if we wanted to do capture) and says that stereo samples will be interleaved (alternating between left and right, instead of having the samples for the two different channels in two separate buffers.) Set 1 for resample; this will allow ALSA to convert sample rates in software if you're playing back at a rate that the hardware doesn't support (the hardware supports all of the rates you will be implementing, however.) Set the latency to 500000; we're not trying to synchronize audio so we'll allow it to be as much as half a second late. Setting it lower will result in less latency (time between writing samples and actually having them played) but higher risk of buffer underruns that will result in dropped samples and having to recover the audio device.

Then, call snd_pcm_writei in a loop, giving it buffers of data from the file. Note that the size argument in writei is the number of frames that you are writing; a frame is defined as a complete set of samples for all channels. In other words, in mono, a frame is one sample. In stereo, a frame is two samples. Make sure that you don't split frames between calls to writei; you must write a whole number of frames each time!

Finally, when your program exits, you should call snd_pcm_close to clean up.

Part 3: Sine table synthesis

In this part of the lab, you will design a synthesizer that uses a table-based oscillator.

1. The sin() function in math.h implements the sine function for angles in radians with floating-point arithmetic. Since the support for floating point numbers is limited on the iMote2, you should precalculate the sine table on attu and compile it into your synthesizer. You should write a C program that you can run to produce the table for you.
   • Your program should produce as output a valid .c file that defines your table as an array of type int16_t that you can include in your synthesizer program.
   • You can think of each sample as a 16-bit signed fixed-point number, where an integer value of 32767 represents the maximum value of 1, and -32767 represents -1. If you exceed this range, the numbers will "wrap around" and create crackly artifacts in the sound.
   • It is up to you to determine the size of your table. You may want to make the table size an argument to your program so you can easily create tables of different sizes as you are designing your synthesizer.
2. Now that you have your fixed-point sine table, write a program that uses it to generate continuous tones.
   • Set up ALSA to accept samples for playback like you did with your playback application. You should configure it to use 16-bit samples at 44100 Hz.
   • Your program should use a fixed-point counter that is incremented on each sample to track the current position in your sine table. To attain higher frequencies, you increment by a larger amount on each sample, and to attain lower frequencies, you increment by a smaller amount.
   • You should not write each sample to the audio device immediately as it is ready; rather, your program should copy samples to a buffer of about the same size you used for your au playback program, and then when the buffer is full, you write it to the audio device.
3. Next, write another program to run on attu that will generate another table containing an amplitude envelope. Like your sine table generator, this program should produce a valid .c file containing the table in an array that you can include in your synthesizer.
   • Once again, the size of the table is up to you. It doesn't necessarily have to be the same size as your sine table; you'll be incrementing through it with a different counter at a different speed. Whether you use 8- or 16-bit fixed-point numbers is up to you as well.
   • Your program should take arguments that let you customize the table size, as well as the attack, sustain, decay, and release parameters of the envelope. (Refer back to the lecture notes on sound for definitions of these parameters.)
4. Modify your synthesizer to apply your amplitude envelope to the generated sound. You should use a separate counter for the envelope table, and make sure that you stop producing sound when you reach the end of your envelope.
   • Remember the rules for multiplying two numbers with a fixed number of decimal places. The same rules apply to binary fixed-point numbers. If you multiply a number with 16 bits of fraction by a number with 8 bits of fraction, the resulting value will have 24 bits of fraction. Use the right shift operation to move the "binary point" back where you want it to be.
   • Be careful not to exceed the maximum value that you can represent in a 16-bit signed number when writing samples to the audio codec. If you do, you'll hear it in the form of pops and clicks in the audio.

Part 4: Do something interesting with your synthesizer

At this point, you should have a working synthesizer that generates sounds from a sine table and applies an amplitude envelope. Now is your chance to be creative and extend your synthesizer program to do something with the sounds it can generate.

• One option is to create a simple sequencer and have your synthesizer play a song. Calculate the increment values for your table that produce different notes on the chromatic scale, and add constants to your program that define the names of the notes ('C4', etc) as their numeric increment values. Write a function that takes the increment value and a duration as arguments, then call this function with a sequence of notes that play a song.
  • You can obtain the frequencies of the chromatic scale by taking the frequency of your base note and multiplying by the twelfth root of two. Each time you do this, you increase the pitch by a half step. You can go up an octave at a time by doubling the frequency.
  • You can also try something like playing back MIDI files with your synthesizer.
  • Instead of building tables with sine waves, you can also place a sampled audio file into the table and play it back at different pitches and with different envelopes. If you do this, your program should load the sound files at runtime; don't try and compile them into your C program.
• Alternately, come up with a different project idea for your synthesizer. There are several sensors on the boards available to you that could make interesting control inputs for a synthesizer. Run your idea by a T.A. prior to starting.
• Make sure you make a back-up copy of your working synthesizer code from Part 3 before modifying it.
• Extra credit may be awarded for particularly creative and well-implemented ideas.

Extra Credit: Implement FM synthesis

For extra credit, you can add a second oscillator and generate sounds via frequency modulation synthesis. See this PDF for some examples.

Deliverables

Turn in all required documents via an e-mail attachment to cse466-tas@cs. For all files turned in, the comments at the top of the file should contain:

• • Both partners' full name, login and student number.
  • The lab number and the part of the lab (e.g. "Lab4, Part2").

1. Demonstrate your .au player and your synthesizer application to a T.A. You can either do this during the first 1/2 hour of the next lab.
2. You should turn in source files for:
   • your .au player,
   • your table generation utilities, and
   • your synthesizer application.
   You should create a .zip or .tar archive of all of the files you are turning in.