Final Project
The final project for this course will be a self-guided research project completed in groups of 2-3 people, using the concepts we've learned throughout the semester. We will evaluate you on the following skills:
- Developing a well-scoped research question
- Designing experiments to answer this question
- Implementing these experiments independently (no TA helper code)
- Explaining your findings
What we don't care about:
- Novelty. We don't expect you to run experiments that no one has ever run before. However, your experiments should be your own. Downloading a running and existing repository is not sufficient.
- State-of-the-art performance. Almost no one achieves this, even at top-tier conferences.
To help you understand what we're looking for, we'll walk through an example project and show you how to approach each of these four steps.
1. Developing a Well-Scoped Research Question
Pick something that you are interested in! It can be diving more deeply into a topic from the class or it can be applying methods we learned to a problem of your choice.
Let's say you start with: "What is the best method for bird identification?"
How are you going to find the best method? Are you going to try every possible method? Every combination of architecture + model size + activation + optimizer + etc.? That's not feasible.
Instead, narrow it down. What are you most interested in exploring?
- Maybe you're interested in what kinds of pretraining data best transfer to bird classification?
- Maybe you're interested in what kinds of data augmentation are most important when training for bird classification?
- Maybe you're interested in whether a small specialized model trained only on bird data makes the same kinds of mistakes as a large-scale vision-language model like ChatGPT?
- Maybe you're interested in how much information about bird species is communicated by the background?
These are all good questions! Let's select one to continue refining this project.
For our example, we'll explore: "Can a model perform bird identification using only the background?"
2. Designing Experiments to Answer Your Question
Now we want to design experiments for the question: "Can a model perform bird identification using only the background?"
You can approach this in different ways, but I usually like to think about what plot I want to create. The numbers don't matter at this point in the projects but physically draw out some plots with random numbers to understand the best way to present your ultimate findings. Its much easier to figure out what you want to show and then work backwards from there. For our example, we want a plot that shows us the accuracy of a model on a test set of images with just the background. Maybe a bar chart like this:
This is a start, but it's hard to interpret. How do we know if this is a high or low number? What are we comparing it to?
If we want to know how well the model performs with just the background, we should compare it to how well it performs to some control. This is called the baseline. For our example, we want to know how well a model can do with just the background, so we might want to compare this how well the model performs with the full image. Now we have a sense of whether the "just background" number is "good" or "bad." We can go a step further and also train/test a model on images with ONLY the bird (no background) to see how that performs.
We can also add more experiments. For example, in out initial experiments, we were removing information about the bird by "cutting out" the bird from the image using its exact outline, which still includes information about the bird's shape. So maybe we also run the "with bird" and "without bird" experiments by cutting out a box around the bird instead of using an exact outline.
3. Implementing Your Experiments
Now that we know what results we want, it's time to code! We need to train and test a model on a bird dataset, modified in different ways.
This means we need:
- A model
- A dataset
- A way to modify the images in our dataset
Pick your model. How do we choose? Well, our research question doesn't focus on which model we're using, so let's just pick a standard one! If this were a real paper we wanted to submit to a conference or use for company decisions, we might run experiments on multiple models to ensure our findings are consistent. But for this project, just pick a standard model like ViT or ResNet.
Pick your dataset. We could create a dataset—maybe scrape images of birds from a website where people upload photos and use the titles as labels. Or we can search for a preexisting dataset. In this case, there are many existing bird datasets, so we can just use one of those.
Modify the images. Now we need a way to "cut out" the birds from the background. One option is doing this by hand, but that's very impractical. Let's automate it! We can use an off-the-shelf segmentation model to locate the birds and remove them from the image.
Now we just need to write a training loop (which you'll be well-practiced at by the end of the course!) to:
- Train your model on normal images and test on normal images
- Train your model on "just background" images and test on "just background" images
- Train your model on "just bird" images and test on "just bird" images
4. Explaining Your Findings
Create a poster that walks us through everything you've done. Start by explaining your research question—why was this interesting to you? Then explain your process and results. Remember: visuals > text.
This is also a good time to dive deeper into your findings. Did the background help equally for all species of birds? Did the "just background" models perform very well for certain types? Here is where you can create additional plots.
Additional Exercises
You're done! As an extra exercise, pick another research question from Step 1. We explored research option (d), but think about how you would design a project around (a)–(c).
Example Project
Below is an example of a successful final project from a previous year. Think about what their research question was and how they translated it into experiments.