Final Project
The final project for this course will be a self-guided research project completed in groups of 2-3 people, using the concepts we've learned throughout the semester. We will evaluate you on the following skills:
- Developing a well-scoped research question
- Designing experiments to answer this question
- Implementing these experiments independently (no TA helper code)
- Explaining your findings
What we don't care about:
- Novelty. We don't expect you to run experiments that no one has ever run before. However, your experiments should be your own. Downloading a running and existing repository is not sufficient.
- State-of-the-art performance. Almost no one achieves this, even at top-tier conferences.
To help you understand what we're looking for, we'll walk through an example project and show you how to approach each of these four steps.
1. Developing a Well-Scoped Research Question
Pick something that you are interested in! It can be diving more deeply into a topic from the class or it can be applying methods we learned to a problem of your choice.
Let's say you start with: "What is the best method for bird identification?"
How are you going to find the best method? Are you going to try every possible method? Every combination of architecture + model size + activation + optimizer + etc.? That's not feasible.
Instead, narrow it down. What are you most interested in exploring?
- Maybe you're interested in what kinds of pretraining data best transfer to bird classification?
- Maybe you're interested in what kinds of data augmentation are most important when training for bird classification?
- Maybe you're interested in whether a small specialized model trained only on bird data makes the same kinds of mistakes as a large-scale vision-language model like ChatGPT?
- Maybe you're interested in how much information about bird species is communicated by the background?
These are all good questions! Let's select one to continue refining this project.
For our example, we'll explore: "Can a model perform bird identification using only the background?"
2. Designing Experiments to Answer Your Question
Now we want to design experiments for the question: "Can a model perform bird identification using only the background?"
You can approach this in different ways, but I usually like to think about what plot I want to create. The numbers don't matter at this point in the projects but physically draw out some plots with random numbers to understand the best way to present your ultimate findings. Its much easier to figure out what you want to show and then work backwards from there. For our example, we want a plot that shows us the accuracy of a model on a test set of images with just the background. Maybe a bar chart like this:
This is a start, but it's hard to interpret. How do we know if this is a high or low number? What are we comparing it to?
If we want to know how well the model performs with just the background, we should compare it to how well it performs to some control. This is called the baseline. For our example, we want to know how well a model can do with just the background, so we might want to compare this how well the model performs with the full image. Now we have a sense of whether the "just background" number is "good" or "bad." We can go a step further and also train/test a model on images with ONLY the bird (no background) to see how that performs.
We can also add more experiments. For example, in out initial experiments, we were removing information about the bird by "cutting out" the bird from the image using its exact outline, which still includes information about the bird's shape. So maybe we also run the "with bird" and "without bird" experiments by cutting out a box around the bird instead of using an exact outline.
3. Implementing Your Experiments
Now that we know what results we want, it's time to code! We need to train and test a model on a bird dataset, modified in different ways.
This means we need:
- A model
- A dataset
- A way to modify the images in our dataset
Pick your model. How do we choose? Well, our research question doesn't focus on which model we're using, so let's just pick a standard one! If this were a real paper we wanted to submit to a conference or use for company decisions, we might run experiments on multiple models to ensure our findings are consistent. But for this project, just pick a standard model like ViT or ResNet.
Pick your dataset. We could create a dataset—maybe scrape images of birds from a website where people upload photos and use the titles as labels. Or we can search for a preexisting dataset. In this case, there are many existing bird datasets, so we can just use one of those.
Modify the images. Now we need a way to "cut out" the birds from the background. One option is doing this by hand, but that's very impractical. Let's automate it! We can use an off-the-shelf segmentation model to locate the birds and remove them from the image.
Now we just need to write a training loop (which you'll be well-practiced at by the end of the course!) to:
- Train your model on normal images and test on normal images
- Train your model on "just background" images and test on "just background" images
- Train your model on "just bird" images and test on "just bird" images
4. Explaining Your Findings
Create a poster that walks us through everything you've done. Start by explaining your research question—why was this interesting to you? Then explain your process and results. Remember: visuals > text.
This is also a good time to dive deeper into your findings. Did the background help equally for all species of birds? Did the "just background" models perform very well for certain types? Here is where you can create additional plots.
Additional Exercises
You're done! As an extra exercise, pick another research question from Step 1. We explored research option (d), but think about how you would design a project around (a)–(c).
Example Project
Below is an example of a successful final project from a previous year. Think about what their research question was and how they translated it into experiments.
Project Milestones
The final project consists of three graded milestones designed to guide your team through developing a well-executed research project.
1. Project Proposal (3% of Total Grade)
The project proposal is a brief written document (1-2 paragraphs) that articulates your team's initial project idea. This is an opportunity to get early feedback before investing significant time in implementation. Your proposal should demonstrate that you have thought carefully about what you want to explore and have a reasonable plan for doing so.
Your proposal should include:
- Research Question: A clear, well-scoped research question. Avoid overly broad questions (e.g., "What is the best model?") and instead focus on something specific and testable.
- Motivation: Why is this question interesting to your team? What do you hope to learn?
- Data: Initial ideas for what dataset(s) you will use. You don't need to have finalized this, but you should have identified candidate options.
- Approach: A high-level sketch of your experimental approach. What will you compare? What will you measure?
Rubric (3 points total)
| Criterion | Points | Full Credit |
|---|---|---|
| Research Question | 1.5 | Clear, specific, and appropriately scoped question that can feasibly be addressed in the project timeline |
| Methodology | 1.5 | General idea of experimental approach: what data, model, and comparisons you are considering |
2. Mid-Point Check-In (5% of Total Grade)
The mid-point check-in ensures your team is on track and allows us to provide feedback before the final submission. By this point, you should have made concrete progress on your implementation and have a clear plan for completing the project.
You will submit three components:
- Draft Poster: A preliminary version of your final poster with the structure and content outlined. This should include your research question, methodology description, and planned visualizations. Leave placeholders (e.g., "[Results Graph Here]") where your final results will go. The poster layout should be essentially complete—only the actual experimental results should be missing.
- Progress Summary: A short paragraph (approximately 150-250 words) that clearly states your methods: the model architecture you are using and why, the dataset you are using and any preprocessing steps, what experiments you plan to run, what results you expect to generate, and your planned analysis approach for interpreting the results.
- Related Works: A short paragraph describing the 3 most relevant related works. For each work, describe what it is in a sentence or two and how it relates to your project in a sentence or two.
Rubric (5 points total)
| Criterion | Points | Full Credit |
|---|---|---|
| Draft Poster Structure | 1 | Complete poster layout with clear sections; only results are missing; professional appearance |
| Research Question & Motivation | 1 | Poster clearly presents refined research question and explains why it is interesting |
| Methods Specification | 2 | Summary clearly identifies model architecture with justification, dataset with preprocessing details, planned experiments, expected outputs, and analysis approach |
| Related Works | 1 | Describes 3 relevant related works, what each work is, and how each relates to your project |
3. Final Poster and Presentation (12% of Total Grade)
The final poster is the culmination of your project work. You will create a polished poster that communicates your research question, methods, results, and conclusions. During the poster session, your group will stand by your poster and present your work to course staff and fellow students as they visit.
Poster Requirements:
- Clear statement of research question and motivation
- Description of experimental methodology (model, data, procedure)
- Results presented through clear, well-labeled visualizations
- Discussion of findings—what did you learn? Were there surprises?
Presentation Requirements:
- As visitors arrive, give a brief (~2 minute) walkthrough of your poster
- Be prepared to answer questions (~1 minute) about your methodology and results
- Every team member must contribute meaningfully to presenting and answering questions
Poster Content & Quality (7 points)
| Criterion | Points | Full Credit |
|---|---|---|
| Research Question & Motivation | 1 | Question is clear, well-motivated, and appropriately scoped |
| Visualizations | 2 | Clear, well-labeled figures that effectively communicate findings; appropriate choice of visualization type |
| Experiments & Analysis | 3 | Methods clearly explained; appropriate baselines/controls; thoughtful interpretation of results; reproducible |
| Poster Design & Clarity | 1 | Professional appearance; logical flow; visuals > text; easy to follow |
Presentation (5 points)
| Criterion | Points | Full Credit |
|---|---|---|
| Individual Contribution | 2 | Each team member presents a substantive portion and demonstrates understanding of the full project |
| Clarity & Organization | 2 | Logical flow; clear explanations; appropriate pacing |
| Q&A Response | 1 | Answers demonstrate understanding; honest about limitations; engages thoughtfully with questions |
Summary of Grade Breakdown
| Milestone | Points | % of Grade |
|---|---|---|
| Project Proposal | 3 | 3% |
| Mid-Point Check-In | 5 | 5% |
| Final Poster & Presentation | 12 | 12% |
| Total | 20 | 20% |
Important Reminders:
- We evaluate your process and understanding, not whether you achieved state-of-the-art results.
- Negative results are valid results—if your hypothesis was wrong, explain what you learned.
- All team members are expected to contribute equally. We reserve the right to adjust individual grades based on peer feedback.
- Start early! Research projects often encounter unexpected obstacles.