Course Project

The main expected outcome of the course is a good research project that makes a reasonable contribution to the field. Some possibilities include (but not limited to):

  • Design new reinforcement learning or imitation learning methods that are safer/more sample efficient/more interpretable
  • Design novel meta-learning or transfer learning methods that can allow for quick learning of novel tasks
  • Design human in the loop reinforcement learning algorithms for novel tasks
  • Implement existing state of the art algorithms on a set of novel tasks and show unexpected behavior/emergent behavior
  • Perform a thorough experimental evaluation/reproduction of a class of learning methods for robotics
  • Show the ability of learnning methods to show superior generalization and adaptation in robotics problems
  • Collect a new dataset and show how this may enable novel robotic learning applications
  • Design multi-task and continual learning systems that can ingest large amounts of data and show continual improvement
  • Show a novel theoretical result on a new/existing algorithm, shedding light on where/when it can work.
  • Show simulation to reality transfer on a new and challenging set of robotics tasks
  • Show the use of large scale pretrained/foundation models on robotics problems
  • Show how learning can help with classic pipelines in robotics such as TAMP, MPC, motion planning
  • Develop offline RL / transfer learning methods that can learn from diverse/heterogeneous sources of data
  • Exploring RL applications in domains like robotics, vision, NLP, smart grids, traffic, finance, biology etc.
  • Contributing highly stable/performant implementations of recent algorithms that reproduce results from the papers.

Submission

Date     Due
Jan 23 Project Proposal (1-2 pages)
Feb 15 Milestone (3-4 pages)
Mar 13 Final report (8 pages max)
Mar 13 Final presentations

At the conclusion of the project, your team will be responsible for writing a short research paper that summarizes the project (6-8 pages not including references). The project proposal and reports should use the template of the NeurIPS conference paper.


Writing the Research Project Paper

The project proposal should be a first draft of your introduction section of your final paper. It should try to answer the following questions:

  • Why did you choose this problem, why is it important?
  • Describe the state of related work and why it is insufficient.
  • Introduce your unique insight to tackle the problem or research question.
  • Articulate the technical challenges you are likely to encounter and potential ideas to try (need not be tried yet)
  • Plan out the experiments that justify the utility of the insight or answers the question,

The project milestone should be a draft of your final course paper. It should include all sections except for the experiments section, which will be incomplete.

If this is your first time writing a research project paper, here is a rough outline of sections that we recommend for your final report:

  • Abstract (summarize the project’s motivation, methodology, and main findings),
  • Introduction (explain why the project matters, lay out the primary innovation, lay out the findings, and end with a broader implications of your results.),
  • Related work (contextualize your research amongst its closest related research projects),
  • Methods (describe the process you used to conduct your research),
  • Experiments/Study (explain your experimental or study setup and its main findings),
  • Discussion (How do your findings can be interpreted and what others should take away from your project).

Final Presentation

We will have a final project presentation.

We have around 18 groups with 90 minutes. Each group can do a 4 minute presentation (hard cap) with 1 minute for questions. The presentation should cover the motivation, methodology, and results of your project. No other restrictions really, just tell us what you did and why it matters.

Please put your group slides in this deck.