Ethics Review Assignment
Purpose
- Give you concrete experience thinking about ethical implications of ML/NLP work
- Help you check yourself in understanding the kind of thing you (and others) might miss when thinking about ethical considerations
- Train you as a qualified ethics reviewer!
What you will do
You will play the role of an ethics reviewer. First, read the following documents:
- NeurIPS 2021 Ethics Guidelines
- NeurIPS 2021 Ethics Guidelines blog post
- NeurIPS 2021 Ethics Reviewing Retrospective blog post (optional)
Then choose one of the 5 provided machine learning papers to review. These papers have all been published, but you should not look them up. (If you have read one of them before, do not choose it for this assignment.) We have provided copies of the papers with all discussion of ethical concerns obfuscated out. You will review the paper as though these sections are entirely missing.
As a rough guide, you might consider the following questions:
- What are the potential negative societal impacts of the work?
- What might be some strategies to mitigate these impacts?
- Does the paper discuss any of these strategies?
- Does the research described in the paper meet ethical standards of research conduct?
- Is there enough information provided in the paper to assess this?
- If not, what information is missing?
You will be graded on careful consideration of the ethical issues relevant to your chosen paper. There is no specific length requirement; expect to write around 300 words, but it may be more if there are many issues. As examples, you can refer to these papers to read some ethics reviews and the ensuing discussions with authors:
- Liang et al. (2021): Exploring Forensic Dental Identification with Deep Learning
- Lim et al. (2021): Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methods
Turnin
You can turn in your review in .txt, .md, or .pdf format. Please upload it in this Google Drive folder, in the subfolder corresponding to your chosen paper, by 11:59pm on Feb 24.
After turning in your review, feel free to look at others’ reviews for comparison and to check your work. After the due date we will post links to the NeurIPS reviews so you can also check your review against the official ones.