Optional:
Alan Turing’s classic paper “Computing machinery and intelligence” (aka “The imitation Game” paper) Mind LIX (1950): 433-460. Or, you could watch the movie: The Imitation Game ;-)
Chapter 1 of Artificial Intelligence: A Modern Approach, 4th ed. (S. Russell and P. Norvig)
Douglas Engelbart’s Augmenting Human Intellect: A Conceptual Framework. Stanford. 1963
Kittur et al. The Future of Crowd Work. CSCW 2013
Krishna et al. Embracing error to enable rapid crowdsourcing. CHI 2016
Hata et al. A glimpse far into the future: Understanding long-term crowd worker quality
Mary Gray et al. Ghost Work: How to stop Silicon Valley from building a new global underclass
Vaish et al. Crowd Research: Open and Scalable University Laboratories. UIST 2017
Gaikwad et al. Boomerang: Rebounding the Consequences of Reputation Feedback on Crowdsourcing Platforms. UIST 2016
Gaikwad et al. Daemo: a Self-Governed Crowdsourcing Marketplace. UIST 2017
For commentaries:
Ben Schneiderman and Pattie Maes. "Direct Manipulation vs. Interface Agents". Interactions CHI 1997 (an infamous debate about the role of AI agents vs. human-directed control)
Eric Horvitz’s Principles of Mixed-Initiative User Interfaces
Optional (no need for commentaries):
Eric Horvitz. Reflections on Challenges and Promises of Mixed-Initiative Interaction. AAAI Magazine 28, Special Issue on Mixed-Initiative Assistants (2007) (what will work in designing interactions with AI agents using interleaved actions by computers and people)
Optional:
Don Norman’s article titled “How Might People Interact with Agents”. 1997.
Ben Shneiderman’s Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Douwe Kiela et al. Dynabench: Rethinking Benchmarking in NLP. ACL 2021 (Iteratively creating benchmarks)
Jaemin Cho et al. DALL-EVAL: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Models. ArXiv 2022
Tuomas Kynkäänniemi et al. Improved Precision and Recall Metric for Assessing Generative Models. NeurIPS 2019
Zixian Ma et al. CREPE: Can Vision-Language Foundation Models Reason Compositionally? ArXiv 2022
Sharon Zhou et al. HYPE: Human eYe Perceptual Evaluation of Generative Models. NeurIPS 2019
For commentaries:
Saleema Amershi et al. Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine 2014 (A pre-deep learning era view of human interactions)
Ruyuan Wan et a. User or Labor: An Interaction Framework for Human-Machine Relationships in NLP. (How NLP researchers see human interactions)
Norman Makoto Su et al. The Affective Growth of Computer Vision. CVPR 2021 (How computer vision researchers are affected)
Optional:
Buçinca et al. Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems (We are measuring progress in explainable AI incorrectly)
Andres Campero et al. A test for evaluating performance in human-computer systems. ArXiv 2022 (Explanations don’t human-AI performance)
Vasconcelos et al. Explanations can Reduce Overreliance on AI Systems during Decision-Making. CSCW 2022 (A framework for understanding when explanations do help)
For commentaries:
Simon Kirby. Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language. PNAS 2008 (how structure emerges in language when people are asked to communicate)
Satwik Kottur et al. Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog. EMNLP 2017 short paper. (Training tricks to make structure emerge)
Ankit Vani. Iterated learning for emergent systematicity in VQA. ICLR 2021 (Cultural transmission used in a machine learning training protocol)
Optional:
Pranav Khadpe et al. Conceptual Metaphors Impact Perceptions of Human-AI Collaboration. CSCW 2020 (The words used to describe a model affects human behavior)
Ranjay Krishna et al. Socially situated artificial intelligence enables learning from human interaction. PNAS 2022 (Real world deployment where models learn from social human interactions)
More TBD
For commentaries:
Andrea Thomaz et al. Teachable robots: Understanding human teaching behavior to build more effective robot learners. AI 2008 (how do people want to provide feedback to robots)
Long Ouyang et al. Training language models to follow instructions with human feedback. ArXiv 2022 (InstructGPT)
Optional:
Joshua C. Peterson et al. Human uncertainty makes classification more robust. ICCV 2019.
Ruairidh M. Battleday et al. Capturing human categorization of natural images by combining deep networks and cognitive models. Nature Communications 2020.
Pulkit Singh et al. (2020). End-to-end deep prototype and exemplar models for predicting human behavior. Proceedings of the 42nd Annual Conference of the Cognitive Science Society.
Subjectivity in data collection.
For commentaries:
Mitchell Gordon et al. Jury Learning: Integrating Dissenting Voices into Machine Learning Models. CHI 2022
Aida Mostafazadeh Davani et al. Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations. TACL 2022.
Optional:
Aditya Ramesh. Zero-Shot Text-to-Image Generation. ICLR 2020 (DALL-E)
Kevin Frans et al. CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders. NeurIPS 2022 (Producing drawings)
David Bau et al. Semantic Photo Manipulation with a Generative Image Prior. SIGGRAPH 2019
Taesung Park et al. GauGAN: semantic image synthesis with spatially adaptive normalization. SIGGRAPH 2019
Francesca Gino et al. The Dark Side of Creativity: Original Thinkers Can Be More Dishonest. Journal of Personality and Social Psychology, 2012
Aaron Hertzmann. Can Computers Create Art? (Who is an artist?)
Controllable image generation.
For commentaries:
Chitwan Saharia et al. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding. ArXiv 2022 (Image generation)
Uriel Singer et al. Make-a-video: text-to-video generation without text-video data. (Video generation)