How to Build Ethics into Robust Artificial Intelligence (2017-2018)

The goal of this project was to combine computational methods, philosophy, game theory and psychology to develop robust and ethical moral artificial intelligence (“moral AI”) to direct autonomous agents.

The project team’s research address several of the main challenges in designing moral AI through three subprojects.

  1. The team used in-person experiments and online platforms such as Amazon Mechanical Turk (MTurk) to ask people what types of factors they believe kidney exchange algorithms should and should not take into account. Team members built a website for large-scale data collection for online participants, analyzed participants’ responses to find common ethical themes and then revised current kidney exchange algorithms so that their recommendations of how kidneys should be allocated will take these ethical themes into account. This “bottom-up” approach relied primarily on new data to learn the moral features that should be incorporated into AI algorithms.
  2. The team also combined principles from moral philosophy and economic game theory to design scenarios and games that asked participants to judge the actions described or displayed in specific moral situations. Team members began by focusing on the “trust game” and had MTurk participants play the trust game against one another and then report how morally wrong or acceptable they thought their actions and their partner’s actions were. Team members collected data, began analysis on performance and used participants’ morality ratings to refine game theoretic notions so that they account for behavior in the trust game more accurately. This “top-down” approach relied primarily on game theory and ethical theory to generate algorithms that can make ethical choices.
  3. Team members also asked participants whether they thought humans or computers should be making decisions in a wide range of scenarios and applications.


Fall 2017 – Spring 2018

Team Outcomes

Adapting a Kidney Exchange Algorithm to Align with Human Values (paper by Rachel Freedman, Jana Schaich Borg, Walter Sinnott-Armstrong, John P. Dickerson, Vincent Conitzer)

Adapting a Kidney Exchange Algorithm to Align with Human Values (poster by Rachel Freedman, Jana Schaich Borg, Walter Sinnott-Armstrong, John P. Dickerson, Vincent Conitzer), presented at Bass Connections Showcase, April 18, 2018

Team website

Who Should Get the Kidney (game aligned with kidney exchange algorithm)


Cultures of Collaboration: Managing the Moral AI Lab (Kenzie Doyle)

This Team in the News

These Ph.D. Graduates Incorporated Bass Connections into Their Doctoral Education

Should AI Decide Who Gets a Kidney?

Finding Your Way: Cultivating Humanistic Versatility

Jana Schaich Borg on Duke, Data and MIDS

Duke Professor Advances to Final in New Yorker Caption Contest

See related team, Moral Artificial Intelligence (2018-2019). This project was selected by the Franklin Humanities Institute as a humanities-connected project.

Team Leaders

  • Vincent Conitzer, Arts & Sciences-Computer Science
  • Jana Schaich Borg, Social Science Research Institute
  • Walter Sinnott-Armstrong, Arts & Sciences-Philosophy

/graduate Team Members

  • Cassandra Carley, Interdept Comp Sci/PPS (BS)
  • Lok Chan, Philosophy-PHD
  • Abbas Zaidi, Statistical Science - PHD, Statistical Science - MS

/undergraduate Team Members

  • Anika Mukherji, Interdept Neuro/Comp Sci (BS)
  • Weiyao Wang, Mathematics (BS), Computer Science (BS2)

/yfaculty/staff Team Members

  • Emmanuel Chevallier, Arts & Sciences-Statistical Science
  • Kenzie Doyle, Duke Institute for Brain Sciences
  • Joshua Skorburg, Fuqua School of Business
  • Siyuan Yin, Arts & Sciences-Philosophy

/zcommunity Team Members

  • John Dickerson, University of Maryland