2nd ICML 2020 Workshop on Human in the Loop Learning

Recent years have witnessed the rising need for learning agents that can interact with humans. Such agents usually involve applications in computer vision, natural language processing, human computer interaction, and robotics. Creating and running such agents call for interdisciplinary research of artificial intelligence, machine learning, and software engineering design, which we abstract as Human in the Loop Learning (HILL). HILL is a modern machine learning paradigm of significant practical and theoretical interest. For HILL, models and humans engage in a two-way dialog to facilitate more accurate and interpretable learning. It includes active learning, where the learners choose which examples to label and achieve better accuracy with less data compared to the classical approach of passively observing labeled data; It also includes the explainable learning, where the human doesn't merely tell the machine whether its predictions are correct, but provides reasons in a form that is meaningful to both parties.

The workshop aims to bring together researchers and practitioners working on the broad areas of human in the loop learning, ranging from the interactive/active learning algorithm designs for real-world decision making systems (e.g., autonomous driving vehicles, robotic systems, etc.), models with strong explainability, as well as interactive system designs (e.g., data visualization, annotation systems, etc.). It provides an opportunity for scientists to share their perspectives, discuss solutions to these questions, and highlight the challenges in the field to help guide future research. In particular, we aim to elicit new connections among these diverse fields, identifying theory, tools and design principles tailored to practical machine learning workflows. In this year’s HILL workshop, we emphasize on the interactive/active learning algorithms for real-world decision making systems as well as learning algorithms with strong explainability. We continue the previous effort to provide a platform where researchers can discuss approaches that bridge the gap between humans and machines and get the best of both worlds.

July 18

Virtual Conference

Speakers

FEATURED TALKS

Tom Griffiths

Professor at Princeton University

Raquel Urtasun

Professor at University of Toronto

Chelsea Finn

Assistant professor at Stanford University

Richard Zemel

Professor at University of Toronto

Wenwu Zhu

Professor at Tsinghua University

Sergey Levine

Assistant professor at UC Berkeley

Kalesha Bullard

Christian Lebiere

Pradeep Ravikumar

Postdoc Researcher at Facebook AI Research (FAIR)

Research Scientist at Carnegie Mellon University

Associate professor at Carnegie Mellon University

Anca Dragan

Assistant professor at UC Berkeley

Zeynep Akata

Professor at

University of Tübingen

Agenda (TBD)

July 18

Call for Papers

We welcome high-quality submissions on algorithms and system designs in the broad area of human in the loop learning. A few (non-exhaustive) topics of interest include:

Active/Interactive machine learning algorithms for autonomous decision-making systems,
Online learning and active learning,
Psychology driven human concept learning,
Explainable AI,
Systems for online and interactive learning algorithms,
Systems for collecting, preparing, and managing machine learning data,
Design, testing and assessment of interactive systems for data analytics,
Model understanding tools (debugging, visualization, introspection, etc).

These topics span a variety of scientific disciplines and application domains like machine learning, human-computer interaction, cognitive science, and robotics. It is an opportunity for scientists in these disciplines to share their perspectives, discuss solutions to common problems and highlight the challenges in the field to help guide future research. The target audience for the workshop includes people who are interested in using machines to solve problems by having a human be an integral part of the learning process.

We invite submissions of full papers, as well as works-in-progress, position papers, and papers describing open problems and challenges. While original contributions are preferred, we also invite submissions of high-quality work that has recently been published in other venues or is concurrently submitted. We encourage creative ML approaches, as well as interdisciplinarity and perspectives from outside traditional ML. Papers should be 4-8 pages in length (excluding references) formatted using the ICML template. All the submissions should be anonymous. The accepted papers are allowed to get submitted to other conference venues.

Papers can be submitted through CMT:

https://cmt3.research.microsoft.com/HILL2020

Important Dates

Submission deadline: 25th June 2020 (23:59 AoE)

Acceptance notification: 10th, July 2020 (23:59 AoE)

Workshop Date: 18th July