October 27, 2023: Multimodal Human-AI Interaction
Snehesh Shrestha, PhD Candidate, The department of Computer Science, University of Maryland College Park
People communicate through verbal and non-verbal cues. AI and ML have made tremendous progress in language understanding. Audio tone, gestures, gaze, and touch, along with speech, offer new challenges and opportunities. My work dissects multimodal human expression, focusing on Human-AI interaction in Robotics and Music. In the first part, I discuss creating a robot capable of understanding natural commands, emphasizing multimodal repair mechanisms. I’ll briefly share data collection challenges, which greatly impact data quality and validity. We used a Wizard-of-Oz setup, deceiving participants into believing we had a human-level AI robot, to capture ‘natural’ interactions. Verbal and non-verbal strategies were studied to train machine learning algorithms for multi-modal commands, highlighting the importance of combining gestures with speech. In the second part, I explore AI-mediated Student-Teacher Interaction systems towards violin education. I will discuss challenges in remote music lessons, which became particularly pronounced during the COVID-19 pandemic. I will discuss data collection challenges for precise motion capture, especially with young students. I share insights into using audio to enhance pose estimation algorithms for 3D player visualization. Lastly, I introduce a novel haptic band designed for remote feedback, prompts, and metronome functions, enhancing online music education experiences.
Snehesh Shrestha is a Ph.D. candidate at the University of Maryland College Park. He works in the Perception and Robotics Group (PRG) lab in the Department of Computer Science under the guidance of Prof. Yiannis Aloimonos (CS), Dr. Cornelia Fermüller (UMAICS), Dr. Ge Gao (INFO), and Dr. Irina Muresanu (School of Music). He has also worked with Dr. Michelle Gelfand (Department of Psychology) in the Culture Lab. Additionally, he works at NIST, developing new standards towards recommended practices for the design of human subject studies in human-robot interaction. His research is at the intersection of robotics, artificial intelligence, human factors, arts, and culture. He is interested in multidisciplinary research aimed at building rich and intuitive experiences that ‘amplify human abilities, empowering people and ensuring human control’ inspired from Dr. Ben Shneiderman’s Human-Centered AI book. His recent work has focused on human-robot interaction and AI for music education.
October 20, 2023: Conflict and Compromise among Museum Exhibit Teams: The Impacts of Organizational Change and Professionalization on Curating Smithsonian’s Fossil Halls
Diana E. Marsh, Assistant Professor of Archives and Digital Curation, University of Maryland, College Park
In this talk, I will highlight interdisciplinary teamwork behind large-scale exhibitions and the politics among different experts who curate scientific knowledge for the public. Presenting highlights from my book, Extinct Monsters to Deep Time: Conflict, Compromise, and the Making of Smithsonian’s Fossil Halls, I describe participant observation among the Smithsonian’s exhibition team tasked with the National Museum of Natural History (NMNH)’s largest-ever exhibit renovation, Deep Time. I highlight how process of negotiating, planning and designing scientific knowledge in exhibits is shaped by the intersections of different expertises involved in the planning process—including Education, Design, Exhibit Writing, Project Management, and three subfields of Paleobiology—as well as broader institutional cultures and pressures. Drawing on ethnographic fieldwork as well as interview, oral history and archival research, the work contextualizes the contemporary exhibits process by tracing trends in exhibit development from late-19th century to the present. I show how telling the story of the Deep Time is mediated through 1) different techniques and technologies for museum communication, 2) the recent professionalization of museum disciplines, and 3) the expanding institutional split between the museum’s missions of “research” and “outreach,” leading to new “frictions” and “complementarities” among exhibit teams.
Diana E. Marsh is an Assistant Professor of Archives and Digital Curation at the University of Maryland’s College of Information Studies (iSchool) who explores how heritage institutions communicate with the public and communities. Her current research focuses on improving discovery and access to colonially-held archives for Native American and Indigenous communities. Previously, she completed her PhD in Anthropology (Museum Anthropology) at the University of British Columbia, an MPhil in Social Anthropology with a Museums and Heritage focus at the University of Cambridge in 2010, and a BFA in Visual Arts and Photography at the Mason Gross School of the Arts of Rutgers University in 2009. Her recent work has appeared in The American Archivist, Archival Science, Archivaria, and Archival Outlook, and her book, From Extinct Monsters to Deep Time: Conflict, Compromise, and the Making of Smithsonian’s Fossil Halls was released in paperback with Berghahn Books in Fall 2022.
September 23, 2023: Virtual Team Creativity and Innovation
Roni Reiter-Palmon, Distinguished professor of I/O Psychology, University of Nebraska at Omaha
As communication technology capabilities have improved and the globalization of the workforce has resulted in distributed teams, organizations have been shifting towards virtual teams and virtual meetings over the last decade. This trend has been accelerated with current work-from-home orders due to COVID-19. Even though virtual collaboration has, in the past, been the focus of multiple studies, there are some surprising gaps in our knowledge. For instance, there are few empirical studies examining the impact of virtual devices and tools on creative problem-solving. While there is a substantial body of research on electronic brainstorming and the use of virtual tools for idea generation, less is known about earlier processes such as problem construction or later processes such as idea evaluation and idea selection. Furthermore, as a dynamic process, creativity and innovation is heavily influenced by the people engaged in the process and their collaborative environment, yet there is a gap in the literature regarding the type of virtual tools used in the process (for example, audio + video vs. audio alone, or the use of file-sharing technologies). In this paper, we will review the current literature on virtual teams, virtual meetings, and creativity. We will then explore theoretical frameworks such as media richness theory that can help us understand how virtuality and virtual tools may influence team creativity across the dynamic range of the creative problem-solving process. Finally, we provide questions to help guide future research.
Dr. Roni Reiter-Palmon is a Distinguished Professor of Industrial/Organizational (I/O) Psychology at the University of Nebraska at Omaha. She is also the Director of Innovation for the Center for Collaboration Science, an inter-disciplinary program at UNO. Her research focuses on creativity and innovation in the workplace, cognitive processes of creativity, team creativity, development of teamwork and creative problem-solving skills, and leading creative individuals and teams. Her research has been published in leading journals in I/O psychology, management, and creativity. She is the former Editor of The Psychology of Creativity, Aesthetics and the Arts and the current editor of Organizational Psychology Research. She serves on multiple editorial boards of I/O, management, and creativity journals. She has obtained over 9 million dollars in grant and contract funding focusing on creativity, leadership, and teams. She is a fellow of Divisions 10 and 14 of APA, and has won the system wide research award from the University of Nebraska system in 2017.
April 28, 2023: Building Human-agent Teams in Multi-agent Systems
Dr. Susan Campbell
Prior work on the interaction of humans and autonomous systems has focused on
human control of systems. Though meaningful human control is imperative, only some
humans will exercise control over systems. Other humans will act as teammates,
sharing goals and interdependence with systems that they perceive to be autonomous.
Unfortunately, current systems have a long way to go before they can perform the
behaviors that would make them effective team members. The Artificial Intelligence and
Autonomy for Multi-Agent Systems (ArtIAMAS) human-machine teaming area seeks to
bridge the gap between what is currently possible with autonomous systems and future
human-machine teaming concepts. In this talk, I will discuss our goals and current
Dr. Campbell is interested in what makes people good at interacting with complex
technologies and technological systems. She holds a joint appointment between the
Applied Research Laboratory for Intelligence and Security (as an Associate Research
Scientist) and the College of Information Studies (as a Senior Lecturer) at the University
of Maryland. She is the university lead for human-machine teaming on the ArtIAMAS
cooperative agreement. Her work focuses on measuring individual differences related to
cognitive performance, training cognitive skills, and building systems that complement
human strengths. She holds a PhD in Psychology from the University of Maryland at
College Park and a BS in Cognitive Science from Carnegie Mellon.
March 17, 2023: Human-Machine Teaming: What Skills Do the Humans Need?
Dr. Samantha Dubrow, Lead Human-Centered Engineering Researcher, The MITRE Corporation
Over the last few decades, technology has become increasingly intelligent. Technology is no longer a passive tool that supports a single human in their work, but an active teammate that collaborates and learns as a critical entity of the team. To date, human-machine teaming research has primarily focused on the machines – how to design them, what their capabilities are, and how they can “learn.” This presentation takes the opposite view, focusing on the importance of selecting and training humans to be effective human-machine teammates. The presentation addresses two questions: What unique skills do humans need to work well with machines as teammates, and how are those skills different from those required for effective human-human interactions? Details about the human traits and abilities that can be selected for and the human skills that can be trained to maximize human-machine teaming effectiveness will be discussed.
Dr. Samantha Dubrow is a Lead Human-Centered Engineering Researcher at The MITRE Corporation. At MITRE, Samantha conducts applied research and development in Industrial-Organizational Psychology, teamwork and leadership, hybrid teaming, decision-making, human factors, user experience, human-machine teaming, and multiteam system collaboration management. She helps teams and multiteam systems across a variety of government agencies utilize technology to improve their teamwork processes and job performance. Samantha holds a PhD in Industrial-Organizational Psychology from George Mason University under Dr. Stephen Zaccaro. Her dissertation focused on team mental models and leadership transitions in ad hoc decision-making teams. During graduate school, Samantha was also involved with projects regarding multidisciplinary teams, multiteam systems, team leadership, simulation and training, and social network analysis.
February 24, 2023: Promoting Astronaut Autonomy in Human Spaceflight Missions (Seminar)
Dr. Jessica Marquez, NASA, Ames Research Center
Mission operations will have to adapt for long duration, long distance human spaceflight missions. This change is driven mainly by the significantly different communication availability between Earth and space. As astronauts travel farther from Earth, the one-way communication latency increases; the amount of bandwidth will be limited; and there will be period of long and/or no communication. Currently, ground flight controllers collaborate and cooperate with astronauts in space to accomplish essential operational functions. Astronaut autonomy, i.e., the crew’s ability to work more independently from mission control, will be a key enabler in future exploration missions. Over the last several years, the NASA Ames Human-Computer Interaction (HCI) Group has investigated various ways to promote and support astronaut autonomy in human spaceflight missions. Software prototypes are researched, designed, implemented, and assessed for their ability to enable astronaut autonomy. From integrated Internet of Thing for Space, advanced procedures interfaces, comm-delayed chats, and self-scheduling tools, the HCI Group has explored different aspects of astronaut autonomy. Specifically, the self-scheduling tool Playbook has been evaluated in analog extreme environments and onboard the International Space Station, successfully paving the way for future autonomous astronauts.
Short Bio: Since 2007, Dr. Jessica Marquez has been working at the NASA Ames Research Center within the Human Systems Integration Division. As part of the Human-Computer Interaction Group, she has supported the development and deployment of planning and scheduling software tools for various space missions, including the International Space Station Program. She now leads the team that is developing Playbook, a web-based planning, scheduling, and execution software tool. Her work has led to supporting different NASA analog missions that simulate planetary missions and spacewalks. Dr. Marquez also is a subject matter expert for space human factors engineering, specifically in human-automation-robotic integration. She lends her expertise across different NASA research programs, like the Space Technology Research Institutes and the Human Research Program. She currently is the PI for the research project “Crew Autonomy through Self-Scheduling: Guidelines for Crew Scheduling Performance Envelope and Mitigation Strategies.” Dr. Marquez has a Ph.D. in Human Systems Engineering and S.M. in Aeronautics/Astronautics from the Massachusetts Institute of Technology and a B.S.E. in Mechanical Engineering from Princeton University.