[Long read: 12mins]

The UX and Ethnography team joined the ACM Conference on Human Factors in Computing Systems (CHI), the most important conference in the field of Human-Computing Interaction. In this article we share our highlights and how we lived this intensive event.

This year the conference was held in the Scottish city of Glasgow, famous for the kindness of its people, the lively music scene of its bars and, like many places in Scotland, it’s not so great weather. Fortunately the weather didn’t stop the very motivated 3885 registered participants to join this edition, making CHI 2019 the biggest CHI conference ever. Over 1400 accepted papers presented in 23 parallel tracks forced our team to split and apply the “divide and conquer” strategy to get the most out of these busy days.

Georgia Tech GVU Center created this interactive visualization that allows you to explore the accepted papers and filter by topic. Among the top keywords mentioned we found VR, accessibility, AR and privacy. We were glad to see areas like accessibility and privacy at the center of the research agenda of the HCI community.

Between each paper session, we regrouped for coffee, shared our discoveries and tried some of the many demos displayed in the main hall. Ranging from VR systems augmented with thermal and olfactory feedback to experiments to test ethical conflicts in the design profession, as good researchers we couldn’t hide our curious side and we put our hands on every prototype we could find. Those breaks were just one of the many opportunities we had to meet and interact with the other attendants. Indeed, every evening there were several social events meant to facilitate networking, which was one of the most interesting elements of the conference.

Jutta trying a VR system that matches the physical and the virtual space to create a highly immersive experience.

Poster presentations of NAVER LABS Korea.

We also met with some of our Korean colleagues from Naver, Naver Labs Korea and Clova and discussed the work they presented at the conference:

• Turtle Talk: An educational Programming Game for Children with Voice User Interface by Hyunhoon Jung, Hee Jae Kim (presenter), Seongeun So, Junjoong Kim, Changhoon Oh.
• Tell Me More: Understanding User Interaction of Smart Speaker News Powered by Conversational Search by Hyunhoon Jung (presenter), Changhoon Oh, Gilhwan Hwang, Cindy Yoonjung Oh, Joonhwan Lee, Bongwon Seo.
• It Sounds Like a Woman: Exploring Gender Stereotypes in South Korean Voice Assistants by Gilhwan Hwang (presenter), Jeewon Lee, Cindy Younjung Oh, Joonhwan Lee.

All the korean colleagues from Naver. From the left, Hwang Gilhwan (Naver Clova), Sruthi Viswanathan, So Seogeun (Naver Line), Danilo Gallo.
From the right, Kim Hee Jae (Naver Line), Hyunhoon Jung (Naver Line/Clova), Shreepriya Shreepriya, Yesook Im (Naver Labs Korea), Kim Nam Yoon (Naver Labs Korea) and Jutta Willamowski.

Opening Keynote

Opening keynote by Dr. Aleks Krotoski.

The opening keynote for CHI 2019 was given by Dr. Aleks Krotoski, an academic and journalist who is best known for producing radio content and television documentaries (with a focus on technology – internet, digital media, privacy, etc.) for the BBC. She has a PhD in psychology, and while not really a member of the CHI academic community, has an interest in human machine interaction. Her talk was focused on what she considers three key challenges in the development of future human-machine interfaces:

• Attention (as a cognitive function), and how to capture and sustain it
• Plans, actions and tacit knowledge
• Emotions and emotional labour in human-robot interaction

She addressed these challenges using examples from her professional and personal life, for example discussing the impact of images on cognitive load and why producing content for radio and television requires completely different approaches.  In the case of plans and actions, she drew on her side business which is baking cakes, to discuss the relationship between following instructions (recipes) and a satisfying (as opposed to standard) outcome. The talk was very professionally delivered and engaging but the topics are quite familiar to a CHI audience and there was nothing especially challenging or original in the actual substance of the talk.

Workshop on Situationally Induced Impairments and Disabilities (SIIDs)

It all started early for us, attending one of the 35 workshops on the weekend before the beginning of the main conference. SIIDs is an area that studies temporary accessibility issues faced by users that can be triggered by contextual or personal conditions, i.e. wanting to send an SMS while driving or needing to pick up a call while holding a baby. This is a relevant topic for our team as we develop more natural and un-intrusive interfaces to provide navigation support for runners in unknown places.

During the workshop we reflected on topics such as the ethical considerations required when conducting user studies, the need to generate research parameters that can be replicated across studies and even to actually question the actual relevance of studying SIIDs and the opportunities of transference to the field of disabilities. We had the opportunity to discuss these topics and create connections with some of the most renowned figures in the field of accessibility, such as Jacob O. Wobbrock, professor at the University of Washington who received the 2017 ACM SIGCHI Social Impact Award for his work on accessible computing, Yeliz Yesilada, lecturer at the METU Northern Cyprus Campus and co-author of the book “Web Accessibility”, along with a motivated group of researchers that study the field around the world.

SIID Workshop Dinner

Sessions

With several simultaneous tracks, the feeling was that of always missing something interesting. On many occasions the rooms were full (a problem that should be addressed in future editions) and we had no other option than to follow the session outside via live-streaming until a spot would open or we would give up and attend a plan B session. Still, our strategic planning and the division of the sessions among the four of us helped us cover the most relevant sessions for our activity and get the most out of the conference. Next, you’ll find highlights of the most interesting presentations we attended, grouped by topic.

Accessibility and inclusion

• Varsha Koushik and Shaun K. Kane, from the UniversIty of Colorado Boulder studied the work of an association that successfully provides computer science education to people with cognitive disabilities (“It Broadens My Mind”: Empowering People with Cognitive Disabilities through Computing Education). Through field observations and interviews, they defined instructional strategies used by the teachers and the main challenges. Specifically, they recognised difficulties in reading and working with complex code, challenges related to the characteristics of the tool and the need to place visual aids on the physical space. However, the most interesting insight was how participants leverage their acquired skills to build solutions for themselves and, even more importantly, for others with disabilities. This shows the importance of empowering this population with the right skill sets to create solutions that answer their own needs.
• Sooyeon Lee et al. described challenges Uber’s Deaf or Hard of Hearing (DHH) drivers experience and how they address those difficulties via Uber’s accessibility features and their own workarounds. They used content analysis of in-app driver survey responses, customer support tickets, tweets and face-to-face interviews of DHH drivers to better understand the DHH driver experience. They discuss design and product opportunities to improve the DHH driver experience on Uber, specially focusing on improving the rider to driver connection by better supporting their communication through solutions such as real-time voice transcriptions.
• Robin Brewer and Vaishnav Kameswaran from the University of Michigan studied accessibility and transportation for people with vision impairments in ridesharing, an increasingly used mode of transportation (Understanding Trust, Transportation, and Accessibility through Ridesharing). They interviewed 16 visually-impaired individuals about their active use of ridesharing services like Uber and Lyft. Their findings show that, while people with vision impairments value independence, ridesharing involves building trust across a complex network of stakeholders and technologies which is easier to build in ridesharing than public transportation. This data is used to start a discussion on how other systems can facilitate trust for people with vision impairments by considering the role of conversation, affordances of system incentives and increased agency.
• Leandro Flórez-Aristizábal et al. from Institución Universitaria Antonio José Camacho, proposed a framework for the design of tools to support teaching children with disabilities. Developing educational tools aimed at children with disabilities is a challenging process for designers and developers because existing methodologies or frameworks do not provide any pedagogical information and/or do not take into account the particular needs of users with some kind of impairment. Their framework provides the necessary stages for the development of tools (hardware-based or software-based) and must be adapted to a specific disability and educational goal. For this study, their framework was adapted to support literacy for deaf people and the experts’ evaluation of the framework shows that it can be well adapted for other types of disabilities (DesignABILITY: Framework for the Design of Accessible Interactive Tools to Support Teaching to Children with Disabilities)

AI

• Quentin Roy et al. from the University of Waterloo explored the impact of controllability when facing inaccuracies in automated systems (Automation Accuracy is Good, but High Controllability is Better). They argue that automation does not need to be perfect (or as close as it can be) to be implemented and that investing in controllability should be considered. Through a 750-participant crowdsourced experiment using a controlled, gamified task they recognised that, when presented with high controllability, users self-reported satisfaction remained constant even under very low accuracy conditions. Besides user satisfaction, the system can benefit from human intervention by fine-tuning the results and providing feedback that can help improve it.
• The session on conversational interactions included a paper from Hao-Fei Cheng et.al. from University of Minnesota on Explaining Decision-Making algorithms through UI. Their online experiment results indicate that interactive explanations and white-box explanations (i.e. that show the inner workings of an algorithm) can improve users’ comprehension. However, understanding the algorithms did not affect the trust of the user!
• Maurice Jakesch et al. studied how perception and trustworthiness change when facing AI mediated communication. They compared the effect of displaying the same text description of an Airbnb profile between one presented as a human generated description and the other as an automatically generated description by labelling it with a specific tag. Although they didn’t recognise a difference in how people perceive them, they found that when there is the suspicion that the unlabelled description might have been automatically generated by the system, the trustworthiness was lower. They also touched on the topic of ethics, discussing how automated responses might perpetuate biases included in the training data that’s fed into the system.
• Carrie J. Cai et al. from Google Health and Google Brain presented their work on machine learning (ML). They focused on an application of ML that retrieves visually similar medical images from past patients (e.g. tissue from biopsies) to reference when making a medical decision with a new patient. Currently, no algorithm can perfectly capture an expert’s ideal notion of similarity for every case: an image that is algorithmically determined to be similar may not be medically relevant to a doctor’s specific diagnostic needs. In this paper, they identified the needs of pathologists when searching for similar images retrieved using a deep learning algorithm, and developed tools that empower users to cope with the search algorithm on-the-fly, communicating what types of similarity are most important at different moments in time. The tools were preferred over a traditional interface, without a loss in diagnostic accuracy. They also observed that users adopted new strategies when using refinement tools, repurposing them to test and understand the underlying algorithm and to disambiguate ML errors from their own errors.
• Kenneth Holstein et al. discussed the potential for ML systems to amplify social inequities and unfairness and the increase of popular and academic attention given to the subject. A surge of recent work has focused on the development of algorithmic tools to assess and mitigate such unfairness. However, if these tools are to have a positive impact on industry practice, it’s crucial that their design be informed by an understanding of real-world needs. Through 35 semi-structured interviews and an anonymous survey of 267 ML practitioners, they conducted the first systematic investigation of commercial product team challenges and needs for support in developing fairer ML systems. They identified areas of alignment and disconnect between the challenges faced by teams in practice and the solutions proposed in the fair ML research literature.
• Mirnig et al. discussed the trolley problem and its viability to help solve the questions regarding ethical decision making in automated vehicles. Automated vehicles have to make decisions, such as driving maneuvers or rerouting, based on environment data and decision algorithms. There is a question whether ethical aspects should be considered in these algorithms. When all available decisions within a situation have fatal consequences, this leads to a dilemma. Contemporary discourse surrounding this issue is dominated by the trolley problem, a specific version of such a dilemma. Based on an outline of its origins, they show that the trolley problem serves several important functions but is an ill-suited benchmark for the success or failure of an automated algorithm. They argue that research and design should focus on avoiding trolley-like problems at all rather than trying to solve an unsolvable dilemma and discuss alternative approaches on how to feasibly address ethical issues in automated agents. (Trolled by the Trolley Problem: On What Matters for Ethical Decision Making in Automated Vehicles).

Robotics

• Zhenhui Peng et al. propose a framework that models three levels of proactivity for service robots that provide decision-making support in public spaces. They conducted a within-subject experiment with 36 participants to evaluate the effects on user perceptions and behaviors. The study shows that highly proactive robots are perceived as inappropriate, the medium one was a good compromise because it helps reduce the decision space while maintaining users’ sense of engagement. Low, proactive ones are not able to communicate all their capabilities. Their work is an interesting first approach to defining frameworks that can guide the design of successful interactions with a technology that will become more integrated into our everyday lives.
• Jessy Ceha et al. analysed how a social robot’s verbal expression of curiosity is perceived in an educational context and how it impacts learning. They ran a between-subjects experiment where 30 participants played a game designed for teaching rock classification with a robot. The robot expressed different levels of curiosity. The study shows that the levels of curiosity expressed had an impact on the participants, producing both emotional and behavioural curiosity contagion effects in students. This is a specific element that can be leveraged to create more successful interactions between robots and users.
• Dmitry Dereshev et al. discuss that, despite ubiquitous visions of a robotic future, very few fully-fledged social robots are currently available to people. To improve their design, studies of their long-term use are particularly valuable, but are currently unavailable. To address this gap, they report on interviews with four long-term users of Pepper – a social robot introduced in 2014. Their thematic analysis elicited insights across three kinds of value Pepper brought to its users: utilitarian functionality; the community that formed around Pepper; and a personal value of affection. They focus on two contributions those values bring to social robot design: social robots as social proxies, alleviating disabilities or acting akin to social media profiles; and robot nurturing as a design construct, going beyond purely utilitarian or hedonistic perspectives on robots (for more info read Dmitry’s blog).

Recommender Systems

• Briane Paul V. Samson and Yasuyuki Sumi discuss how the recommendations of navigation applications are followed and what factors affect their adoption by presenting the results of a semi-structured qualitative study with 17 drivers, mostly from the Philippines and Japan. With the intention to circumnavigate congested roads, the route guidance always follows the basic assumption that drivers always want the fastest route. In their study, they recorded their daily commutes and occasional trips, and inquired into their navigation practices, route choices and on-the-fly decision-making. They found that, while drivers choose a recommended route in urgent situations, many still preferred to follow familiar routes. Drivers deviated because of a recommendation’s use of unfamiliar roads, lack of local context, perceived driving unsuitability, and inconsistencies with realized navigation experiences. Their findings and implications emphasize their personalization needs, and how the right amount of algorithmic sophistication can encourage behavioral adaptation.
• Yonggeol Jo et al. investigated the factors involved when a human judges the credibility of information to develop a classification model for weblogs, a primary source of information for many people. Considering both computational and human-centered approaches, they conducted a user study designed to consider two cognitive procedures–(1) visceral, behavioral and (2) reflective assessments–in the evaluation of information credibility. They used Naver blogs as the credible source of information. The results of the 80-participant study highlight that human cognitive processing varies according to an individual’s purpose and that humans consider the structures and styles of content in their reflective assessments. They experimentally proved these findings through the development and analysis of classification models using 16,304 real blog posts written by 2,944 bloggers.
• Daniel Trielli et al. presented an algorithm audit of the Google Top Stories box, a prominent component of search engine results and powerful driver of traffic to news publishers. As such, it is important in shaping user attention towards news outlets and topics. By analyzing the number of appearances of news article links they contribute a series of novel analyses that provide an in-depth characterization of news source diversity and its implications for attention via Google search. They present results indicating a considerable degree of source concentration (with variation among search terms), a slight exaggeration in the ideological skew of news in comparison to a baseline, and a quantification of how the presentation of items translates into traffic and attention for publishers. They contribute insights that underscore the power that Google wields in exposing users to diverse sources of news, and raise important questions and opportunities for future work on algorithmic news curation.

UX Practitioners vs UX researchers

We attended a UX Event that focused on the disconnection between practitioners and researchers and explored ways to bridge the gap between the two communities. Leading UX practitioners and researchers like Stuart Reeves from Mixed Reality Lab, University of Nottingham, presented their findings on “Studying Practitioners’ Work”. There was an emphasis on the fact that research in practitioners’ work is valuable to the HCI community. . For example, some of the ideas proposed the creation of new role, a facilitator that can mediate between the two groups and foster dialogue, or the creation of spaces where both groups can collaborate towards a common goal and build understanding during the process.

Closing Keynote

“Can the world be your interface?”- envisioned Ivan Poupyrev.

The closing keynote was given by Ivan Poupyrev, Director of Engineering and Technical Projects Lead at the Google’s Advanced Technology and Projects (ATAP). He has over 20 years of experience leading invention, development and productization of breakthrough technologies such as the internet of things, VR/AR and haptic interactions. During the talk, he focused on the need to break from the limited interactions confined to our digital devices and proposed a vision in which “the world is the interface”. He aims to generate a future where interfaces allow us to interact with technology in a more natural way, taking advantage of the physical objects that surround our everyday life.

To illustrate this he presented the work he is currently developing at Google, his most recent explorations in creating a pico-radar sensor for touchless gesture interaction (Project Soli), and designing a platform for manufacturing interactive, connected soft goods at scale (Project Jacquard). In fact, he wore the Levi’s Commuter Jacket produced during this project to fashionably control his presentation, casually touching his left arm to go through the slides (though we must say he faced some technical difficulties that forced him to rely on the old clicker!) Even though many questions remain about the actual usefulness and potential problems these kind of products can present, the talk was inspiring and invited us to look into the future. It was interesting to see the approach he took to develop this product, embedding the technology in the very raw material used to produce the jackets, aiming to enable non technical producers to enter the field of smart garments and thus change the way we interact with the world at a large scale. (Keynote video on YouTube)

Sruthi, Shreepriya, Jutta, Tom and Danilo from NAVER LABS Europe and Nam (upfront) from NAVER LABS Korea.

This keynote marked the end of CHI2019 and of our visit to Glasgow. CHI was a great platform to learn more about current research areas popular in the HCI community and a great place to create connections and learn from one another.