Ambient Intelligence needs human understanding to be usable, useful and ethical

Ethnography has become an established methodology in designing technology for people.

Invited to speak at an “Innovation and Design” symposium [1] a few months ago, I decided to share how user centric methods, and ethnography in particular, have changed since the Web was invented at the end of the 80s. Its appearance coincided with the end of my studies in computer science and the beginning of a new life where I’d be involved in developing technology to be put into the hands of thousands of people – an exciting prospect but one that also profoundly disturbed me. How could I create tools that, for the most part, would be used in the workplace when I myself had absolutely no idea of how work actually worked?

In an attempt of self-reassurance, I told myself there would be specs, requirements and other documentation I could put my hands and, even if they were a pretty weak substitute for real, firsthand experience, they’d be of some help at the beginning.

Then, in the midst of this turmoil and doubt, I bumped into a professor at university, Giorgio De Michelis, who  was among the founding members of a community of researchers who were arguing for and practicing design and development in a very different way. They were collaborating with anthropologists and sociologists specialized in observing and reporting in great detail the way people work and live. This community, which was just coming together, was based on the approach being promoted at Xerox PARC, the cradle of most modern computing that emerged in the 70s and 80s. It was there that Lucy Suchman started a project to assess why a new, very user-friendly-designed office copier, was proving to be difficult to use [2,3]. This copier was meant to open a new era of machines, where the “casual user” could simply walk up to it and easily put it in motion to get their job done with no prior knowledge. A number of user interviews and site visits didn’t shine any light on the problem so Suchman suggested they install one in the lab and record how successfully their very smart researchers managed to use it.  By observing the persistent troublesome interactions that the group had with the technology, she was able to support the refinement of the expert system embedded in the user interface of the copiers because it was the UI that was at heart of the problem. She made the point that, not only did the fault not lie with the users but rather that rules, scripts and interfaces are but a guide to action. It’s only by observing their use in practice and understanding their limitations and assumptions, that one can realistically bridge the gap between the plan and reality. No evidence could have been more powerful than the narrative conveyed by the Suchman video recordings of the ‘human struggle with the machine’.

The Suchman episode brought new found legitimacy to human sciences in Human Computer Interaction (HCI). Anthropology and sociology were even applied to the more mundane daily activities at work and ethnography became a method for studying not only foreign, far away or sub-urban cultures, but also ones very close to us. The current legitimacy of the approach in industry is witnessed in conferences like EPIC [4] and by a number of university curricula which have integrated ethnography in fields like Computer Science and Industrial Design.

Ethnography literally means “writing about the people”. It’s a qualitative method where people are trained to spend time in the field to observe activities as they occur and with no pre-set objectives or bias. It’s only afterwards, in the lengthy analysis phase, that the notes, videos, recordings and artefact examples reveal insights. The material is typically rich providing different levels of analysis. After many years in working with this approach, I can say with confidence that there are three levels of analysis that the researcher can use to focus the analysis, depending on where they want to intervene. These levels are usability, human-machine configurations [3] and ethical issues.

The usability level is exemplified well by Ellen Isaacs in her talk at TEDxBroadway [4]. An ethnographer with fresh eyes can bring to the surface the hidden obvious i.e. they can pinpoint things in environments that people are blind to simply out of familiarity.  To illustrate this, Issacs quotes the cryptic street parking signs in the USA. These signs seem to have looked like that forever when in fact they could quite easily be a whole lot easier to understand.

Parking sign, USA

“It’s 5pm. Can I park here?”

Usability is the first stage, where the analysis remains at the level of direct, immediate interaction with the artefact but it’s often more interesting to broaden the scope and move to the socio-technical level, where the interplay of technologies and social activities is studied.

At the end of the eighties there was a lot of hype around paper documents and how they were going to disappear, forever, and altogether, very soon. In this context, a seminal study was undertaken to observe the reality of how paper documents were used by knowledge workers in the financial sector [5]. An examination of employee activities while carrying on their work, revealed a number of features of paper documents some of which related to their usability whilst others referred to the socio-technical organization of work. Features that were observed include how easy it is to annotate a hard copy but also how they support the mental processes related to conceptually understanding and organizing content. These types of observations concerning paper documents are important when addressing the usability aspects of digital ones such as e-readers. But these studies started to go beyond usability, as the researchers observed that the affordances of paper documents were also functional to social activities and contextual communication i.e. personally handing a document to a colleague would ensure that the priority and urgency of it be understood highlighting a practice that email does not yet support in the same way.

This pioneer study of the so called “paperless” office, paved the way for many more. In my team we examined public administration processes [7] where the company management wanted to reduce print volume (referred to in terms of football fields full of documents) on the assumption that it was due to “bad habits” on the side of the employees. The study showed that there was effectively a massive duplication of paper and digital records in place but that was to ensure that every employee dealing with a case had all the information they needed to be able to process it. As there was no common electronic digital record for reference, the only way to trust “the system” was to create a copy.

A paperless office

Finally, ethical analysis, at the third level is when existing socio-technical set ups may be questioned with respect to societal values. A good example of this is the outcome of ethnographic work on microtask crowdsourcing. A study to evaluate if low skill business processes, like the digitalization of hand written forms, could be done more cheaply and conveniently though crowdsourcing [7] revealed that, what was assumed to be a fairly straightforward process, was actually pretty complex as it had to ensure a certain level of security, quality and turnaround. But the results also uncovered a system where working conditions lie below the standards established by law in western European countries. It unveiled a world of low pay, little trust and complete lack of interest in the professionalization of the workforce or even in establishing the basics of a healthy working relationship [10]. Beyond the ethical questions, these may be mandatory requirements in standard contracts even for low skill processes in terms of labour conditions. The problem is that this type of work is still in a grey zone when it comes to the enforcement of different national legislation in terms of protection of worker rights and could put both the company and the workers at risk.

Today ethnography has become almost a routine element of the UX designer’s toolbox. As a method, it can equally well support the creative process that leads to new services and the many iterations required to adapt technology to actual practices. Far too often technology fails to live up to its promise of innovation. In the best case it’s abandoned, in the worst case it may actually be harmful and there is growing concern around technology and the impact it has on our lives i.e. how the “attention economy” is hijacking our lives [11] or the new loneliness that technology is inducing [12, 13].

As we create the future of intelligence in our ambient surroundings, all three levels of usability, socio-technical interconnections and ethical issues need to be well understood. How usable is a system that always needs our attention? How well understood are our social needs across a range of family, work and leisure set-ups? What is our responsibility when designing systems that, to be effective, need massive amounts of personal data?

These questions and others are central to Ambient Intelligence which is why, as ethnographers and designers, our mission is to ground LABS technology in a strong understanding of human activities.

————-

About the author: M. Antonietta Grasso leads the UX and ethnography team at NAVER LABS Europe.

————-

REFERENCES

1. Design and Innovation symposium. University of Milano Bicocca, 27 October 2017.
2. Video about photocopier use at PARC [link]
3. Lucy Suchman keynote at CHI’10 conference. [link]
4. EPIC Conference [link]
5. Eileen Isaacs TedX Broadway presentation, Ethnography [link]
6. Abigail J. Sellen and Richard H.R. Harper. 2003. The Myth of the Paperless Office. MIT Press, Cambridge, MA, USA.
7. O’Neill, J., Grasso, A., Willamowski, J.: RFID: enhancing paper documents with electronic properties. In: CSCW, Computer Supported Cooperative Work, Workshop Collaborating Over Paper and Digital Documents (2006)
8. Jacki O’Neill, David Martin, Tommaso Colombino, and Antonietta Grasso. 2011. When a little knowledge isn’t a dangerous thing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’11). ACM, New York, NY, USA, 1667-1676.
9. Jacki O’Neill, Shourya Roy, Antonietta Grasso, and David Martin. 2013. Form digitization in BPO: from outsourcing to crowdsourcing? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’13). ACM, New York, NY, USA, 197-206.
10. O’Neill J., Martin D. (2013) Relationship-Based Business Process Crowdsourcing. In: Kotzé P., Marsden G., Lindgaard G., Wesson J., Winckler M. (eds) Human-Computer Interaction – INTERACT 2013. INTERACT 2013. Lecture Notes in Computer Science, vol 8120. Springer, Berlin, Heidelberg
11. Tristan Harris, The need for a new design ethics [link]
12. Sherry Turkle. Alone Together: Why We Expect More from Technology and Less from Each Other New York: Basic Books, January 2011.
13. Sherry Turkle, TED presentation ‘Connected but alone’.