This story was written as an academic paper during TU Delft’s Msc Design for Interaction (December 2018).
The recent development of artificial intelligence has not only increased the presence of chatbots and personal assistants in people’s lives but also created new types of human-robot interactions. One of these is enabling an AI system to communicate with another human in the user’s behalf, as demonstrated by Google’s new assistant called Duplex. Google Duplex is able to call determined businesses over the phone and arrange appointments for its user while sounding so similar to a human that the recipient might not presume that the caller is actually a robot. Google Duplex sparked controversy, which led the project to change in subsequent iterations. By analysing the concerns raised by Duplex and how it evolved, I propose in this paper five ethical guidelines for when AI speaks on behalf of humans: identity, privacy, consent, human backup and task-oriented scope.
Trending AI Articles:
1. From Perceptron to Deep Neural Nets
2. Keras Cheat Sheet: Neural Networks in Python
3. Neural networks for solving differential equations
4. Turn your Raspberry Pi into homemade Google Home
Since Siri was integrated with the iPhone 4S by Apple in October 2011, virtual assistants and chatbots have increasingly become part of people’s lives. The first batch of assistants was mostly focused on performing personal tasks for its users, like setting up an alarm clock, so both the user and the assistant were only acting on their own behalf. However, it did not take long for the technology to be used in new kinds of contexts, namely the growing trend in utilizing chatbots as representatives of companies and commercial establishments, so as to reduce the investment and human workload in areas such as customer service, marketing and sales (Botelho, 2017). In fact, research and advisory company Gartner expects that by 2020, more than 85% of all customer interactions will utilize virtual customer assistants (Moore, 2017).
Still, the disclosure of Google Duplex assistant by Google on May 2018 revealed yet another type of interaction between human and artificially intelligent assistants. While people are becoming accustomed to chatbots speaking on behalf of businesses, Duplex turns the experience around by enabling the voice assistant to speak on the customer’s behalf, while the business recipient is the actual human in the interaction. Not only that, but Duplex sounds strikingly similar to a real human and it did not identify itself as a robot during its inaugural exhibition to the public, so the first recipients were unaware that the caller was actually a robot. This is a significant difference to typical chatbots such as Siri and Alexa, whose voices do not replicate speech disfluencies only present in actual human speech (Cognilytica, 2018).
The novelty of Google Duplex raised multiple discussions over ethical issues around the project and human-robot communication as a whole. These concerns and reactions can be investigated to formulate ethical guidelines on how artificially intelligent machines should behave when speaking in the name of real people.
About Google Duplex
Objectives of the project
First unveiled on 8 May 2018 at Google I/O Conference in Mountain View, California, Google Duplex quickly became the most discussed announcement of the event. Built upon Google Assistant, the company’s current assistant released in 2016, Duplex awed the public with its use of advanced artificial intelligence and human-sounding conversations to autonomously carry tasks with businesses over the phone with the same apparent naturalness as a real person would.
The project was conceived to simplify activities that currently require the user to make phone calls to commercial establishments. As stated by Google’s Principal Engineer Yaniv Leviathan and the Vice President of Engineering Yossi Matias (2018) in Google’s AI Blog, the task is simpler because the user needs only to request their assignment to their voice assistant, saving the time of phone calls and removing worries about calling during off-hours or under limited connectivity. Furthermore, difficulties caused by speech disabilities or language differences are also reduced, since the user can type his request or speak in his own language. As for the businesses’ perspective, it serves as a way to decrease the effort, costs and problems of managing bookings, especially for establishments that do not possess web-based solutions for this yet.
How Google Duplex works
As shown in Figure 1, Google Duplex acts after the user triggers Google Assistant to perform a specific task, like booking a table at a restaurant, a haircut for the user or asking about a place’s business hours. Currently, these are the only tasks Duplex is being trained to accomplish, and for each different assignment, the AI requires extensive real-world data to learn from. Therefore, Google Duplex is unable to carry out general conversations or dialogues outside its domains.
After being activated and receiving the necessary information from the user such as the desired date and number of people for a restaurant booking, Duplex will autonomously call a business and speak to the recipient. The remarkable trait of its speech is how realistic it is: Duplex utilizes human-like pauses, colloquial expressions, different tones and even speech disfluencies such as “ahs”, “umms” and “mm-hmm”, as noted by Washington Post (Roberts, 2018), one of multiple media outlets that covered the project’s announcement. These disfluencies can be identified in Figure 2. Because of these human-like traits, the recipients of the calls exhibited at Google I/O did not suspect they were talking to a robot, and Duplex did not identify itself as such in those examples. Additionally, Duplex shows some capacity for fixing the conversation in case of unexpected events or misunderstandings. During Google’s I/O demonstration, for example, the assistant successfully steered the dialogue when the recipient of one of the calls mistook the desired time of a booking for the number of people that needed a table. This happens because Duplex is trained to comprehend different contexts, as seen in Figure 3.
If the interaction carries out favourably, Duplex will complete its task without human involvement and inform the user of the final appointment details. However, if the assistant understands that it cannot complete its assignment, it is programmed to call a human operator to take over, as stated in Google’s AI Blog (Leviathan and Matias, 2018). Also, all conversations carried by Google Duplex are recorded.
Risks and concerns
The demonstration of Duplex at Google I/O 2018 quickly generated reactions. While the project was praised for its impressive advance of artificial intelligence, human-like speech and the possibility of streamlining time-wasteful interactions with businesses over the phone, it also raised various concerns over the possibility of humans being fooled by robots, possible malicious uses of this technology and the ethics of computational artefacts speaking in behalf of real people.
Spamming businesses is one worry mentioned by Matthew Fenech (O’brien, 2018), a researcher on the policy implications of AI, as during the conference Google did not go into detail on how the recipients would consent to automated calls. Beyond agreeing to receive these calls, the recipient should also be notified that the conversation is being recorded. For instance, it is prohibited in various states of the United States to record phone calls without the authorisation of the caller and the receiver.
Alongside spamming, Fenech also suggests that the technology could be used to scam commercial establishments and people, a concern that is shared among other critics of the project. Possible negative scenarios are using a robot to schedule all available tables in a popular restaurant to later resell them or utilizing personal data on the internet to scam victims for money by making the voice assistant pose as someone they know (Cognilytica, 2018). Other potential uses could be contacting targets in droves to instigate them all to do the same thing (Roberts, 2018) or smaller hoaxes like reporting fake crimes to the police (Cognilytica, 2018). Scamming, manipulation and pranking could also happen the other way around. Ill-disposed people might find a way to prank voice assistants to make them do something undesirable (Cognilytica, 2018) or exploit the way they are programmed to extract personal information about the user (Roberts, 2018).
Furthermore, there are risks involved even if neither parties have malicious intents. While Google Duplex possesses exceptional human-like speech capabilities, it cannot comprehend all the subtle hints, intentions and nuances embedded in people’s conversations. It might use inadequate language (Cognilytica, 2018) or simply cause the wrong impression. For instance, an online article by Forbes Magazine mentions that many attendees at a subsequent Google event in New York perceived the voice assistant to sound sarcastic sometimes (Phelan, 2018).
Many of these concerns are caused by the fact that Google Duplex did not identify itself as a robot during the project’s first demonstrations. Since the voice assistant sounds so similar to a human, the only way to find out the truth would be to force it to improvise, asking questions outside its domains or completely out of sense, as stated by David Gunkel (Cole, 2018), a professor at Northern Illinois University focused on the ethical consequences of information and communication technology. Still, the idea of doubting the humanity behind each telephone call makes people distrust something they always believed to be true, and for many, that might imply giving up an essential trait of their own humanity (Roberts, 2018). David Polgar, an ethicist researcher on the impact of social media and technology, thinks that Google Duplex offers great potential in simplifying transactional interactions between people on the phone, but all parties must be totally informed about the nature of the interaction, otherwise, people’s capacities to trust and form bonds might be damaged. At the same time, he also worries that ceding control of our own communication to computational artefacts might cause us to slowly lose part of what makes us human (Spoonauer, 2018).
How Google Duplex changed
Weeks after its first appearance in May 2018 at Google I/O, Duplex was exhibited again to the public, and it showed important alterations compared to its original demonstration. While it is difficult to determine how many of those changes were originally planned and how many happened in response to raised concerns, a Google spokesperson did state that the first showing was designed to be an early technology demonstration and not a full-fledged product, and the company aspired to apply the received feedback into the project (Hern, 2018). Moreover, in the period after Google I/O, Google provided additional details on how the system will function, especially concerning controversial topics as businesses consent on automated calls.
The most striking change was that, in the new demonstrations, the assistant began each call by identifying itself as “Google’s automated booking service” or “Google Assistant”, and it also stated that the conversation would be recorded. The casual speech using pauses and disfluencies, however, was still utilized (Waters, 2018). According to the company, the assistant’s use of human-like verbal communication is not aimed at deceiving users but is actually an adaptation to early tests of the project, where it was found that recipients were highly inclined to end the call immediately if they perceived the caller voice to be computerised (Broersma, 2018).
Scott Huffman, head of engineering on the project, reiterated that even though people might have mistaken Duplex to possess almost human capacities because of the way it sounds, it is actually only trained to accomplish highly specific assignments, which currently are: asking about businesses’ holiday hours, booking a haircut and reserving a table at a restaurant (Broersma, 2018). At first, the use of Duplex will be restricted to a small set of users, and the assistant will only be able to call Google partner businesses. Calls will be limited to prevent spam, and the establishments will have the freedom to opt out of the program at any time.
Another point of clarification was how human support takes part in the project. Google personnel can take over not only if the assistant is unable to continue, but also if the recipient does not consent on being recorded. In one of the demonstrations, the person who received the call refused to be recorded, so Duplex proposed calling again on an unrecorded line. When it did, the caller was actually a human, who proceeded to opt out the establishment from future automated calls (Phelan, 2018).
The experiences evidenced by Google Duplex seem to point a direction on what kinds of future interactions humans will have with computational artefacts equipped with the autonomy to speak on others behalf. At the same time, the concerns it raised and the way the project changed also provide a glimpse of the implications of this technology and what types of measures its designers can take to prevent it. Analysing the reactions around Google Duplex and the reflections of other sources over the topic of human-robot communication, I now propose five guidelines to help ensure an ethical voice interaction between a human and an artificially intelligent machine speaking on another human’s behalf: identity, privacy, consent, human backup and task-oriented scope. These guidelines are a product of my own reflection, although I’ve researched other authors who mentioned one of these principles.
It’s important to state that these guidelines are not conclusive nor cover all possible contexts of robots speaking for humans since they only use Google Duplex as inspiration. Still, I believe they serve as a starting point for discussing interactions enabled by this technology. Even if Google already updated the protocols of Duplex, new similar products are bound to emerge in the coming years.
The robot should always begin the conversation by identifying itself as a robot, so the human recipient is not exposed to the risk of thinking that they are talking to another person (Wong, 2018). Furthermore, all individuals and corporations that are accountable should also be named (Andras et al., 2018).
The robot should always state that the conversation is being recorded if that is the case. Also, the recipient should have the freedom to decline being recorded, which should terminate the conversation.
The robot should only reach recipients who agree with receiving automated communication to avoid potential situations of spam, abuse, annoyance or frustration.
A human should be available to take over the place of the robot at the request or necessity of either party.
It is advisable that the communication remains limited to transactional or task-oriented assignments, so as not to outsource human to human interactions that are valuable in providing people with their own sense of humanity.
Google Duplex offers great potential in reducing complexity and saving time in effortful phone interactions with businesses and service providers, but the novelty of this technology and the experience it provides generates anxieties of possible malicious uses. Still, as David Pogue writes in his article in Scientific American, it is expected to fret about new technologies. Casual machines such as the microwave were feared in the past when they were first introduced. People stop being afraid when the technology is tested, the side effects are investigated and design guidelines are created (Pogue, 2018). With this article, my goal was to propose what could be the first design guidelines for when artificial intelligence speaks on behalf of humans.
Andras, P. E., Esterle, L., Guckert, M., Han, T. A., Lewis, P. R., Milanovic, K., … & Urquhart, N. (2018). Trusting Intelligent Machines. IEEE Technology and Society Magazine.
Botelho, B. (2017, April). Chatbot technology raises ethical questions. Tech Target. Retrieved from https://searchcrm.techtarget.com/opinion/Chatbot-technology-raises-ethical-questions
Broersma, M. (2018, June 28). Google’s Duplex Now Identifies Itself As A Robot. Tech Target. Retrieved from https://searchcrm.techtarget.com/opinion/Chatbot-technology-raises-ethical-questions
Cole, S. (2018, May 10). How to Tell If You’re Talking to a Robot. Motherboard. Retrieved from https://motherboard.vice.com/en_us/article/gykpqy/how-to-tell-if-youre-talking-to-a-robot-google-duplex
Hern, A. (2018, May 11). Google’s ‘deceitful’ AI assistant to identify itself as a robot during calls. The Guardian. Retrieved from https://www.theguardian.com/technology/2018/may/11/google-duplex-ai-identify-itself-as-robot-during-calls
Leviathan, Y. & Matias, M. (2018, May 8). Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone [Web log post]. Google AI blog. Retrieved from https://ai.googleblog.com/2018/05/duplex-ai-system-for-natural-conversation.html
Moore, S. (2018, February 19). Gartner Says 25 Percent of Customer Service Operations Will Use Virtual Customer Assistants by 2020. Gartner. Retrieved from https://www.gartner.com/en/newsroom/press-releases/2018-02-19-gartner-says-25-percent-of-customer-service-operations-will-use-virtual-customer-assistants-by-2020
O’Brien, M. (2018, May 9). What happens when the robots sound too much like humans? Phys.Org. Retrieved from https://phys.org/news/2018-05-robots-humans.html
Phelan, D. (2018, June 30). Why Google’s Duplex Phone-Call Bot Is Exciting, But Needs To Change. Forbes. Retrieved from https://www.forbes.com/sites/davidphelan/2018/06/30/google-announces-summer-rollout-of-google-duplex-bot-concierge-calling-service-release-date-ai-chatbot/#9f1f35537243
Pogue, D. (2018). I’ll have my ai call your ai. Scientific American, 319(2), 26–26.
Roberts, M. (2018, May 10). Could Google’s creepy new AI push us to a tipping point? Washington Post. Retrieved from https://www.washingtonpost.com/blogs/post-partisan/wp/2018/05/10/could-googles-creepy-new-ai-push-us-to-a-tipping-point/
The Implications of Google Duplex. (2018, May 17). Cognilytica. Retrieved from https://www.cognilytica.com/2018/05/17/the-implications-of-google-duplex/
Waters, R. (2018, June 27). Hello, my name is Google Duplex and I am a robot. Financial Times. Retrieved from https://www.ft.com/content/df97f518-79c6-11e8-bc55-50daf11b720d
Wong, S. (2018). Should you let Google’s AI book your haircut? New Scientist, 238(3178), 21–21. doi:10.1016/S0262–4079(18)30882–0