How to Humanize Your AI?
Given the popularity of Conversational AI, today’s tech companies developed their own ‘Digital Assistants’ by building a human-in-the-loop annotation tool that collects conversational data between human-beings and machines. To that end, they are involved in the development and continuous learning of pre-trained ML models by allowing human annotators to play the role of Assistant behind the scene and converse with users in real time.
What does “human in the loop” mean, and why “Wizard”?
“Human-in-the-loop,” or HITL, describes conversations between a user and an automated system that’s actually being controlled by a human behind the scene. Remember how the Wizard in the The Wizard of Oz was actually just a man behind a curtain? It is similar to that. Using HITL expedites data collection and makes for more naturally flowing, realistic conversations with less user frustration.
To collect realistic, multi-turn conversational data, users’ voice commands are sent to annotators in a tool. Within 60 seconds, an annotator needs to locate the tasks to execute and respond to users’ request. To be successful, annotators need to capture a series of key values in users’ utterance：
Three values annotators need to know to generate a response
Values
Domain: The topic related to users’ inquiry
Task: The work the user wants to accomplish
Slots: The specifications of the tasks
All three are intertwined — the slot required for a response always depends on the task selected, and tasks are grouped under different domains. If a slot is missing or unclear before executing a task, the annotator will need to take Actions to clarify with users.
Challenges
- Flexibility: Annotators need to easily switch between values and make selections. If a slot is missing from the user’s utterance, annotators should be able to quickly take action to clarify on the slots.
- Discoverability: There are a wide range of domains, tasks, and slots that annotators can choose from. Information needs to be clearly available without overwhelming annotators, so that annotators can pick up the tool easily.
- Efficiency: Because annotations are happening in real time and annotators have to go through many steps with time restriction, interaction must be simplified and intuitive to ensure timely response.
Design Highlights
You can consider the interface as being composed of “Chat,” “Composer,” and “Showing on Device sections.” Users’ conversation with Assistant are displayed in the Chat window. Annotators select tasks based on user’s requests and determine the slot required for each task. If a slot is missing, they can take further action and send clarification questions to the user; if a slot isn’t missing, annotators can fill in all slot values and complete the task. Meanwhile, useful information from the device (e.g. Portal), along with current device status, will update on the left side to provide contextual information.
Mega menu
Given the complex information required for annotators to generate a response, a mega menu can be designed in the composer with a three-dimensional panel system to strengthen the information architecture. The goal is to help annotators build a mental model by grouping information under domain, tasks and slots. To reduce cognitive load and help annotators find the exact information they need to fill in, the Task panel and Slot panel only display relevant information based on the option selected in the previous step.
Human-AI Collaboration
Pretrained models can assist annotators label more accurately and efficiently through automation, while annotators can support ML models improvement by providing feedback. The relationship between annotators and AI in the tool is collaborative, and we want to support that through the design. For example, the task suggested by ML models is always selected by default to save time, and the tool always pre-populates the slot value dropdown based on users’ utterance. Alternatively, the design makes it easy for annotators to override AI suggestions — they can type freeform to make corrections if none of the automated options in the dropdown are correct.
Trending AI Articles:
1. Neural networks for algorithmic trading. Multimodal and multitask deep learning
2. Back-Propagation is very simple. Who made it Complicated ?
3. Introducing Ozlo
4. Artificial Intelligence Conference
Human-AI Collaboration
Fluidity in taking actions
In prior models, annotators only realized certain slots were missing after going through a long process, and had to start over to reselect the task. To solve this, you can break down actions into task-related and slot-related actions. Annotators can navigate between tasks to check relevant slots required, and all slot-related actions are grouped under the Slot panel in the new design. If any slot value is not clear, annotators can directly take actions instead of starting over. We also added a search function in the composer to increase discoverability of the options.
Fluidity in taking actions
Message Preview and Feedback
Real-time feedback can be provided to guide annotators through the process.
There’s always more than one way to represent information:
For the Composer component, you can explore approaches representing tasks as searchable entities by graphical cards, to a more text-based interface approach that supports keyboard-first experience. Information can be represented by different paradigms and structured by different interaction methods, so designers should get creative to convey information clearly.
A list of every possible action can help users learn a tool quickly, yet this can be overwhelming. An oversimplified information structure can be even more confusing. As tooling design covers a wide range of information, it’s essential to keep a medium level of granularity and find the balance.
Humanizing AI to ease communication
Annotators are as important as end users of technology. AI models and pipelines constitute sophisticated concepts that can impose heavy cognitive load on annotators. A designer’s role is to translate technical concepts into understandable notions and map relationships between them.
This three-dimensional Mega menu solution offered here indicates well the logical connection among Domain, Task, and Slots for annotators.
Don’t forget to give us your 👏 !
https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href
How to Humanize You was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.
Credit: BecomingHuman By: The AI LAB
