Use Case Part II
When you start to dig into artificial intelligence (AI) and deep learning (DL) natural language processing and generation (NLP and NLG), oh my, scenes from Alice in Wonderland come to mind, and for good reason.
Language and the philosophy of language and meaning have been around for quite some time, so there is a lot of domain knowledge therein. And, as is the nature of philosophical discourse, I’m quite sure the discourse will continue for quite some time, increasing the substantial documentation.
Yet, with all the discussion about language and the application of computers to parse out words for automated processing, NLP and meaning, there is still a well documented and replicable issue, specifically generalization and context within the event of the discourse. Said another way, cutting to the chase, NLP has no way to know the context about its engagement beyond its limited training. I’ve included references below for those that enjoy the argument.
The solution, in a nutshell, to the NLP “generalization” problem is typically referred to as artificial general intelligence or AGI.
“Some believe true NLP requires general artificial intelligence, AI that is on par with the human mind. And that is nowhere in sight.”
Does AI Speak Our Language? PC Mag
Trending AI Articles:
1. Let’s build a simple Neural Net!
2. Decision Trees in Machine Learning
3. An intuitive introduction to Machine Learning
4. The Balance of Passive vs. Active A.I.
AGI is expected to provide computer-based processing of conversations with a broader contextual understanding of the situation it is involved in, like a human, enabling the computer-based AI system to “understand” concepts beyond what today’s AI DL training is able to do. And the inability to understand contextual concepts beyond training is the hard limit of today’s AI DL.
A real-world example of this limit is when a person starts talking to an Alexa-type speaker system beyond the expected command driven structure. The person might start talking about a memory they have, and thinking out loud they might start explaining their memory. The NLP system would have no context with which to parse the meaning of the event, what the person is actually doing, and would be stuck trying to parse a request from the audible stream coming from the human.
One Solution Architecture
Let’s use Alexa as the low hanging fruit example, only because I have no ties to Amazon and Alexa is something most people are aware of.
Today Alexa responds to specific voice commands, as long as they are relatively simple commands within the repertoire of the system’s vocabulary.
So, when you ask Alexa, “What’s the weather going to be like today,” the device records your voice. Then that recording is sent over the Internet to Amazon’s Alexa Voice Services which parses the recording into commands it understands. Then, the system sends the relevant output back to your device. When you ask about the weather, an audio file is sent back and Alexa tells you the weather forecast all without you having any idea there was any back and forth between systems.
…As a subset of artificial intelligence, natural language generation (NLG) is the ability to get natural sounding written and verbal responses back based on data that’s input into a computer system. Human language is quite complex, but today’s natural language generation capabilities are becoming very sophisticated. Think of NLG as a writer that turns data into language that can be communicated.
Machine Learning In Practice: How Does Amazon’s Alexa Really Work?
Imagine that the Alexa side had an additional component that enabled the system to “understand” the additional context beyond the command response structure, such that the automated responses shifted based on that “understanding.”
How might that happen?
Well, in today’s cloud service world, software can make a request of a cloud-based API and the cloud-based API response can then be used to build a more refined response for the software’s customer. If you’re not a software geek, no worries, just consider that application programming interfaces, or APIs, are in many cases provided in the form of cloud-based REST APIs for other entities, like the Alexa’s of the world, to consume and use as needed.
So, imagine the Alexa scenario where Alexa is engaged with a customer and the inbound string doesn’t match the pattern for a command-driven well understood communication exchange, for example parsing out the dream text. Imagine at that point, behind the scenes, the Alexa code makes a request to an AGI REST API that takes the request and generates a “context” return value which enables Alexa to stitch together a broader context of understanding. Alexa then responds to the customer,
“Yes…, well, I’m not the dream subject matter expert. You would be far better served by asking…”
Or some such appropriate response based on the context and string issued by the customer — but mainly that the response now makes much more sense to the customer.
How might an AGI REST API service work?
First, consider that the NLP and NLG industry is competitive. There are a number of providers, each looking for ways to provide differentiated services to increase their attractiveness, to generate customers. There are also research institutions as well as other industry verticals, think self-driving cars. This is the combined demand set, the consumers of our theoretical AGI REST API.
The demand set cares that:
- The AGI REST API service provides reasonably accurate return values.
- The cost of said service is also reasonable (something like $.02 per request/response time/second, with an SLA of less than 3-second response)
The demand set, in most but not all cases, isn’t structured to create their own AGI solution. Most within the demand set are producers of some type, consuming the material and services they need to produce their value-added product or service. Therefore, as long as the cost and result are within an acceptable range, the value to help the demand set provide improved results for their customers will help them differentiate, and therefore be of value to the demand set.
The point: The demand set cares that the AGI REST API, as a black box, something they consume, is reasonably accurate in its return values and at a cost that they, the demand set, can recover.
What happens under the AGI REST API is not something the demand set cares about. And the area below the AGI REST API is where the magic happens.
The provider of said AGI REST API will enjoy the benefits being first, of learning and then improving the service based on usage feedback and improvements based thereon, further extending their early lead. And, an early lead (this link is a great observation about the value of early lead) can be a huge market and financial benefit as history has proven time and time again. Remember Betamax vs. VHS, CellOne, and most recently, AWS?
To summarize, advancing AI natural language processing is close to the tip of the spear in the AI space going into 2019, for a couple of good reasons. First, enabling computer-based conversations to be more human-like, witness Alexa, enables improved customer experiences. And secondly, by doing so, these NLP and NLG “generalization” solutions begin to close the gap between a real AGI and human-computer communication.
Finally, to illustrate the amount of thought exchange being invested in this specific domain, take a gander at the linked Y Combinator dialog snippet below. This exchange illustrates a couple of key points. First, it illustrates that there are domain experts with incredibly strong beliefs, which within their own domain make sense. Yet, unfortunately for them, the world and the universe is much more complex and diverse than their expertise, which is a great segway into the next observation. While there are some who are comfortable within the constraints of their understanding (think flat earth, and earth at the center of the universe, etc.), there are others who actively explore possibilities. And so, yes, there is agreement with the observation and articulation about the complexities inherent in communication. Yet, and this is key, there are ways to enhance AI DL NLP and NLG with an AGI REST API, today. More about the architecture of “how” in subsequent writings. The “why” is simply to provide a “type of” solution to the AGI problem today as a baseline for learning and improving from, vs. all the reasons it’s not possible. The question is not “is it possible?” rather “will the AI DL NLP and NLG community engage with the solution as a learning platform?” My guess is no, which is the topic for a future post.
Now for the YC dialog snippet:
…“‘a mouth without a brain’ analogy is good one. Current NLP is impressive but there are limits.
People have spatiotemporal model of the world, different physical models, social and behavioral models of the world, organizational model of the society, economic model, etc. Humans parse the language and transform it into multiple models of the world where many indented meanings and semantics are self-evident and it becomes “a common sense”. They have crude understanding of how fabrics, paper, gas, liquid, rubber, iron, rock, etc. behave and they understand written text based on this more complete model zoo.
There is similar limit in computer vision. Humans reason about 2d images using internal 3d model. Even if they see a completely new object shape, they can usually infer what the other side of the object looks like using basic symmetries and physical models.
Image understanding must eventually transform into spatiotemporal + physical model and there are several approaches underway. NLP has much harder problem, because the problem is more abstract and complex.”…
So, what does tomorrow bring to AI DL NLP and NLG?
“At the pace it’s going, we’ll be seeing an even greater interest natural language processing — both in speech recognition and language generation — in the coming year. That means Alexa won’t only be able to understand what we say…”
Top Six AI And Automation Trends For 2019, Forbes
Stay tuned, there’s more writing to come on the topic of supporting artificial intelligence deep learning and natural language processing and generation supplemented with a general intelligence REST application programming interface and how such an API could be implemented in the very near future…
“Cheers!” “乾杯 / Kanpai” “Salud!” “Prost” “Salud!” “Santé!” “건배 / Geonbae” “skål” “Gesondheid” “gānbēi” “Υγεία / Yamas”
Context, Language, and Reasoning in AI: Three Key Challenges
Deep Learning for Natural Language Processing — Part I
Top Six AI And Automation Trends For 2019
A Light Introduction to Transfer Learning for NLP
Does AI Speak Our Language?
What’s the biggest challenges in NLP (Natural Language Processing) field? And why?
Global Natural Language Processing Market 2019- HPE, Microsoft Corporation, NetBase Solutions, IBM Incorporation
Infosys scouting for AI start-ups to overcome talent crunch
NLP’s generalization problem, and how researchers are tackling it
Machine Learning In Practice: How Does Amazon’s Alexa Really Work?