Why ML is Unlike Human Learning
In the previous post, I briefly explained the three main types of machine learning, and compared them to their human learning theory equivalents. Using Dolan & Dayan’s (2013) discussion on goals and habits in the brain, this post will discuss about the difference between model-based learning and model-free learning, before showing how it helps to differentiate human and machine learning.
Model-based vs Model-free Learning
Model-based learning is a top down approach where the learner starts with a preconceived model of the environment, and learning is based on the planning, predictions and actions that are made according to the known probabilities and rewards of this model. Model-free learning on the other hand, does not begin with any model of the environment, but the rewards of each action are learnt and stored in a bottom-up way, and future actions are decided based on what was learnt. Dolan & Dayan (2013) posited that learning in humans is model-based when reinforcement is used in a goal-directed fashion, while a model-free approach results in habit formation .
While both model-free and model-based mechanisms can be found in human learning, the mechanisms that are generally observed are model-based. As described in the theories of human learning, both classical and operant conditioning involve rewards for a certain behaviour to be reinforced. These rewards are what Dolan & Dayan (2013) term as goals, and the behaviours are the goal-directed actions. People normally learn by drawing the connections between their behaviours and achieving a certain goal, and in the course of doing so, they are building a causal model which can be used for planning future actions to take.
the depth of the output layer.
Hyperparameters of Conv layer — Depth, Stride, Padding.
The number of filters used is called Depth of the Conv layer. It determines the depth of output volume.
Trending AI Articles:
1. My solution to achieve top 1% in a novel Data Science NLP Competition
2. Application of Deep Learning in Front-end Web Development (HTML + CSS)
3. Investigating Focal and Dice Loss for the Kaggle 2018 Data Science Bowl
4. Paper repro: “Self-Normalizing Neural Networks”
This is evident in social learning as well, which is in fact an extension of operant conditioning (Bandura & Walters, 1977) . Humans live in social environments, so besides personal experience, causal models can also be built by observing the consequences of others’ behaviours, which is also known as vicarious reinforcement. Determining causality in the environment requires humans to draw inferences, a skill that is currently still not possible in machine learning.
Just like human learning, there are both model-based and model-free approaches in machine learning. Supervised learning would be considered a model-based learning approach, as it relies on a pre-labelled training dataset to determine its predictions. Unsupervised learning such as artificial neural networks focuses on searching for patterns in the dataset that do not build a model, hence it is model-free. Reinforcement learning has both model-free and model-based methods, but it is unclear which are more dominant (Kaelbling et al., 1996) .
Model-based reinforcement learning is similar to the model-based operant conditioning mentioned earlier, where the computer program has an end-goal and it learns to reach that end-goal through the outcomes of its actions. On the contrary, model-free reinforcement learning does not have an end-goal, but the outcomes of the actions still contain rewards which the program uses to learn in a trial-and-error fashion. A popular example of model-free reinforcement learning is Q-learning, which was successful at learning how to complete Atari video games like Pac-Man, up to the expert levels (Watkins & Dayan, 1992) .
Why is Machine Learning Unlike Human Learning?
Model-based and model-free machine learning have different costs and benefits. While model-based algorithms provide a comprehensive outline of the task environment, they require a large amount of data in order for the model to be built, making them statistically well-supported but computationally inefficient. Model-free algorithms on the other hand do not require a lot of data as there is no attempt to understand the environment, thus allowing them to be computationally efficient but less statistically supported.
This is where the key difference with human learning lies. As explained, human learning may be model-based, but it is often computationally efficient. Humans use mental models that generally require very few examples, but are able to make new predictions through extrapolation. This is because humans are able to draw inferences from underlying concepts, and generalize those concepts to novel situations. Machine learning is unable to do this because programs only interpret data and not concepts.
The type of mental models that humans use is sometimes known as heuristics. Heuristics have been compared to machine learning models to show that despite using simple rules and few training examples, they are not only able to perform almost as well as machine learning models, but are also more generalizable to novel situations (Czerlinski et al., 1999; Martignon et al., 2008)  . This is due to a phenomenon in model-based machine learning known as overfitting.
Overfitting occurs when the training data used to construct a model explains the data very well, but fails to make useful predictions for new data (Pitt & Myung, 2002) . As a simpler model, heuristics supposedly do not suffer from such a problem, as the variance they allow is usually wider, especially when the training sample is small. Recent findings by Parpart et al. (2017) reveal that heuristics and linear regression are possibly extreme opposites of Bayesian inference, suggesting that the models built by human learning result in flexible heuristics, whereas the models that arise from machine learning are more rigid and complex .
In addition, machine learning is lacking in a number of aspects in order to emulate human learning (Lake et al., 2016) . For a start, people are born with a package of intuitive physics and intuitive psychology. These early intuitions already act as a pre-existing model for babies to figure out how physical and social interactions work in their environments, and can be built upon through reinforcement and conditioning. In this respect, a machine learning algorithm needs to incorporate some form of start-up software that gives it the same advantage. But even so, it is currently still difficult for machine learning to make sense of social interactions, while babies are able to distinguish friend from foe even at a very young age (Hamlin, 2013) .
As mentioned a couple of times, machine learning is also unable to draw inferences and understand underlying concepts. These are important skills required to build a causal model, but the models built by machine learning are mostly correlational. And finally, what is unique about human learning is the ability to learn how to learn and improve learning. While machine learning is able to learn from its actions and rewards, the same algorithm is unable to change its learning strategy unlike a human. If machine learning were to become more human-like, it certainly needs to be able to achieve the above-mentioned points.
* * * * * * * * * *
Machine learning has often been compared to human learning, even though the two are not exactly the same. Learning in humans has been shaped by evolutionary processes to become what it is today. While many theories have tried to explain the mechanisms of human learning, the dynamic nature of it probably means that different strategies can be used simultaneously or separately, depending on the situation. This makes it difficult to comprehensively capture the entirety of human learning, much less compare it to machine learning. After all, machine learning has been programmed by humans, which may not be as efficient as the design of nature.
Nonetheless, the basic idea of machine learning can still be said to have been inspired by human learning, especially through the various examples in reinforcement learning. Perhaps an important reason to design more human-like machine learning is for a better understanding of human learning to be developed through computational models. But is it even possible for machine learning to emulate human learning? While the answer may not be clear, it probably does not matter as much once machine learning is able to surpass human learning in every imaginable aspect of problem solving and decision making.
If you would like to know more about the similarities between machine learning and human learning, check out the first part of this series:
If you found this article interesting, you might also enjoy this one:
- Dolan, R. J., & Dayan, P. (2013). Goals and habits in the brain. Neuron, 80(2), 312–325.
- Dayan, P., & Niv, Y. (2008). Reinforcement learning: the good, the bad and the ugly. Current opinion in neurobiology, 18(2), 185–196.
- Bandura, A., & Walters, R. H. (1977). Social learning theory.
- Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of artificial intelligence research, 4, 237–285.
- Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine learning, 8(3–4), 279–292.
- Czerlinski, J., Gigerenzer, G., & Goldstein, D. G. (1999). How good are simple heuristics?. In Simple heuristics that make us smart (pp. 97–118). Oxford University Press.
- Martignon, L., Katsikopoulos, K. V., & Woike, J. K. (2008). Categorization with limited resources: A family of simple heuristics. Journal of Mathematical Psychology, 52(6), 352–361.
- Pitt, M. A., & Myung, I. J. (2002). When a good fit can be bad. Trends in cognitive sciences, 6(10), 421–425.
- Parpart, P., Jones, M., & Love, B. C. (2018). Heuristics as Bayesian inference under extreme priors. Cognitive psychology, 102, 127–144.
- Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2016). Building machines that learn and think like people. arXiv preprint arXiv:1604.00289.
- Hamlin, J. K. (2013). Moral judgment and action in preverbal infants and toddlers evidence for an innate moral core. Current Directions in Psychological Science, 22(3), 186–193.