The fascination of learning agents
Krause is particularly fascinated by questions of optimal information gathering that require efficient, active learning known as reinforcement learning. The key factor here is dealing with uncertainty when not all the information is yet available or when there are a multitude of alternative solutions.
In conventional “passive” machine learning, a learning algorithm is trained using large data sets to deduce certain patterns from data annotated by experts – enabling, for example, a classification rule to be identified that recognizes whether photos show pedestrians or traffic signs. “Active” learning methods on the other hand decide for themselves what data they require to perform the task in hand effectively. For instance, they can go through an image data set, select any images that will help learning progress and have them annotated by experts, which saves time and money and reduces errors. Other active methods propose experiments where the outcome will support information acquisition.
Such computer programs are also known as learning agents. “I’m fascinated by active learning methods, where a learning agent decides for itself which data will be useful in helping it to make good decisions,” Krause says.
Robotics is one field in which such issues arise. Krause illustrates this using a drone that learns to solve certain tasks independently by means of experimentation. The difficulty here is that at the beginning, it isn’t possible to say exactly what might cause the drone to crash and what won’t. At first, the drone must behave cautiously; then, the more data it obtains and analyses, the better it will be able to perform without putting itself or others in danger.
Dilemma between old and new data
However, for a learning agent to gather information independently is no trivial process. To optimise its fulfilment of a task, it must find the right mix of existing data and new, self-acquired data.
This trade-off is what researchers refer to as the “exploration-exploitation dilemma”: if the learning agent independently decides which experiments to conduct in order to obtain additional data, then its decisions also influence which data it does and does not have available as it learns.
As one of his seminal achievements, Krause developed the first mathematical learning method for which, based on certain assumptions, it is possible to prove that it will solve the exploration-exploitation dilemma effectively, even in complex applications. Mathematically speaking, this is a variation of Bayesian optimisation, which also underlies the how drones learn not to crash, and which provides certain formal safety guarantees under specific conditions.
Pioneering and passionate researcher and teacher
Krause’s research is indeed very mathematical. For example, active learning methods require very specific “submodular” features to acquire useful data as efficiently as possible. Today, Krause is viewed as the pioneer who brought submodular optimisation to machine learning. The findings from a very influential publication by Krause from his time in the US even led to practical applications in water distribution networks, addressing the question of where to best place the sensors in order to ensure optimum measurement of water quality.
Krause is not only a sharp thinker when it comes to the mathematical underpinnings of machine learning; he is also someone who reflects on the impact these technologies might have on business and society. For him, it is essential to ensure that the algorithms or calculation rules underlying the learning procedures are reliable, explainable and traceable, and that the results, decisions and recommendations are fair and trustworthy for whoever they affect.
And Krause relays this conviction in his lectures. Alongside the fundamentals of mathematics and computer science, he is passionate about teaching future AI experts a sense of responsibility regarding the use of the technologies. The Golden Owl Award he received from ETH students in 2012 for his teaching and the fact that over a thousand students attend his “Introduction to machine learning” lecture bear testament to his commitment. He was also instrumental in designing the data science Master’s programme and the DAS in data science at ETH; in the ETH AI Center, he ensures that more business aspects are incorporated into courses so that spin-offs can increasingly put the acquired AI skills into wider practice.
Credit: Google News