What is Machine Behavior?

Understanding the behavior of artificial intelligence(AI) agents is one of the pivotal challenges of the next decade of AI. Interpretability or explainability are some of the terms often used to describe methods that provide insights about the behavior of AI programs. Until today, most of the interpretability techniques have focused on exploring the internal structure of deep neural networks. Recently, a group of AI researchers from the Massachusetts Institute of Technology(MIT) are exploring a radical approach that attempts to explain the behavior of AI observing them in the same we study human or animal behavior. They group the ideas in this area under the catchy name of machine behavior which promises to be one of the most exciting fields in the next few years of AI.

The ideas behind machine behavior might be transformational but its principles are relatively simple. Machine behavior relies more on observations than on engineering knowledge in order to understand the behavior of AI agents. Think about how we observe and derive conclusions from the behavior of animals in a natural environment. Most of the conclusions we obtain from observations are not related to our knowledge of biology but rather on our understanding of social interactions. In the case of AI, the scientists who study the behaviors of these virtual and embodied AI agents are predominantly the same scientists who have created the agents themselves which is the equivalent of requiring a PH.D in biology to understand the behavior of animals. Understanding AI agents goes beyond interpreting a specific algorithm and requires analyzing the interactions between agents and with the surrounding environment. To accomplish that, behavioral analysis via simple observations can be a powerful tool.

What is Machine Behavior?

 
Machine Behavior is a field that leverage behavioral sciences to understand the behavior of AI agents. Currently, the scientists who most commonly study the behavior of machines are the computer scientists, roboticists and engineers who have created the machines in the first place. While this group certainly has the computer science and mathematical knowledge to understand the internals of AI agents, they are typically not trained behaviorists. They rarely receive formal instruction on experimental methodology, population-based statistics and sampling paradigms, or observational causal inference, let alone neuroscience, collective behavior or social theory. Similarly, even though behavioral scientists understand those disciplines, they lack the expertise to understand the efficiency of a specific algorithm or technique. From that perspective, machine behavior sits at the intersection of computer science and engineering and behavioral sciences in order to achieve a holistic understanding of the behavior of AI agents.

As AI agents become more sophisticated, analyzing their behavior is going to be a combination of understanding their internal architecture as well as their interaction with other agents and their environment. While the former aspect will be a function of deep learning optimization techniques, the latter will rely partially on behavioral sciences.

Understanding the Behavioral Patterns in AI Agents

 
Ethology is the field of biology that focuses on the study of animal behavior under natural condition and as a result of evolutionary traits. One of the fathers of ethology was Nikolaas Tinbergen, who won the 1973 Nobel Prize in Physiology or Medicine based on his work identifying the key dimensions of animal behavior. Tinbergen’s thesis was that there were four complementary dimensions to understand animal and human behavior: function, mechanism, development and evolutionary history. Despite the fundamental differences between AI and animals, machine behavior borrows some of Tinbergen ideas to outline the main blocks of behavior in AI agents. Machines have mechanisms that produce behavior, undergo development that integrates environmental information into behavior, produce functional consequences that cause specific machines to become more or less common in specific environments and embody evolutionary histories through which past environments and human decisions continue to influence machine behavior. An adaptation of Tinbergen’s framework to machine behavior can be seen in the following figure:

Based on the previous framework, the study of machine behavior focuses on four fundamental areas: mechanism, development, function and evolution across three main scales: individual, collective and hybrid.

For a given AI agent, machine behavior will try to explain its behavior by studying the following four areas:

1. Mechanism: The mechanisms for generating the behavior of AI agents are based on its algorithms and the characteristics of the execution environment. At its most basic level, machine behavior leverages interpretability techniques to understand the specific mechanisms behind a given behavioral pattern.
2. Development: The behavior of AI agents is not something that happens on one shot but it rather evolves over time. Machine behavior studies how machines acquire (develop) a specific individual or collective behavior. Behavioral development could be the result of engineering choices as well as the agent’s experiences.
3. Function: An interesting aspect of behavioral analysis is to understand how a specific behavior influences the lifetime function of an AI agent. Machine behavior studies the impact of behaviors on specific functions of AI agents and how those functions can be copied or optimized on other AI agents.
4. Evolution: In addition to functions, AI agents are also vulnerable to evolutionary history and interactions with other agents. Throughout its evolution, aspects of the algorithms of AI agents are reused in new contexts, both constraining future behavior and making possible additional innovations. From that perspective, machine behavior also studies the evolutionary aspects of AI agents.

The previous four aspects provide the a holistic model to understand the behavior of AI agents. However, those four elements don’t apply the same way when we are evaluating a classification model with a single agent than a self-driving car environment with hundreds of vehicles. In that sense, machine behavior applies the previous four aspects across three different scales:

1. Individual Machine Behavior: This dimension of machine behavior attempts to study the behavior of individual machines by themselves. There are two general approaches to the study of individual machine behavior. The first focuses on profiling the set of behaviors of any specific machine agent using a within-machine approach, comparing the behavior of a particular machine across different conditions. The second, a between-machine approach, examines how a variety of individual machine agents behave in the same condition.
2. Collective Machine Behavior: Differently from the individual dimension, this areas looks to understand the behavior of AI agents by studying the interactions in a group. The collective dimension of machine behavior attempts to spot behaviors on AI agents that don’t surface at an individual level.
3. Hybrid Human-Machine Behavior: There are many scenarios in which the behavior of AI agents is influenced by their interactions with humans. Another dimension of machine behavior focus on analyzing behavioral patterns in AI agents triggered by the interaction with humans.

Machine behavior is one of the most intriguing, nascent fields in AI. Behavioral sciences can complement traditional interpretability methods to develop new methods that help us understand and explain the behavior of AI. As the interactions between humans and AI becomes more sophisticated, machine behavior might play a pivotal role to enable the next level of hybrid intelligence.

 
Bio: Jesus Rodriguez is a technology expert, executive investor, and startup advisor. A software scientist by background, Jesus is an internationally recognized speaker and author with contributions that include hundreds of articles and presentations at industry conferences.

Original. Reposted with permission.

Related:

• How DeepMind and Waymo are Using Evolutionary Competition to Train Self-Driving Vehicles
• This New Google Technique Help Us Understand How Neural Networks are Thinking
• Introducing AI Explainability 360: A New Toolkit to Help You Understand what Machine Learning Models are Doing