AI for the Physical World: Sensing Not Optional

Recent advances in AI have initiated transformative changes in many areas of non-physical work. However, what do these advances imply for agents that operate in the physical world? Will current approaches to developing AI-based systems provide meaningful advances toward General Physical Intelligence (GPI) – namely the ability to quickly understand, robustly plan for, and efficiently execute novel physical tasks?

In this talk, I will offer a perspective on the path toward GPI, with a focus on identifying some of the gaps that stand in the way of real-world practical systems. Some of these gaps stem from today’s data-intensive recipe for AI, but closely related are limitations in terms of both physical embodiment and physical sensing. I will ground these perspectives with examples emerging in the field of sensor-based robotics as well as results achieved by myself and my colleagues in understanding and developing systems that perform complex tasks such as surgery. Along the way, I will offer some perspectives how future advances toward general physical intelligence might be achieved through the combined efforts of academia, industry, and government support.

Toward Truly Intelligent Cyber-Physical Systems: Learning, Adaptation, and Strategic Autonomy

Intelligent cyber-physical systems (CPS) are tightly integrated, heterogeneous systems that combine physical processes with computation, communication, and control. These systems rely on seamless interactions between analog and digital components, interconnected through communication networks that enable real time data exchange. Among the most critical elements of intelligent CPS are sensors and actuators, which directly influence system performance, efficiency, and adaptability. Sensors provide situational awareness by capturing data from the physical environment, while actuators enable the system to respond and steer its behavior toward desired objectives. To address the inherent cognitive and computational limitations of intelligent CPS, this talk introduces principles of bounded rationality for autonomous decision-making, leveraging tools from control theory and reinforcement learning. In particular, we develop level-k thinking and cognitive hierarchy frameworks within both nonlinear and linear noncooperative differential games, where agents are characterized by varying levels of reasoning depth. Building on this foundation, we present a meta-learning framework for multi-agent environments that enables advanced decision-making strategies. This framework allows autonomous agents not only to learn from their environment but also to anticipate, influence, and strategically respond to the learning behaviors of other agents, enabling capabilities such as adaptive coordination, strategic manipulation, and deception. Furthermore, we introduce data-driven methods for actuator and sensor selection in intelligent CPS, with a focus on enhancing system resiliency. Model-free, learning-based approaches are proposed to optimize key system properties, including controllability, observability, and robustness against adversarial disruptions. These methods leverage reinforcement learning to dynamically select sensing and actuation configurations using both state and output feedback in continuous- and discrete-time settings. The talk concludes with simulation studies on large-scale systems, demonstrating the effectiveness and scalability of the proposed frameworks in designing resilient, adaptive, and intelligent cyber physical systems.