Sensing & Reasoning Lab · Rutgers University

Sensing & Reasoning Lab

We design AI systems that operate in the physical world under uncertainty. Our work studies how sensing, decision-making, and human interaction couple in high-stakes environments.

About the Lab

The Sensing & Reasoning Lab studies AI systems embedded in dynamic physical environments. These systems perceive through multimodal sensing, act under partial observability, and interact with humans whose behavior is adaptive and strategic.

We investigate how such systems remain stable under distribution shift, how they calibrate their own competence, and how they support intervention through causal reasoning.

Directed by Jorge Ortiz.

Research Focus

  • Verified Agentic AI Systems
  • Formal Verification & Runtime Monitoring
  • Perception-to-Decision Systems
  • Competency-Aware AI
  • Causal Multimodal Reasoning
  • AI Under Distribution Shift

Research Programs

Verified Agentic AI

Formal verification, runtime monitoring, and auditability for multi-agent systems. TraceFix (Best Paper, ACM CAIS 2026) checks agent coordination before execution and monitors it at runtime.

Causal World Modeling

Learning world models that support intervention and counterfactual reasoning under partial observability.

Multimodal State Estimation

Constructing compact latent state representations from heterogeneous sensor streams for sequential decision-making.

Adaptive Human–AI Systems

Modeling strategic human behavior and designing agents that calibrate action, abstention, and deference.

Urban-Scale AI Systems

Deploying sensing and control architectures that remain stable under distribution shift and infrastructure constraints.

Algorithmic Reliability

Formalizing competence calibration, abstention, and deployment constraints in public-facing AI systems.

People

Director

Jorge Ortiz

Associate Professor, Electrical & Computer Engineering

Site Director & PI, CRAIG (NSF Center on Responsible AI & Governance) • Lead Site PI, CS3 (NSF Center for Smart Streetscapes) • Research Analyst, New York Yankees

PhD Students

Taqiya Ehsan

PhD Student

Causal world models, policy learning

Shuren Xia

PhD Student

Agentic systems, formal verification, runtime monitoring

Navid Salami Pargoo

PhD Student

Autonomous systems, sensor fusion

Sonya (Yuan) Sun

PhD Student

3D occupancy prediction, generative models

Qiwei Li

PhD Student

Agentic systems, urban AI, causal discovery

Tianyi Chai

PhD Student

Multimodal learning, AI systems

MS Students

Raymen Shu

MS Student

Multimodal vision, urban sensing (DataCity Smart Mobility)

Recent Publications

Full publication list available on Google Scholar.

Xia, S., Li, Q., Ehsan, T., Ortiz, J. "TraceFix: Repairing Agent Coordination Protocols with TLA+ Counterexamples" ACM CAIS 2026. 2026. Best Paper Award (Outstanding Solution Paper)

Sun, Y., Contreras, J., Ortiz, J. "DFGauss: Dynamic Focused Masking for Autoregressive 3D Occupancy Prediction" NeurIPS 2025. 2025.

Ehsan, T., Xia, S., Ortiz, J. "PolicyGrid: Acting to Understand, Understanding to Act" NeurIPS 2025 Workshop on Embodied World Models. 2025.

Li, Q., Ortiz, J. "TellMe Why: Towards Causal Discovery from Urban Video" NeurIPS 2025 Workshop on UrbanAI. 2025.

Sun, Y., Pai, N., Ramesh, V.V., Aldeer, M., Ortiz, J. "GeXSe (Generative Explanatory Sensor System): An Interpretable Deep Generative Model for Human Activity Recognition in Smart Spaces" IEEE. 2023.

Lab News

Apr 2026
ACM CAIS 2026 Outstanding Solution Paper Award: "TraceFix: Repairing Agent Coordination Protocols with TLA+ Counterexamples." Shuren Xia, Qiwei Li, Taqiya Ehsan, and Jorge Ortiz. GitHub · Blog · Project
Dec 2025
NeurIPS 2025: Three papers accepted - one main conference paper and two workshop papers on causal world models and urban AI.
2025
New NSF Centers: Lab director appointed as Site Director & PI for CRAIG and Lead Site PI for CS3.
Jul 2022
Media Coverage: New Scientist features our robot-assisted feeding research in "Robot that learns social cues could feed people with tetraplegia" .

Contact & Location

Lab Information

Director: Jorge Ortiz
Department: Electrical & Computer Engineering
University: Rutgers University
Location: Piscataway, NJ

Prospective Students

We are looking for motivated PhD and MS students interested in agentic AI systems, formal verification, runtime monitoring, causal AI, multimodal systems, and human-centered computing.

Prospective students should apply to the Rutgers ECE graduate program. If you are already at Rutgers, reach out to Prof. Ortiz directly.