Battlefield AI and Autonomy for Non-Technical Beginners
Mr. Jerry L. Schlabach
8:00 AM–12:00 PM | Room 3L6

The U.S. Government, its military competitors, and the global defense industry are competing to militarize Artificial Intelligence (AI) and Machine Learning (ML) for future autonomous systems. This Working Group 35 (AI and Autonomy) sponsored tutorial will:

• Define and characterize the various levels of military autonomous systems with respect to AI/ML capabilities, human direction, and human trust.
• Dispel and re-characterize common misperceptions about AI/ML and battlefield autonomy, to include the likely technical, moral, and operational limits to weaponization.
• Introduce at a conceptual level the AI and ML fields, with example applications.
• Explain the extraordinary dependency of modern Deep-Learning ML upon the acquisition and conditioning of large amounts of training data (or synthetic models).
• Frame the likely military utility of integrating AI/ML into military systems at the various levels of the cognitive domain (Bloom’s Taxonomy). Identify which cognitive tasks are likely to remain with humans, and which are candidates for machine reasoning.
• Highlight and discuss select OR analytic implications from battlefield AI/ML integration with respect to traditional paradigms such as Commander’s Intent and decision-making.
• Briefly characterize and begin explaining Large Language Models (LLMs) such as ChatGPT, in the context of the more basic AI/ML foundation this tutorial addresses.
• Outline select AI/ML issues related to the future of warfare.

Target audience are those MORS participants with significant holes in understanding of AI and Autonomy. WG-35 objective is to answer basic questions about AI/ML and Autonomy in this tutorial. This will facilitate more advanced WG session presentations later in the week.

Antifragility and Future Conflict: A Tutorial
William Buppert
8:30 AM–10:30 AM | Room 3L4

Operations research needs to acquaint itself with the limits of modeling and the pitfalls of insufficient and misguided asymmetrical evidentiary bars. Antifragility offers not only an explanatory framework of black swan events but creates opportunities for organizations to build themselves from the ground up to be adaptable and resilient in the face of crisis and conflict.

This tutorial will introduce the novice and intermediate practitioner to the concepts of antifragility and how the model may create new ways of looking at future conflict, achieve adaptive frameworks, improve conflict forecasting and better explain ways to build military organizations that respond to violence and capability degradation in a way that parallels the stressor strength improvement in complex systems.

This brief will discuss how the adoption of antifragility models to template conflict dynamics and build resilient learning organizations that improve with stress optimizes the western ability to survive peer and near-peer conflicts in the future. Problem structuring methods (PSM), morphological analysis and other operations research methodologies will be used to tackle the wicked problem sets in antifragility.

Introduction to Modeling and Predicting System Resilience
Dr. Lance Fiondella
9:00 AM–10:30 AM | Room 3L1

This tutorial will introduce participants to resilience engineering concepts as well as statistical models and tools created to predict resilience as a function of factors that degrade and restore performance. The concepts are crosscutting and have been applied to a range of domains, including cybersecurity, infrastructure, autonomous systems, and economic systems.

Ethics for Analysts and Data Scientists: Drawing the Line and Holding the Line
Mr. Terrance James McKearney, FS
9:00 AM–10:30 AM | Room 3L5

Operations research analysts and data scientists are routinely asked to deliver insights that inform high-stakes defense decisions. Doing so requires not only technical excellence, but sound professional judgment—especially when assumptions, data limitations, organizational pressures, or competing priorities test where lines should be drawn. This popular tutorial, hosted by the MORS Ethics Committee, offers an interactive opportunity to engage with senior leaders in the profession who have navigated these challenges firsthand. Drawing on their experience, presenters will discuss what it means in practice to uphold professional standards while producing rigorous, credible, and highly defensible analytical products. The tutorial explores how analysts and data scientists can integrate principled decision-making into day-to-day analytical work. Topics include current expectations and regulations, the MORS Code of Ethics, and the real-world tensions that arise at the intersection of analysis, organizational context, and personal responsibility. The tutorial combines a focused presentation with a facilitated workshop with realistic scenarios that challenge professional judgment. Participants will leave with a clearer understanding of how experienced analysts “hold the line,” maintain credibility under pressure, and deliver analysis that stands up to scrutiny—technically, professionally, and institutionally.

Design and Analysis of Experiments – Next Level Methods with Case Studies
Dr. Thomas A. Donnelly and Dr James Wisnowski
9:00 AM–5:00 PM | Room 3L2

Aligned with the 94th MORSS theme, “Sharpening Our Analytical Edge,” this one-day course is built for analysts and testers who already understand the fundamentals of test science and want to advance their skills with modern, real-world, practical methods used across the DoW test community. The course emphasizes Design of Experiments (DoE) applications seldom covered in most short courses and integrates computer aided approaches with JMP demonstrations to show how advanced design and analysis techniques can be implemented effectively.
NOTE: Topics are covered in stand-alone modules enabling attendees to benefit from seeing even small sections of the presented content.

Design Topics: Participants will learn to create DoEs for complex, constrained design spaces. Topics include exploratory data analysis techniques to mine historical information; custom algorithmic DoE to simultaneously handle continuous, categorical, discrete-numeric, mixture, blocking, and hard-to-change factors; and designing under constraints to avoid infeasible test conditions. The course covers augmenting existing data or repairing broken designs, leveraging historical data, and using active learning strategies to select the best test to run next to meet response goals. Modern screening methods include designs that collapse to support predictive response-surface models, supersaturated designs (when more factors than runs), and nearly orthogonal arrays for multi-level categorical factors. Additional topics include mixture design best practices, sequential designs for efficient computer simulation, accelerated life testing approaches, and measurement system evaluation.

Analysis Topics: Analytical instruction focuses on model building, validating (for large and small datasets), and applying predictive models. Topics include model development using ordinary least squares, stepwise regression, logistic regression, and generalized regression methods such as LASSO, ridge, elastic net, Dantzig, backward elimination, forward selection, and Self-Validating Ensemble Models (SVEM). The course also covers model averaging, Gaussian Process (Kriging) models for deterministic simulation, random effects modeling, comparability and equivalence testing, functional data analysis for curve or spectral data, nonlinear modeling, multiple response optimization, trade space analysis, and the use of machine learning methods when appropriate.

The day concludes with a Q&A session to help participants apply these methods to their own T&E challenges.

Supply Chain Analytics: Hands-On Tutorial
Ralph Asher
9:00 AM–5:00 PM | Room 3L3

Ready to level up your supply chain analytics skills? Our full-day Supply Chain Analytics tutorial is aimed at operations research and analytics professionals working in roles that support logistics, supply chain, operations, and manufacturing.

Unlike other training for data professionals, which focuses solely on technical skills and is absent of sector-specific context, this training combines in-depth technical skills with supply chain examples inspired by real life. Participants will be led by a trainer with experience in the Fortune 500, military, and academia, in both data science and supply chain management.

Participants will dive into real-world supply chain challenges and walk away with skills they can immediately apply to their work. The tutorial will use the R programming language and will include code examples.

If you’re a Python user, the concepts and learnings from these two days goes beyond just “code” – enabling you to be a better data scientist than when you started.

This full-day tutorial is an abbreviated, modified version of a two-day course that we have conducted for audiences outside of the national security sphere.

Edge AI: Running and training LLMs and AI models on your own device
John Babick and Jack FitzGerald
10:30 AM–12:00 PM | Room 3L5

In this tutorial, we will guide you through the use of several edge AI tools that can be run without an internet connection, either for Denied, Disrupted, Intermittent, and Limited (DDIL) environments or for classified compute environments that prohibit broader internet connectivity. Tools will include (a) a military-trained local AI assistant with a familiar chatbot interface and retrieval augmented generation, (b) a push-button local AI model training system, and (c) an agentic interface for general purpose task execution. Edge compute frees the user from the constraint of internet connectivity, and it is also the most private and secure possible system architecture. This tutorial will be hands-on, and time will be given to download necessary software. By the end of this tutorial, you’ll be able to run AI on your own device(s).

The Art of Successful Analysis
Mr. Arthur H. Barber, III, FS
11:00 AM–12:00 PM | Room 3L4

This tutorial presentation will describe how to approach the process of organizing an analytic project, how to lead a team that carries it out successfully, how to assure its quality, and then most importantly how to create, staff, and deliver the presentation that is the product of the project. This will include guidelines on slide organization and on how to stand in front of a senior audience and deliver a briefing. The presenter has 25 years of Pentagon experience in doing and leading analytic projects and then briefing them at the four-star level, including 12 years as the Navy’s SES chief analyst and MORS sponsor. He has 11 years of subsequent experience as the chief analyst at a company with several hundred OR staff that provides analytic work to a wide range of government national security organizations. He was the recipient of the 2024 MORS Vance Wanner award.

Functional Risk Analysis and Consequence Estimation: A National Security Practitioners Guide
Dr. Ruby Booth and Dr. Cyrus Jian Bonyadi
1:00 PM–5:00 PM | Room 3L1

National civilian and military functions face serious risks in our modern world. Yet, the systems that conduct these functions are highly complex, and both decision-makers and risk managers often lack practical, lightweight tools to deal with that complexity. National security practitioners would benefit from a shared, straightforward approach to managing those risks.

When seeking to manage risk to national civilian or military functions, practitioners face many challenges – the greatest of which are evaluating and communicating the complexity of these functions in a useful manner. Calculating the hazard impact requires, first, a robust approach to modeling these complex systems, then, a means of making the approach tractable.

Use of a Function-Behavior-Structure ontology allows risk analysis of disruption based on the consequences of partial or complete failure of specific function in a specific region for a specific duration. Risk mitigation also requires knowing which assets support these functions and who owns, manages, and/or maintains those assets. In this tutorial, participants will learn to conduct a functional decomposition alone or with stakeholders, estimate consequences of disruption, and prioritize mitigations given limited resources.

SNL is managed and operated by NTESS under DOE NNSA contract DE-NA0003525.

Building Trust in Cutting Edge AI: Observability & Evaluation in LLMs and Agentic AI Systems
Amir K. Saeed
1:30 PM–5:00 PM | Room 3L6

Large Language Models (LLMs) and agentic AI systems are increasingly used in decision-support tools, data analysis pipelines, and automated reasoning workflows. However, these systems introduce major challenges in reliability, evaluation, and transparency due to their open-ended outputs, multi-step architectures, and non-deterministic behavior. Traditional monitoring and testing approaches are often insufficient for understanding or improving their performance. This tutorial introduces practical methods for observability and evaluation in LLM and agentic AI systems. Participants will learn how to instrument AI pipelines using trace-based observability techniques that capture spans, traces, and key operational metrics such as latency, token usage, and tool invocation success. The session also demonstrates LLM-as-a-judge evaluation, which uses language models to assess outputs for correctness, relevance, and task completion. Through an example agentic pipeline, attendees will learn strategies for diagnosing failures, improving prompts and system design, and building reliable AI systems for real-world operational environments.

How to Validate Your Models and Simulations
Dr. Averill M Law
2:00 PM–4:00 PM | Room 3L4

All models and simulations are surrogates for physical experimentation with the system of interest, which is usually impossible, disruptive, or not cost-effective. Thus, if a model is not reasonably “valid,” then any conclusions drawn from the model results might, very well, be erroneous. In this tutorial we present a comprehensive set of techniques for building valid and credible simulation models, and for validating existing models. Ideas to be discussed include the importance of a definitive problem formulation, discussions with subject-matter experts, interacting with the decision-maker on a regular basis, development of a written “assumptions document” (not the same as a requirements document or conceptual model), structured walk-through of the assumptions document, use of sensitivity analysis to determine important model factors, comparison of model and system output data for an existing system (if any) using numerical statistics and graphical plots, and comparison of model output data with the comparable output data from another model that is thought to be “valid.” Each idea will be illustrated by one or more real-world examples. We will also discuss the considerable difficulty in using formal statistical techniques (i.e., confidence intervals and hypothesis tests) to validate simulation models, due to the unavailability of model and system output data with the correct characteristics.