Bringing Analytical Rigor to Joint Warfighting Experimentation
Joint Forces Staff College
Norfolk, Virginia
3-5 October 2006

Terms of Reference
(Last Updated 19 July 2006)

1.  Overview

Collaborative experimentation with customers is a critical component of the vision for military experimentation.  The Joint Warfighting Experimentation process provides an ideal opportunity to examine the risks inherent in military operations in both the uncertain present and future temporal domains.  This workshop will place a particular emphasis on analytical methods and recent experimentation efforts that address assessing analytical rigor in the military environment.

Without assessment, an experiment is just a demonstration.  Assessment and analysis do not happen at the end of the experimentation process, they are integral to it.  Some of the highest payoff in assessment will be early in the concept exploration phase and must be planned and resourced.  Analytical perspective must be embedded throughout the process, especially planning.  If analysis is integral to the experimentation process, the main point to consider is how analytical rigor can be applied throughout the process.

For instance, how can analytical rigor be applied in experimental design?  What are the right factors and levels?  When designing the experiment, when is the right time to consider data collection and metrics evaluation?  How are the experiment objectives linked to collection and evaluation?  What are the design differences and considerations that should be taken into account when conducting an experiment in a laboratory experiment compared to a field/at-sea experiment?  How do you correlate testing data with simulation data?  What tools are used as experimentation stimulators?  What methodologies are used to analyze the data obtained during the experiment?

Additionally, for the human factor, how do you balance using Subject Matter Experts (SMEs) and the resources available?  What are the social and cognitive domain metrics that should be analyzed?  How do you assess human-in-the-loop captured data in post-experiment analyses?  What types of qualitative data should be captured?

Furthermore, per Title 10, Joint Warfighting Experimentation is much more than just experiments.  It includes analysis, simulations, Advanced Concept Technology Demonstrations (ACTDs), wargames, and joint exercises, in addition to experiments.  How can processes from the whole experimentation definition be used to support the transition from successful ACTD to an acquisition program?  How involved should the Testing and Evaluation community be in the warfighting experimentation arena?

Finally, can the FAME (Full spectrum Analysis employing Modeling techniques, tools, and Experiments) and M-E-M (Model-Exercise-Model, sometimes referred to as Model-Experiment-Model or M-T-M Model-Test-Model) techniques be used to apply analytical rigor to experimentation?  Should the DoD Command and Control Research Program (DODCCRP) Codes of Best Practice for Experimentation be adopted as analytical best practices?  How will the DoD collaborate within the DoD and with other U.S. Agencies in experimentation?

This Workshop will research and examine the above questions, as well as many others, in how analytical rigor can be applied in Joint Warfighting Experimentation.

2. Background

The last MORS Special Meeting on Joint Experimentation was conducted in March 1999, at the Armed Forces Staff College in Norfolk, VA.  The goal of that special meeting was to contribute to planning, conduct and exploitation of Joint Experimentation by leveraging the experience and expertise of the analytical community.  The Mini-Symposium was designed to educate the DoD analytical community about the nature of military experimentation, recently completed events, experimental plans and the issues involved and future challenges.  The workshop was designed to identify critical issues and develop conclusions and recommendations to improve Joint Experimentation.

The major findings of the 1999 workshop were in the following areas:

The complete MORS report is available here (it is in portable document format (.pdf)).

Since the special meeting, the Department of Defense Command and Control Research Program (DODCCRP) has published two texts related to experimentation:

Both of these texts build upon the findings from the 1999 Workshop and can be used as a foundation when developing organization-specific experimentation processes.  These texts are available on the DODCCRP web-site (www.dodccrp.org), just click “Publications.”

Finally, recent guidance concerning experimentation is found in the February 6, 2006, Quadrennial Defense Review (QDR) Report.  It states. “(in) experimenting …, the Department will seek to identify and rapidly develop preferred capability are solutions that will facilitate open and agile decision-making.” 

Before discussing the specifics of this workshop – i.e., the goals, objectives, approach, agenda, etc. – a couple of definitions must be presented.  First, according to Title 10, Section 485, Joint Warfighting Experimentation “should include analysis, simulations, wargames, experiments, advanced concept technology demonstrations, joint exercises conducted in virtual and field environments, and, as a particularly critical aspect, assessments of ‘red team’ vulnerability.”

Next, from the draft “Joint Concept Development Experimentation Campaign Plan (Joint CDE CPLAN) 2006-2013,” Joint Warfighting Experimentation is “a process to assess the effectiveness of varying proposed joint warfighting capabilities or conditions.  The purpose is to ascertain whether a joint intervention causes changes in military effectiveness.”  Experiments can be further characterized as:

Analytic Wargame Experiment — employs partial real forces in a simulated environment with human participation as decision makers.

Constructive Experiment — employs simulated forces in a simulated environment supported by force-on-force modeling.

Virtual Experiment — employs partial real forces in a simulated environment with human-in-the-loop (HITL) participation.

Field Experiment — employs real forces in an actual environment as part of a field exercise.

Limited Objective Experiment —a narrowly scoped experiment, analytically focused concept assessment, or prototype validation event.  It provides final dress rehearsal of a concept or major component of a concept prior to its final validation in a full joint warfighting experiment (also referred to as LOE).

Finally, what is meant by analytical rigor in experimentation?  Combining the definitions for analysis, analytical, rigor, experiment, and experimentation from the Encarta^®, Oxford and Merriam-Webster On-Line Dictionaries and the Joint CDE CPLAN , the following definition was developed:

Analytical Rigor in Experimentation – The application of precise and exacting standards in the examination of a question carried out under controlled conditions to better understand and draw conclusions in order to discover an unknown effect, to test a hypothesis, or demonstrate a known fact, and usually based on careful consideration or investigation.

3. Goals and Objectives

Many organizations are interested and involved in Joint Warfighting Experimentation.  The analytic community needs to support the military services with analytical rigor as experimentation is being conducted.  The community should play a leadership role in creating/refining these needed metrics, processes, methodologies, models and simulations.  The community should share efforts, successes and failures in the key capabilities.

This meeting will provide a forum for discussing Army, Navy, Marine Corps, Air Force and Joint approaches to military experimentation.  It will provide an opportunity for military and civilian operations research analysts to examine topics, methodologies, analyses, and innovations pertinent to all aspects of designing, executing, analyzing, and reporting joint warfighting experiments.  

The intent of this meeting is to provide an opportunity to bring a multi-disciplined team of analysts, operators, engineers and academicians from those organizations together to share their work, develop a common view of the state of practice, expose members of the broader analytic community to their needs, identify shortfalls and potential solutions.  Where the goal of the 1999 Joint Experimentation Workshop was to develop an experimentation process and using the two published Codes of Best Practice for Experimentation, the goal of this workshop is to determine how to apply analytical rigor across the process.

There are several overarching issues each working group will consider.  They are:

The overall objectives of this special meeting on Bringing Analytical Rigor to Joint Warfighting Experimentation will be to provide an assessment and a roadmap to revitalize the state of analytical rigor being applied to the practice and to recommend priorities for any initiatives identified.  Some specific objectives for the Working Groups include recommending:

4. Approach

A. Afternoon Before Special Meeting - Tutorial – From 1300-1700 on the afternoon before the Special Meeting, a Tutorial on "Design of Experiments" will be presented for those attendees that arrive early.

B. 1st Day – Mini-Symposium – The meeting will commence with a mini-symposium format that will include operational based discussions as well as progress to date.  The purpose of this portion is to bring all participants up to speed on the state of the practice and frame the analytical challenges and issues for the working groups. 

(1) Keynote Presentation (~ 1 hour presentation with 15 minutes for Q&A) – From a high, Joint-level perspective:

(a) Define and provide a brief background on Joint Warfighting Experimentation

(b) Identify the overriding challenges the analytical community needs to address

(c) Provide an overview of the progress to date using operations analysis to provide analytical rigor to experimentation, including any problems and paradoxes

(2) Joint Panel Discussion (total – 1 ½ hours with ~ 15 minutes for Joint Organizations and ~ 15 minutes available for questions) –

(3) Service/Coalition Panel Discussion (total – 1 ½ hours with ~ 10 minutes for each Service and ~ 30 minutes available for questions) –

(4) Framework Presentations ( ~ 1 1/4 hours) –

(5) Guidance Presentations (~ ½ hours)  –

C. 2nd & 3rd Day – Workshop – The Mini-Symposium will be followed by a two-day workshop where participants will meet in working groups to further examine specific topics, including discussing the overarching issues of the Workshop.  Working groups will prepare a report on their activities to present to other workshop participants at the last session of the workshop.  To focus the discussion in each of the working groups, a select group of people will be requested to prepare and present papers.  The workshop attendees will be organized into six working groups plus a synthesis group.  The working group structure is detailed below.

(1) WG 1:  Experiment Design – DoD uses simulation models to support its decision-making process.  Defense analysts need experimental designs capable of efficiently searching an intricate simulation model that has a high-dimensional input space characterized by a complex response surface.  Instead of using a simple experimental design, many analysts end up making runs to measure performance from only a single system specification, or they choose to vary of handful of the many potential factors one-at-a-time.  Their efforts are focuses on building – rather than analyzing – the simulation model.  Design of Experiments research are often found in specialty journals, making it difficult for simulation analysts to find out about the variety of methods available.  This working group will explore designing experiments.  Some potential discussion points include:

(a) How do the critical key questions drive the experimental design?

(b) What about factors and levels?

(c) Beyond modeling and simulation, how can experiment design be applied to experimentation that takes place in a live environment that analyzes new CONOPS and TTPs?  How do you use design of experiments to design operational tests in Test and Evaluation?  How can simulators be integrated into live wargame designs to compensate for lack of human resources (as role players)?

(d) What are the design differences and consideration that should be taken into account when conducting an experiment in a laboratory compared to a field/at-sea experiment/exercise? What about a combined laboratory experiment and field experiment?  If so, what are they?  (formerly part of the old WG 3)

(e) Are there approaches for both individual organizations and the broader concept development and experimentation community designed to increase the effectiveness and efficiency over current approaches?

(f) Are there any approaches from the Experimentation Design Template (from NITEWORKS) from the TTCP organization that can be adapted?

(g) How are experiments designed to specifically solve problems?

(h) Transformational Issues (TI’s) are military problems or challenges that through analysis of key areas, create innovative solutions (i.e., new ideas, attributes, competencies, or sources of power) the Joint Warfighter needs against irregular, disruptive, traditional, and catastrophic security threats.  How can TI’s be included in the experiment design?

(2) WG 2:  Data Collection, Metrics Evaluation and Reports – It has been said that “an experiment without good data is just a training exercise.”  When designing the experiment, when is the right time to consider data collection and metrics evaluation?  How do you link the experiment objective to data collection from the systems and simulations in the experiment using the future Joint Warfighting metrics listed in the Joint Operating Concepts (JOCs), Joint Functional Concepts (JFCs), and Joint Integrating Concepts (JICs) under the Conceptual Framework for Net-Centric Operations and the Net-Centric Warfare Tenets?  This working group will discuss these questions as well as others associated with data collection, metric evaluation and reports.  Some potential discussion points include: 

(a) How is the experiment’s objective(s) linked to data collection and metrics evaluation?  How is a testable hypothesis developed from the experiment objective?  How are the dependent and independent variables are determined and operationalized?

(b) Today’s Joint Warfighting Experimentation is a distributed activity with elements across the DoD, Interagency, and other nations.  As each of these organizations uses a somewhat different experimentation design approach, the resultant data often has different characteristics.  Are there methods available to normalize quantitative and qualitative data in a manner that supports development of findings and conclusions across the body of community work?

(c) The collection requirement will dictate the number, type and qualifications of SMEs/data collectors and the quantity and type of instrumentation needed.  What are the “right” types of data to be collected manually.

(d) How do you represent the analytical reports early in the experiment design process to ensure the desired results will occur?

(e) How should data collection and metrics evaluation be reported to be of most post-experiment value?

(f) How can data collection and metrics evaluation be applied to gain the most ROI for the customer?

(g) How do you achieve meaningful results within a time constrained experimentation cycle, i.e., rapidity of results?  Are there methods to design activities that provide both a reasonable level of analytical rigor, that can be conducted rapidly, and modified to address cycle constraints?

(3) WG 3:  Methodologies and Tools – What tools are used as experimentation stimulators and why?  What methodologies are used to analyze the data obtained during the experimentation?  How are qualitative assessments used?  What tools are used to report the results?  These questions will be discussed in this working group as well as others associated with experimentation methodologies and tools.  Some potential discussion points include:

(a) What advantages are available with the methodologies developed for JEFX ’06 (i.e., analysis framework, process for developing operational threads, and toolsets)?

(b) Are there any social science tools that can be used in experimentation?  If so, how?

(c) What are the new approaches that are available?

(d) Are the key critical questions answerable with other tools instead of experimentation?

WG 4: People as Experimental Assets:  Applying SME and Human-in-the-Loop (HITL) Information – The knowledge and experience that Subject Matter Experts (SME) bring to experimentation are powerful tools for increasing analytical strength and validity.  Like all powerful tools, they must be employed properly, or they can do as much damage as good.  SME provide knowledge that cannot be garnered in other ways, but they compete with all other assets for the resources of the sponsor.  They have experience that can shed light on difficult issues, but may commensurately have interests that diverge from those of experiment’s objectives.  How do you balance the use of SME and other assets using the resources available to achieve the best possible result for your experiments?

Like SME, using real operators in experiments both provides great value and offers great challenges.  The knowledge, skills, and abilities of operators provide a rich context of millions of detailed conditions to an experiment that would otherwise not be available.  Likewise, these conditions are for the most part uncontrolled and difficult to measure.  How do we account for the wealth of detail added to an experiment by the use of live operators?  How do we rigorously assess the cognitive factors of concern?

Some potential discussion areas include:

(a) How do you classify and compare the impact of SME knowledge and experience to the impact of other assets on the quality of your experimental results?

(b) How do you manage the controlled and uncontrolled aspects of SME knowledge and experience WRT the design of your experiment?

(c) What types of unique influences on performance that live operators have on an experiment add value to the results?

(d) Do we post-validate the HITL results?  How do you account for different aspects of DOTMPLF (e.g, level of training of participants versus actual anticipated users)?

(e) What types of qualitative data should be collected to characterize human performance?  What are the social and cognitive domain metrics that should be captured?  How do you report these results? 

(f) How is HITL data captured and fed back into mission- and campaign-level warfighting models to enhance their own representation of the battlespace? 

(g) By conducting HITL experiments, does that slow down the analytical process?  If so, what are the costs (i.e., time, effort, etc.) and benefits?

(5) WG 5:  Integrating and Coordinating Experimentation across DoD – How do we integrate and coordinate experimentation across the Department of Defense, especially when different Services are looking at the same issue?  What about collaboration among Services in the same experiment?  These questions will be discussed in this working group to help analysts produce a better experimental design for warfighting experimentation.  Some potential discussion points include:

(a) How is data shared among the Services when collaborating on the same experiment?  How is data correlated?

(b) How can result be shared among the Services when experimenting on the same issues?

(c) How do we incorporate the networked battle command experiments?

(d) As the Joint Experimentation Group, does JFCOM/J-9 coordinate these efforts/

(e) With Commanders hesitant to allow experimentation in their training events, what are the considerations for conducting experiments during exercises and training events?

(f) What is the overlap among operational testing, experimentation, and wargaming among the Services?

(6) WG 6:  Applying Analytical Rigor across the Joint Warfighting Process – Per Title 10, Section 485, Joint Warfighting Experimentation “should include analysis, simulations, wargames, experiments, Advanced Concept Technology Demonstrations (ACTDs), joint exercises conducted in virtual and field environments, and as a particularly critical aspect, assessments of ‘red team’ vulnerability.”

(a) How can the whole experimentation process be used to support the transition from concept to an acquisition program?

>> How can you best use data generated during a proposal phase to use during the acquisition phase of a program? Can you show a connection to data generated during initial phases? Can you track the simulation results from proposal work and show why new results are different?

>> Working against you are: Changes in personnel; personnel moving to other jobs and having limited time to assist; organization of data - can you find what you are looking for?; and, the pace of concept phase may not lend itself to documentation work. 

(b) What kinds of data, metrics, and evaluation should be done for consistency among the steps and used to best support follow-on developmental and operational testing?  Data management. Types of data generated. Are the metrics comparable to initial work done during the concept phase?  What measures of effectiveness are being used in the contract phase as opposed to the concept phase?

(c) What is the effect of cost on the experimentation process?

(d) How does experimentation fit into and support capabilities based planning?

>> How does the acquisition community use experimentation results?  Experimentation is essential in providing system effectiveness data to the customer; hardware does not exist in initial phases of a program, so you are dependent on experimentation; and design development depends on experimentation to refine requirements. 

>> What would the acquisition community prefer to see from experimentation?  Does the experimentation reflect real mission scenarios?  Are the experiments relevant to current conflicts?  How do you stay current with ongoing scenarios?  Does the nature of contractual process and creation of program specifications limit your ability to design up-to-date scenarios?

(e) How involved should the Test and Evaluation community be in the warfighting experimentation arena?  Should they be kept separate, or is there value in brining in testers for some early involvement?  The test and evaluation community can bring analytical rigor to early experiment using the Design of Experiments methodology and providing real test context to experimentation.

(f) What about experiment design from the computer to the field?

>> What are the challenges associated from the road of conception, CONOPS generations, simulation, field experimentation, and finally, transition to employment for a new process or equipment?  How can you deal with the current time lag in model upgrades, lab upgrades when trying to remain current?  What is the best way to demonstrate the connection of various types of experiments?  Should CONOPS drive the experiments or vice-versa?  When is it time for hardware testing?  Should you wait until the design is set?

>> What is the overlap among operational testing, experimentation, and wargaming?  How can you correlate or at least show similarities between wargaming and experimentation?  Inherent setup and requirements differences can work against you in making comparisons. Matching setup conditions and understanding differences is key to correlation.

(g) How is modeling and simulation incorporated into operational tests?  What types of systems are you capable of testing in operations tests?  When does it make sense to model/simulate a system rather than perform an operational test?  Can you design a checklist to assist you in making that decision?

(h) How can operationally-obtained field data be used as an experimental baseline, or to validate experimental results?  Can contractors obtain field data in a timely manner in order to effect designs?  Is it possible to coordinate with field test personnel so data is more readily available?  Can contractors use matching scenarios in their experiments?

(i) What are the similarities in each of these phases (perhaps common metrics) and differences (perhaps experimentation resources)?

>> How do you correlate the data across the phases?  Are the metrics really the same?  Should they be the same?  How can you correlate inputs to make results more compatible?

>> How do you correlate testing data with simulation data?  The key is correlating inputs.

>> How do you correlate data from government and contractor human-in-the-loop laboratories?   How do you apply analytical rigor to humans?  What methods/techniques can be used to apply analytical rigor to human-in-the-loop experiments?

(j) Would the Model-Exercise-Model (M-E-M) technique discussed in the 1999 Joint Experimentation workshop be applicable across the whole process so that the complete set of analytic issues are addressed by the appropriate tool, whether that is a model, or a live exercise event?  Does cost limit your ability to test as often as you would like?  Are we severely limited by the capabilities of simulation models?  Can model developers keep up with the change in systems?  Can threat models keep up with advancement?  Can you design your program to simulate and test appropriate capabilities at incremental times during life of program? 

(k) How could Full spectrum Analysis employing Modeling techniques, tools and Experiments (FAME) be applied across the whole process?  Does the modeling and test community need to have more coordination?  Does the funding process for labs and models severely hinder this coordination?

(l) How can the linkages between experimentation and acquisition through the Joint Capabilities Integration and Development System (JCIDS) be better defined?  How should the results of experimentation be presented to the JCIDS?  How can the JCIDS process give seamless feedback to the experimentation community?  What are the best metrics and measures to support capabilities entering the JCIDS?

5. Agenda (Tentative)

The tentative utilization for the working group sessions will be:

(1) Working Group Session # 1 – Kickoff:  Introduction, agenda, issues & goals; and Provide context to orient WG participants for discussion and debate

(3) Working Group Session # 2 – Technical Papers / Discussion Session #1

(4) Working Group Session # 3 – Frame WG response & collect issues (brainstorming)

(5) Day 2 Optional Working Group Discussions – After lunch (Consider: Speaker to address frontier issues on WG topic)

(6) Working Group Session # 4 – Characterize OA analytical rigor and assess gaps/shortfalls

(7) Working Group Session # 5 – Recommend strategies and roadmaps

(8) Working Group Session # 6 – Refine ideas, arguments, capture WG debate, etc.

(9) Day 3 Optional Working Group Discussions – After lunch, complete presentation for WG Outbrief

6. Attendees

a. Attendance will be by invitation only.  Attendees will include invited experts from OSD, all Services, the Joint Staff, University Affiliated Research Centers, Federally Funded Research and Development Centers, operational commanders, DoD contractors, Department of Homeland Security, US Northern Command, and others, including representatives from our Allied / Coalition Analytical Communities .  Workshop chairs will control membership of their sessions in conjunction with the Organizing Committee.  Attendance will be limited to 200 people.

b. Working Groups (WGs) will be led by a Chair and one to three Co-Chairs.  This leadership group will be comprised of all MORSians or a combination of MORSians and Subject Matter Experts (SMEs).  The responsibilities of this team include:

(1) Chair –

(a) Dynamic individual that is a SME in the WG topic

(b) Solicits analysts and operators in the field to participate in the WG

(c) Guides the WG during the Workshop

(d) Challenged to provide the “substance” of the special meeting WG

(e) Develops the WG’s final product

(2) Co-Chair –

(a) Individual interested in WG topic

(b) Assists Chair in WG membership

(c) provides perspective during Workshop

(d) Assist Chair as Chair requests

c. Another key group of individuals during the Workshop is the Synthesis Group.  This group will provide representation to each of the WGs and assist the Workshop Chairs consolidate the working group results and develop overall assessment/recommendations from the analysis community for the individual service operations analysts to consider.

6. Products

Several products will be generated from the workshop:

8. Milestone Table

See the Bringing Analytical Rigor to Joint Warfighting Experimentation Plan of Actions & Milestones

9. Proponent

Director, Joint Experimentation (J-9), Joint Forces Command

10. Planning and Organizing Committee

General Co-Chairs: Kirk Michealson, Lockheed Martin Center for Innovation and Dr. Michael Cochrane, JFCOM/Joint Experimentation (J-9)

Technical Co-Chairs: Dr. Richard Hayes, Evidence Based Research, Inc and Dr. Dave Alberts, OSD(NII)

Synthesis Chair: Roy Reiss, Air Force Staff / A-9 (S&AA&LL)

Synthesis Group: LtCol Maureen Borgia, OSD(PA&E); Dr. Robert Clemence, Evidence Based Research, Inc.; Dr. Robert Eberth, MCCDC Warfighting Laboratory; Chris Herstrom, Lockheed Martin CforI (WG 33 Chair); Greg Keethler, Lockheed Martin Missiles & Fire Control; Dr. Jerry Kotchka, FS; Terry McKearney, The Ranger Group; Dr. Russ Richards, JFCOM (ORTA Manager); Daniel Serfaty, Aptima, Inc; and, COL Mike Wilmer, Army Capabilities Integration Center

Site Coordinator: Steve Williams, JFSC

Administrative Coordinators: Brian Engler, Executive Vice-President, MORS and Natalie Kelly, Vice-President for Admin, MORS

MORS Bulldog: Lee Lehmkuhl, The MITRE Corporation       

Working Group Chairs:

WG1 – Experiment Design
Chair – Greg Hutto, USAF 53 TMG/OA
Co-Chair – Steve Boothe, COMOPTEVFOR
Co-Chair – Dennis DeRiggi, IDA

WG 2 – Data Collection, Metrics Evaluation and Reports
Chair – Ralph Klingbeil, Naval Warfare Development Command
Co-Chair – Jennie Jastrzembski, Allied Command Transformation
Co-Chair – John Morrison, OSD-P&R (IDA)

WG 3 – Methodologies and Tools
Chair – Scott Hamilton, ACC AFC2ISRC/AFEO (L-3 Comm)
Co-Chair – Doug Clark, ACC AFC2ISRC/C2B (JHU/APL)
Co-Chair – Captain Mike Babilot, USMC, JFCOM / Joint Command – Future
Co-Chair – Bill Euker, ACC AFC2ISRC/AFEO (L-3 Comm GSI Corp)

WG 4 – People as Experimental Assets:  Applying SME & HITL Information
Chair – Alex Hoover, Sparta, Inc.
Co-Chair – Abe Meilich, Lockheed Martin IS&S Net-Centric Integration

WG 5 – Integrating and Coordinating Experimentation across DoD
Chair – Dr. Mike McGinnis, VMASC
Co-Chair – Dr. Robert Tyler, Evidence Based Research, Inc.
Co-Chair – LTC Bryan Luke, Joint Chiefs of Staff (J-7)

WG 6 – Applying Analytical Rigor across the Joint Warfighting Process  
Chair – Teresa Wilson, Lockheed Martin Aeronautics Ft. Worth
Co-Chair – Col Pete Vanden Bosch, Air Force Staff / A-9 Analysis Integration
Co-Chair – Kathleen Baldino, Joint Chiefs of Staff (J-7)
Co-Chair – CDR Kelly Cormican, COMOPTEVFOR

Sponsor/Service Reps:
Air Force: Roy Reiss, HQ USAF/A9
Army: David Markowitz, Center for Army Analysis
Navy: Herb Cupo, OPNAV(N81)
Marine Corps: Col Gregory Reuss, MCCDC Studies and Analysis
Joint Staff: Robert Orlov, JS (J8)

11. Administrative

Name: Bringing Analytical Rigor to Joint Warfighting Experimentation:  Design, Planning, Execution, Analysis and Reporting
Dates: 3-5 October 2006
Location: Joint Forces Staff College, Norfolk, VA
Fees: 

Attendance: 200 people, by invitation

Classification: Unclassified

Some of our links are in portable document format (pdf); they will be indicated by this symbol . To read these files you must have Acrobat Reader^®. You can download Acrobat Reader^® by following this link .

Please read our disclaimer.
Send mail to corrina@mors.org with questions or comments about this web site.
Copyright © 1997-2006 Military Operations Research Society
Last modified: October 20, 2006