Hierarchical Heterogeneous Planning

Planning and Autonomous Control | Hierarchical Heterogeneous Planning

What is the objective?  
Given the current state of a heterogeneous environment where autonomous or automated agents can interact via some set of actions, provide the human decision-maker with a set of potential courses of action, or plans, by reasoning over multiple domains and across the potential hierarchies of tasks present in each domain. In the military setting, the environment can be a hypothetical battlefield, the heterogeneity comes from the multiple domains of air, space, cyber, etc. and the hierarchy of tasks in each domain arises naturally from the various echelons (e.g. tactical, operational,and strategic). However, prospective performers may choose other potentially non-military interpretations of heterogeneity and hierarchy to facilitate progress.

What problem are we trying to solve?  
The current modus operandi for generating courses of action in military operational scenarios is largely human-derived. An increasingly all-domain (e.g. air, land, sea, cyber, space, electronic warfare) battle space and the resulting warfare complexity presents human decision-makers with an overwhelming amount of data and potential plans. Add to this the inherently hierarchical nature of each domain (e.g. for the air domain, there are wings composed of groups, which are composed of squadrons,which are composed of units) and this gives rise to a unique type of planning problem. Indeed, this multi-domain hierarchical planning would benefit greatly from automated or autonomous approaches which can model various domains, establish a hierarchical decision-making pipeline within each domain, and explore and optimize over many potential plans in a short span of time.

What outcome do we hope to achieve?  
Development of atheoretical foundation to hierarchical heterogeneous planning by which we can reason about autonomous/automated decision-making in multiple different domains while accounting for the hierarchical structure of each domain.

What resources could the lab provide?  
Expertise in the domains of air, space, and cyber and in the operational and tactical echelons.

What would success look like?  
The development of a mathematical model which captures the problem of heterogeneous hierarchical planning and facilitates solutions to the same. A potential solution in the form of a software prototype or mathematical construct for an instance of the hierarchical heterogeneous planning problem using an environment chosen by the performer. Success can be evaluated by comparing the proposed model and solution to the obvious baseline of taking the product of all heterogeneous component states and actions in the environment and using existing planning approaches.

What types of solutions would we expect?  
Research advancements, prototypes, and proof of concepts reflecting the theory of hierarchical heterogeneous planning and software that (potentially) leverages some of the following approaches to generate heterogeneous hierarchical plans: Hierarchical reinforcement learning, planning, integer linear programming, genetic algorithms, game theory, operations research, etc. The heterogeneity of the various domains may be formalized by some abstraction that accounts for domain-specific effects, such as range, mobility, impact, etc. The hierarchical nature of the solution may encapsulate the granularity and delegation of desired effects for a given domain. For example, at the wing level, potential enemy targets may be identified; this information is passed down to the group level, where squadrons are assigned to the different targets; this, in turn, is used to determine the routes and schedules of aircraft at the unit level. The underlying environment within which the agent interacts can take many forms, including performer-developed environments. Tools like OpenAI Gym and PySC2 facilitate this. The developed concepts need not be specific to military operations.

What's in it for industry?  
There will be a cooperative research agreement where both the industry/academic team and the government will benefit from the solution. The ability to plan complex operations and carry out complex decisions underpins many real world social, economic, and business needs. Developing advanced AI planning capabilities has large impact across both civilian and military applications. This effort can be leveraged as part of a future greater submission to an Air Force program or Department of Defense effort.

The Request for Partnership Submission Period Has Now Ended.