Causal Abstractions for Markov Decision Processes
A new research paper introduces a property-driven causal abstraction technique for Markov Decision Processes (MDPs) to address scalability issues caused by exponential state space growth. The method uses causal relations over state variable predicates to identify states sharing reasons for fulfilling or violating a given property. The approach is compared theoretically and empirically across different model types including MDPs, interval MDPs, and stochastic games, showing potential for reducing complexity while retaining model characteristics.
Key facts
- Markov Decision Processes are commonly specified over factored state spaces
- Exponential blowup in state numbers challenges reasoning tasks
- Abstractions reduce MDPs to mitigate scalability issues
- Causal abstraction technique retains many original MDP characteristics
- Relies on causal relations over state variable predicates
- Identifies states sharing reasons for fulfilling or violating a property
- Compared theoretically and empirically across MDPs, interval MDPs, and stochastic games
- Evaluation demonstrates potential of the approach
Entities
—