The authors tackle constraint inference from demonstrations when different experts have different objectives—existing methods assume all demonstrations come from experts with identical goals, which is unrealistic. Their MOCI framework jointly learns shared safety constraints (rules everyone follows) and individual preferences (what makes each expert different) from heterogeneous expert trajectories using inverse reinforcement learning. On a grid-world benchmark, MOCI significantly outperforms baselines at predicting behavior while maintaining competitive computational efficiency.
Main takeaways:
- Existing constraint-inference methods assume homogeneous demonstrations (all experts share identical objectives), which limits their practical applicability
- MOCI extracts both shared constraints (safety boundaries everyone respects) and individual preferences (what makes experts different) from the same set of demonstrations
- The framework can learn from diverse and potentially conflicting behaviors exhibited by different experts
- Empirically outperforms existing baselines in predictive accuracy on a grid-world benchmark
- Maintains competitive computational efficiency despite handling the more complex heterogeneous setting