The authors test whether earlier findings about algorithmic refugee matching (which suggested better employment outcomes when refugees are matched to locations using predicted outcomes) hold up when you change the evaluation method. They re-analyze the same US refugee data using several different off-policy evaluation techniques—ways to estimate what would have happened under a different assignment policy without actually running that policy. The core result is that the estimated gains from algorithmic matching stay consistent across methods: all approaches show similar positive impacts, and most reach statistical significance.
Main takeaways:
- Off-policy evaluation estimates the impact of a counterfactual policy (algorithmic refugee matching) using data from the actual policy (administrative assignments)
- Multiple estimation methods (inverse probability weighting, augmented inverse probability weighting variants) all produce similar impact estimates
- The robustness holds across different modeling architectures and assignment procedures
- Results remain consistent with the original 2018 Bansak et al. findings, strengthening confidence in the refugee-matching gains
- Most estimation approaches achieve statistical significance, though the exact confidence varies by method