More Diversity = Less Welfare?