Please consider writing a post on audit studies conducted by sociologists that purport to show unfounded discrimination because, for example, resumes of fictional black candidates are less successful than those of fictional white candidates otherwise equivalent. (Example: https://thesocietypages.org/socimages/2015/04/03/race-criminal-background-and-employment/) Your reasoning helps explain such findings without requiring the kin of racial discrimination we should discourage. These studies have been quite influential, both in academia and in mainstream journalism.
I think an issue is in how the findings of such studies are communicated or interpreted in various fields. Many such audit studies merely test whether group membership affects outcomes. Interpreting the results as indicative of unjust/taste-based discrimination often requires additional assumptions about risk, incentives, and priors that are rarely defended. If discrimination were primarily taste-based, adding information shouldn’t reduce disparities, so the fact that it often does is evidence that uncertainty and risk management are mostly to blame. For example, when resumes include more information on stable work history, references, credentials, verified reliability, etc, the group gaps tend to shrink.
Heckman and Siegelman showed in the 90s that even unbiased decision-makers will appear discriminatory in audit studies if groups differ in variance and selectors are risk-averse. Though the variance critique is devastating to naïve interpretations and canonical in economics, it seems like most journalists and even some sociologists are unaware of this critique.
Also, I think sticking with evidence/data and avoiding premature moral conclusions is best for any angle. It’s entirely plausible that unjust discrimination plays a decisive role in some contexts, but again, many audit studies are not designed to isolate that mechanism. Multiple causal processes can produce the differential treatment seen. Treating all such findings as evidence of unjust discrimination is analytically unsound and practically counterproductive. If one cares about reducing disparities/improving outcomes, first one must identify which mechanisms are operating.
I would guess the reason this isn’t more clear to most people is due to political/identity-based incentives to treat it as a values debate. The problem is once a field is treated as normative, even many experts also adapt to that environment and emphasize constraints less.
I'm familiar with that paper. However, the variance issue wasn't intuitive to me, nor did I follow the derivation. If you know an intuitive explanation, I'd appreciate reading it.
If you're selecting people and want to avoid bad actors (e.g. below -2 SD as above), then groups with identical means and different dispersion will have different proportions of bad actors. A risk-averse selector will thus prefer the smaller dispersion group since the prior probability of being below -2 z is lower.
In some contexts, it is extremely important to avoid critical failures while getting very high performance is less important. Think of piloting. A very good pilot won't fly a typical charter plane much different than an average pilot. But a very poor pilot might kill everyone.
With adjusted observed score, you also get a "market for inverse-lemons" effect, whereby the best scoring unpaired greens will be better than the best scoring unpaired blues.
Yes. In my calculations, I assume only a utility of -100 for a bad renter versus +20 for a good one. In reality, the ratio of these could be much worse than 5 to 1, could even be worse than 20 to 1. If a bad renter causes severe damage to the building (e.g. water damage in floors), and also doesn't pay rent for months, and you have to hire a lawyer to evict them, I can see how this could result in worse than a 20 to 1 ratio.
Please consider writing a post on audit studies conducted by sociologists that purport to show unfounded discrimination because, for example, resumes of fictional black candidates are less successful than those of fictional white candidates otherwise equivalent. (Example: https://thesocietypages.org/socimages/2015/04/03/race-criminal-background-and-employment/) Your reasoning helps explain such findings without requiring the kin of racial discrimination we should discourage. These studies have been quite influential, both in academia and in mainstream journalism.
Here you go. https://www.emilkirkegaard.com/p/understanding-audit-studies
Thank you! Yet another reason why your SubStack is so valuable.
I think an issue is in how the findings of such studies are communicated or interpreted in various fields. Many such audit studies merely test whether group membership affects outcomes. Interpreting the results as indicative of unjust/taste-based discrimination often requires additional assumptions about risk, incentives, and priors that are rarely defended. If discrimination were primarily taste-based, adding information shouldn’t reduce disparities, so the fact that it often does is evidence that uncertainty and risk management are mostly to blame. For example, when resumes include more information on stable work history, references, credentials, verified reliability, etc, the group gaps tend to shrink.
Heckman and Siegelman showed in the 90s that even unbiased decision-makers will appear discriminatory in audit studies if groups differ in variance and selectors are risk-averse. Though the variance critique is devastating to naïve interpretations and canonical in economics, it seems like most journalists and even some sociologists are unaware of this critique.
Also, I think sticking with evidence/data and avoiding premature moral conclusions is best for any angle. It’s entirely plausible that unjust discrimination plays a decisive role in some contexts, but again, many audit studies are not designed to isolate that mechanism. Multiple causal processes can produce the differential treatment seen. Treating all such findings as evidence of unjust discrimination is analytically unsound and practically counterproductive. If one cares about reducing disparities/improving outcomes, first one must identify which mechanisms are operating.
I would guess the reason this isn’t more clear to most people is due to political/identity-based incentives to treat it as a values debate. The problem is once a field is treated as normative, even many experts also adapt to that environment and emphasize constraints less.
I'm familiar with that paper. However, the variance issue wasn't intuitive to me, nor did I follow the derivation. If you know an intuitive explanation, I'd appreciate reading it.
If you're selecting people and want to avoid bad actors (e.g. below -2 SD as above), then groups with identical means and different dispersion will have different proportions of bad actors. A risk-averse selector will thus prefer the smaller dispersion group since the prior probability of being below -2 z is lower.
Ah. Thanks.
In some contexts, it is extremely important to avoid critical failures while getting very high performance is less important. Think of piloting. A very good pilot won't fly a typical charter plane much different than an average pilot. But a very poor pilot might kill everyone.
Is it plausible to you that affirmative action / non-discrimination results in less ethnic conflict than would otherwise occur?
With adjusted observed score, you also get a "market for inverse-lemons" effect, whereby the best scoring unpaired greens will be better than the best scoring unpaired blues.
Yes. In my calculations, I assume only a utility of -100 for a bad renter versus +20 for a good one. In reality, the ratio of these could be much worse than 5 to 1, could even be worse than 20 to 1. If a bad renter causes severe damage to the building (e.g. water damage in floors), and also doesn't pay rent for months, and you have to hire a lawyer to evict them, I can see how this could result in worse than a 20 to 1 ratio.