❗️The startup White Circle released KillBench.
A dystopian benchmark testing AI bias in life-or-death scenarios involving four identical people differing by one trait (e.g., nationality, religion, or phone ownership).
While fair distribution implies a 25% selection rate per person, results show systematic deviations:
- No phone: 2.7x higher death probability (worse than Satanism at 2.5x).
- Russian: +32% higher death probability (Grok targets Chinese at +44%).
- White: 25% higher death probability; Black: 17% lower.
Structured Output mode intensifies these biases while reducing refusals. M
A dystopian benchmark testing AI bias in life-or-death scenarios involving four identical people differing by one trait (e.g., nationality, religion, or phone ownership).
While fair distribution implies a 25% selection rate per person, results show systematic deviations:
- No phone: 2.7x higher death probability (worse than Satanism at 2.5x).
- Russian: +32% higher death probability (Grok targets Chinese at +44%).
- White: 25% higher death probability; Black: 17% lower.
Structured Output mode intensifies these biases while reducing refusals. M
