In spite of our good intentions and explicit egalitarian convictions, we habitually disfavor the underprivileged. The rapidly growing literature on implicit bias – unconscious, automatic tendencies to associate negative traits with members of particular social groups – points towards explanations of this dissonance, although rarely towards generalizable solutions. In a recent paper, Jennifer Saul (2013) draws attention to the alarming epistemological problems that implicit bias carries with it; since our judgments about each other are likely influenced by implicit bias, we have good reason to doubt their veracity. In this paper we explore a novel way to come to terms with the epistemological problem as it manifests itself in ranking situations, i.e. how we can know that the way in which we have ranked a group of people for a certain position, reflects their actual competence (or the best estimate of their competence given the evidence). On our approach, rather than attempting to make people less biased, we suggest that biased behavior can sometimes be corrected after the fact. In particular, the veracity of rankings can sometimes be improved by modifying the rankings directly. We investigate three methods that modify biased rankings, and argue that the last of these solves the epistemological problem that we are concerned with.