Anyone looking into this stuff needs to read "The impossibility of “fairness”: a generalized impossibility result for decisions"[0]
One type of fairness is always trading off against another, and any presentation suggesting you can just magically increase the fairness of your classifier is straight-up lying.
It's not surprising to me that it is impossible to minimize three different functions over the same parameters simultaneously. That would essentially imply their are the same function, right? But while that is a mathematical impossibility result, it doesn't imply that making things more fair is impossible. An ML algorithm that doesn't consider fairness at all might be adjusted to improve on all three of the proposed metrics without violating the findings of that paper.
If the objective function was a separable function of the different parameters you could simultaneously optimize over them with ease, but that's obviously not the case very often in life.
In general optimizing multiple objectives (or one objective subject to additional constraints) would achieve a compromise between them. If you are trying to simultaneously maximize equality of opportunity and equality of outcome you would just achieve a trade-off where neither side is happy. On the other hand if you were starting out just trying to optimize for profit, then adding a constraint or additional objective will just mean you make less profit in achieving that other objective. I suppose that could be called more fair if you're trading off profit for fairness.
If anyone would like a beginners introduction to multi-objective optimisation and understanding trade-offs you could check out this (my) video https://www.youtube.com/watch?v=56JOMkPvoKs. Real-world problems are often multi-objective with conflicts between the considered objectives.
> it is impossible to minimize three different functions over the same parameters simultaneously.
There is a topic called "multi-objective" optimization. Once I took a course in it from a guy, later President at CMU, who was one of my Ph.D. dissertation advisors.
There is a geometric concept, the non-inferior set where it is impossible to do better on all the functions at once.
There is a term, a condition, a situation called Pareto optimality based on the non-inferior set.
Sometimes finding a non-inferior set is a tweak of ordinary sorting.
I used some of that once: The main costs in flying an airplane are due to (i) fuel and (ii) maintenance based on flight time (in some cases also landings). So, buried deep in the math and code, I was able to make the code run faster by keeping just the non-inferior set -- it was easy to prove that this was sufficient, and I found that set via a tweak of sorting. E.g., in part I was looking for flight plans, so I considered only ones non-inferior on both fuel cost and flight time cost.
Also the OP is talking about false positives and false negatives. Commonly it is relatively easy to be able to adjust, set, and know the rate of false positives. Then the challenge is, for that rate of false positives, find the lowest rate of false negatives. Sure, for a given detector, there may be a cost for a false positive and also a cost for a false negative. Then we would want, say, to pick the false positive to minimize the sum of the two costs.
E.g., just now some travelers may be put into quarantine for two weeks because a false positive where we would conclude they are sick when they are well and a false negative would be that we conclude that they are well when they are sick. So, the cost of the false positive would be the quarantine but the cost of the false negative would be the risk of a quarantine of a city or a pandemic for the world.
I got into this material due to a paper I wrote on anomaly detection for server farms and networks: There maybe a false positive has the system management staff waste some time/effort while a false negative might risk a security breach that might bankrupt the company.
Lesson: The costs of (i) false positives and (ii) false negatives might be very different.
Given a detection problem and a rate of false positives, how do we design the detector that will give the lowest rate of false negatives? There is an answer in the classic Neyman-Pearson theorem. Once I worked out a proof based on the Hahn decomposition based on the Radon-Nikodym theorem in measure theory. The problem can be a knapsack problem in optimization, known to be in NP-complete.
In optimization we have to keep in mind that commonly finding a feasible solution can be about as difficult as finding an optimal solution -- in particular, just finding a feasible solution can be very difficult.
For optimization, it is true that even if the feasible region is discrete -- e.g., with some positive integer n, the set of real numbers R, and the set of all points, vectors, in R^n where each vector has components only 0 or 1 -- commonly we can set up a dual problem that is continuous, differentiable, and convex and that will help finding an optimal solution or at least a good approximation to one. E.g., once I had a 0-1 integer linear programming problem with 40,000 constraints and 600,000 0-1 variables and using this duality approach found a feasible solution with objective function within 0.025% of optimality -- this duality method can provide such bounds. So here I may be helping with the remark in the OP about "non-differentiable".
Constrained optimization is great and probably helpful but calling it a path to fairness misses what makes something fair.
Fairness is not a result, it is a process. Any time people come with a model they made in a closed room that is going to make important decisions for or about other people fairness is impossible.
Fairness is the ability to appeal decisions, fairness is taking part in building the model, fairness is voting on whether the model is suitable and giving consent before your data can be fed to the model in development or production.
Optimization strategies can not achieve any of these things. We're not a little numerical tweak from fairness.
Question : In the last chart, we can see the constrained model reduces the FPR for both categories, old and young. However the "old" category still has more than 2 times the FPR of the young category, how did it make the model fairer? (I would imagine an equal FPR between both categories for it to be fair)
This probably won't solve all fairness problems but having such a library is a good step in the right direction, great they're making this available for free!
An AI model is another word for a statistical model of reality, a small scientific theory, which actually works.
It is the most unbiased thing you can have. It's pure applied science/maths. You are only interested in the input that tell you something, and figure out which one to which degree.
You don't trust intuition or common knowledge. You don't trust anything that might introduce bias. You just look at the data and build the best decision mechanism that can be derived from it.
The degree to which you feel you need to "adjust" that is the degree to which you denounce science. You prefer your bias over science.
Today it's probably the cultural marxism / structural oppression narrative that inspires such manipulation. At other times it would have been infallibility of the papacy, or some other brain construct.
If you feel the need to adjust an ML model, be aware how strongly you feel that way. That's the degree to which you reject science.
Then proceed. Or reverse course. But be honest with yourself about what you're doing.
You are missing the point of the entire article here. Quite aside from the fact that you can obviously have biased models because of their design or because of their training data, the key misunderstanding in your comment is this:
> You just look at the data and build the best decision mechanism that can be derived from it.
How do you define "best"? That's the issue. Not all errors are equal, not all distributions of errors are equivalent even if the total is the same.
And that classifier is bad at determining who can pay the loan back when looking at the blue group.
Even using a very short term, entirely selfish view this can be bad for the loan company. It becomes clear that blue group people are being denied loans they could well afford, and so people in that group start moving over to other providers.
If the populations of blue groups are geographically clustered, this may mean losing large portions of business in certain areas, resulting in shutting down local offices if there's a physical presence (e.g. banks).
This is also entirely aside from legal concerns.
The best model is rarely found by simply optimising a basic measure with no context.
In terms of the statistical learning framework, the Bayesian optimal classifiers are NP hard to find under distribution free conditions. The models have rather large confidence intervals to compare with each other. Scientific methods can only lead you to a certain level of certainty, the rest are purely subjective.
One type of fairness is always trading off against another, and any presentation suggesting you can just magically increase the fairness of your classifier is straight-up lying.
[0]: https://arxiv.org/pdf/1707.01195.pdf
In general optimizing multiple objectives (or one objective subject to additional constraints) would achieve a compromise between them. If you are trying to simultaneously maximize equality of opportunity and equality of outcome you would just achieve a trade-off where neither side is happy. On the other hand if you were starting out just trying to optimize for profit, then adding a constraint or additional objective will just mean you make less profit in achieving that other objective. I suppose that could be called more fair if you're trading off profit for fairness.
There is a topic called "multi-objective" optimization. Once I took a course in it from a guy, later President at CMU, who was one of my Ph.D. dissertation advisors.
There is a geometric concept, the non-inferior set where it is impossible to do better on all the functions at once.
There is a term, a condition, a situation called Pareto optimality based on the non-inferior set.
Sometimes finding a non-inferior set is a tweak of ordinary sorting.
I used some of that once: The main costs in flying an airplane are due to (i) fuel and (ii) maintenance based on flight time (in some cases also landings). So, buried deep in the math and code, I was able to make the code run faster by keeping just the non-inferior set -- it was easy to prove that this was sufficient, and I found that set via a tweak of sorting. E.g., in part I was looking for flight plans, so I considered only ones non-inferior on both fuel cost and flight time cost.
Also the OP is talking about false positives and false negatives. Commonly it is relatively easy to be able to adjust, set, and know the rate of false positives. Then the challenge is, for that rate of false positives, find the lowest rate of false negatives. Sure, for a given detector, there may be a cost for a false positive and also a cost for a false negative. Then we would want, say, to pick the false positive to minimize the sum of the two costs.
E.g., just now some travelers may be put into quarantine for two weeks because a false positive where we would conclude they are sick when they are well and a false negative would be that we conclude that they are well when they are sick. So, the cost of the false positive would be the quarantine but the cost of the false negative would be the risk of a quarantine of a city or a pandemic for the world.
I got into this material due to a paper I wrote on anomaly detection for server farms and networks: There maybe a false positive has the system management staff waste some time/effort while a false negative might risk a security breach that might bankrupt the company.
Lesson: The costs of (i) false positives and (ii) false negatives might be very different.
Given a detection problem and a rate of false positives, how do we design the detector that will give the lowest rate of false negatives? There is an answer in the classic Neyman-Pearson theorem. Once I worked out a proof based on the Hahn decomposition based on the Radon-Nikodym theorem in measure theory. The problem can be a knapsack problem in optimization, known to be in NP-complete.
In optimization we have to keep in mind that commonly finding a feasible solution can be about as difficult as finding an optimal solution -- in particular, just finding a feasible solution can be very difficult.
For optimization, it is true that even if the feasible region is discrete -- e.g., with some positive integer n, the set of real numbers R, and the set of all points, vectors, in R^n where each vector has components only 0 or 1 -- commonly we can set up a dual problem that is continuous, differentiable, and convex and that will help finding an optimal solution or at least a good approximation to one. E.g., once I had a 0-1 integer linear programming problem with 40,000 constraints and 600,000 0-1 variables and using this duality approach found a feasible solution with objective function within 0.025% of optimality -- this duality method can provide such bounds. So here I may be helping with the remark in the OP about "non-differentiable".
Fairness is not a result, it is a process. Any time people come with a model they made in a closed room that is going to make important decisions for or about other people fairness is impossible.
Fairness is the ability to appeal decisions, fairness is taking part in building the model, fairness is voting on whether the model is suitable and giving consent before your data can be fed to the model in development or production.
Optimization strategies can not achieve any of these things. We're not a little numerical tweak from fairness.
One way you could argue this example is fairer: the gap used to be ~10%, now it is ~5%.
It is the most unbiased thing you can have. It's pure applied science/maths. You are only interested in the input that tell you something, and figure out which one to which degree.
You don't trust intuition or common knowledge. You don't trust anything that might introduce bias. You just look at the data and build the best decision mechanism that can be derived from it.
The degree to which you feel you need to "adjust" that is the degree to which you denounce science. You prefer your bias over science.
Today it's probably the cultural marxism / structural oppression narrative that inspires such manipulation. At other times it would have been infallibility of the papacy, or some other brain construct.
If you feel the need to adjust an ML model, be aware how strongly you feel that way. That's the degree to which you reject science.
Then proceed. Or reverse course. But be honest with yourself about what you're doing.
> You just look at the data and build the best decision mechanism that can be derived from it.
How do you define "best"? That's the issue. Not all errors are equal, not all distributions of errors are equivalent even if the total is the same.
That's very simple with the example in the article. Who can pay back the loan best? You just fear the answer and rather twist up your reasoning.
Even using a very short term, entirely selfish view this can be bad for the loan company. It becomes clear that blue group people are being denied loans they could well afford, and so people in that group start moving over to other providers.
If the populations of blue groups are geographically clustered, this may mean losing large portions of business in certain areas, resulting in shutting down local offices if there's a physical presence (e.g. banks).
This is also entirely aside from legal concerns.
The best model is rarely found by simply optimising a basic measure with no context.