Researchers have created a new AI debiasing technique aimed at enhancing the fairness of machine-learning models. This method specifically boosts the performance for underrepresented subgroups in training datasets, ensuring that overall accuracy is preserved.
Machine-learning models often encounter issues when predicting outcomes for individuals not adequately represented in their training datasets.
For example, if a model is trained primarily on data from male patients to determine the most effective treatment for a chronic condition, it may struggle to make accurate predictions for female patients in a clinical setting.
To rectify this, engineers sometimes attempt to balance the training dataset by eliminating data points until all subgroups are equally represented. However, this balancing act often requires removing extensive portions of data, which can negatively impact the model’s overall effectiveness.
Researchers at MIT have developed a novel technique that pinpoints and eliminates specific data points in the training dataset that most contribute to a model’s poor performance on minority subgroups. By removing fewer data points than traditional methods, this technique manages to keep the model’s overall accuracy intact while enhancing its performance for underrepresented groups.
This technique also has the potential to discover hidden biases in a training dataset that lacks labeling, as unlabeled data is considerably more common than labeled data in various applications.
Additionally, this method may be combined with other strategies to augment the fairness of machine-learning models, especially those used in critical scenarios. For instance, it could eventually help prevent misdiagnoses among underrepresented patients caused by biased AI models.
“Many existing algorithms tackling this issue assume that every data point holds equal weight. Our research shows that this assumption isn’t accurate. Certain data points in our dataset are responsible for the bias, and we can pinpoint, remove them, and enhance performance,” states Kimia Hamidieh, an electrical engineering and computer science graduate student at MIT and co-lead author of this research paper.
She co-authored the paper with fellow lead authors Saachi Jain, who is pursuing a PhD, and Kristian Georgiev, another EECS graduate student; Andrew Ilyas, a former student at MIT now a Stein Fellow at Stanford University; as well as senior authors Marzyeh Ghassemi, an associate professor in EECS and a member of the Institute of Medical Engineering Sciences and the Laboratory for Information and Decision Systems, and Aleksander Madry, the Cadence Design Systems Professor at MIT. Their findings will be showcased at the Conference on Neural Information Processing Systems.
Eliminating Ineffective Examples
Machine-learning models often rely on vast datasets collected from various online sources. These datasets can be too large for effective manual curation, resulting in the inclusion of ineffective examples that hinder model performance.
It is known among scientists that some data points have a more significant impact on model performance than others when it comes to particular tasks.
The MIT team combined these concepts to create a method that identifies and removes these detrimental samples. They aim to tackle an issue referred to as worst-group error, where models perform poorly on minority subgroups within a training dataset.
This new technique is based on previous work where the team introduced a method named TRAK that identifies the most crucial training examples related to a specific model output.
For the current method, they focus on incorrect predictions made by the model regarding minority groups and leverage TRAK to discern which training examples mainly influenced those incorrect predictions.
“By systematically gathering this information from poor test predictions, we can identify the specific training components that degrade the overall performance on minority groups,” explains Ilyas.
Subsequently, they remove those specific examples and retrain the model with the remaining data.
As more data typically leads to improved overall performance, selectively removing samples responsible for poor minority group outcomes helps retain the model’s general accuracy while also enhancing its performance for these subgroups.
An Easier Approach
Across three machine-learning datasets, this method surpassed various other techniques. In one example, the team increased the accuracy for the worst-performing subgroup while disposing of approximately 20,000 fewer training samples than conventional balancing methods. Their technique also yielded greater accuracy compared to approaches requiring alterations to the model’s internal functioning.
Because the MIT method modifies the dataset rather than the model itself, it is more user-friendly for practitioners and can be applied across various model types.
It is also applicable even when biases are not clearly defined, especially when subgroups in a training dataset are unlabeled. The team can pinpoint data points significantly contributing to a specific feature the model is learning, offering insights into the variables behind its predictions.
“This is a tool for anyone working on a machine-learning project. They can analyze these data points and assess whether they align with the capabilities they wish to instill in the model,” asserts Hamidieh.
Utilizing this technique to reveal unknown subgroup biases will require insight into potential groups to examine; thus, the researchers aim to validate and further investigate this through future human studies.
They also aspire to enhance the performance and dependability of their technique and ensure that it remains accessible and user-friendly for practitioners who may implement it in real-world settings.
“Having tools that enable critical evaluation of data and help identify data points that could lead to bias or other unwanted effects is a significant step towards building fairer, more reliable models,” Ilyas concludes.
This research is funded, in part, by the National Science Foundation and the U.S. Defense Advanced Research Projects Agency.
