add method: Uniform Risk Minimization (URM) #15
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I am first author of "Uniformly Distributed Feature Representations for Fair and Robust Learning" (TMLR 2024) paper. Link: https://openreview.net/forum?id=PgLbS5yp8n
In this pull request I am adding code for the proposed Uniform Risk Minimization (URM) method for subpopulation robustness. The key idea is to encourage the distribution of feature representations learned by the encoder/featurizer to be uniformly distributed. The paper provides theoretical and empirical support for the proposed method in sub-population robustness and also domain generalization.
Changes include:
URM
class inalgorithms.py
hparams_registry.py
networks.py
andlib/wide_resnet.py
to add an activation function to the output of the featurizer/encoder networks (before the linear classifier). This does not affect other algorithms in SubpopBench as the default activation is nn.Identity(): https://github.com/kiranchari/SubpopBench/blob/198ea9548948b5af0a084798878c880e89d248d6/subpopbench/models/networks.py#L31README.md
Thank you.