feat(LM): attributions for SequenceClassification #116

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

jamesjin0516 wants to merge 1 commit into jalammar:main from jamesjin0516:mlm-attributions

jamesjin0516 commented Jul 13, 2024

#64 @jalammar
Single-class sequence classification with BERT and RoBERTa now allows primary attributions. tox command run.

Changes

LM class: new classify method. Mimics generate in tokenization, forward pass, and hidden states collection.
attributions.py: model_forward's softmax step also allows 2D model outputs
https://github.com/jamesjin0516/ecco/blob/558cb1e9d35c2551e1b4ad3add497124b0a48919/src/ecco/attribution.py#L68

TODOs

OutputSeq class: all_token_ids produced by classify currently contains index of predicted label, incorrectly decoded as a token.
Potentially include support for multi-class classification


          feat(LM): attributions for SequenceClassification

558cb1e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet