[`core`] Add safetensors integration #553

younesbelkada · 2023-06-06T14:00:44Z

What does this PR do?

This PR integrates safetensors into peft

The logic now is as follows:

When saving locally a model, by default save the pickle file.
If a user wants to go for a safer serialization (recommended), users should pass safe_serialization=True to save_pretrained method
Same logic applies for push_to_hub method, by default we push on the Hub pickle file, except if a user calls safe_serialization=True to that method.
Inside from_pretrained, first look if the safetensors weights are present locally, then check for the pickle file. If none of them are present locally, try to fetch the safetensors weights from remote Hub, if not try to fetch the pickle file for remote.

What do you think?

An example of safetensors adapter weights on the Hub: https://huggingface.co/ybelkada/test-st-lora/tree/main

Fixes #546

cc @Narsil @pacman100

HuggingFaceDocBuilderDev · 2023-06-06T14:05:13Z

The documentation is not available anymore as the PR was closed or merged.

src/peft/peft_model.py

Narsil

Shouldnt' you make the safetensors loading optional ? Just because there are already some PEFT pickled files out there already ? (Maybe it's handled elsewhere)

younesbelkada · 2023-06-06T15:52:45Z

src/peft/peft_model.py

+                filename = hf_hub_download(
+                    model_id, SAFETENSORS_WEIGHTS_NAME, subfolder=kwargs.get("subfolder", None), **kwargs
                )
+            except:  # noqa
+                try:
+                    filename = hf_hub_download(
+                        model_id, WEIGHTS_NAME, subfolder=kwargs.get("subfolder", None), **kwargs
+                    )
+                except:  # noqa
+                    raise ValueError(
+                        f"Can't find weights for {model_id} in {model_id} or in the Hugging Face Hub. "
+                        f"Please check that the file {WEIGHTS_NAME} or {SAFETENSORS_WEIGHTS_NAME} is present at {model_id}."
+                    )


I don't like the double try / except will find a better workaround

using EntryNotFoundError for now

Will use cached_file method from transformers: https://github.com/huggingface/transformers/blob/02fe3af275f0ad935441b813ef2e58b94d09788a/src/transformers/utils/hub.py#L300

Narsil · 2023-06-07T06:48:48Z

src/peft/peft_model.py

    @classmethod
-    def from_pretrained(cls, model, model_id, adapter_name="default", is_trainable=False, **kwargs):
+    def from_pretrained(
+        cls, model, model_id, adapter_name="default", is_trainable=False, load_safetensors=False, **kwargs


Suggested change

cls, model, model_id, adapter_name="default", is_trainable=False, load_safetensors=False, **kwargs

cls, model, model_id, adapter_name="default", is_trainable=False, use_safetensors=False, **kwargs

For consistency with transformers and diffusers. And it should probably be None instead of False if we want to make it the default.

Narsil · 2023-06-07T06:52:56Z

src/peft/peft_model.py


-        if os.path.exists(os.path.join(path, WEIGHTS_NAME)):
+        # load_safetensors is only used for remote weights
+        remote_weights_name = SAFETENSORS_WEIGHTS_NAME if load_safetensors else WEIGHTS_NAME


Since the default is False this will always use the pickled file iiuc unless user specifically asks for it.

pacman100

Thank you @younesbelkada for adding support to safetensors and related nice tests 🔐🤗, left a couple comments

src/peft/peft_model.py

tests/testing_common.py

younesbelkada · 2023-06-07T12:29:55Z

This is PR is ready for a final review! I have also updated the PR description a bit
Would love to get a final round of review
@pacman100 @Narsil

younesbelkada · 2023-06-07T12:48:49Z

I think this is still to be done?

I don't know if we should add a ternary logic here as safetensors becomes a core dependency on this PR. With these changes from_pretrained first looks for safetensors weights either locally or on the Hub, then looks for pickle weights. So from my understanding there is no need to apply a ternary logic

pacman100

Thank you @younesbelkada for iterating, LGTM, super cool feature addition!

pacman100 · 2023-06-07T12:50:45Z

tests/test_common_gpu.py

            isinstance(whisper_8bit.base_model.model.model.decoder.layers[0].self_attn.v_proj, Linear8bitLt)
        )

+    @require_bitsandbytes


younesbelkada · 2023-06-07T12:51:49Z

src/peft/peft_model.py

    def active_peft_config(self):
        return self.peft_config[self.active_adapter]

+    def push_to_hub(


As huggingface/transformers#24074 will be merged maybe we should remove this method? @pacman100

EDT: let's leave it as it is and remove it on the next transformers release

let's remove it as it is taken care in transformers and the next release is in couple of days

younesbelkada · 2023-06-07T12:52:36Z

Thanks a lot @pacman100 ! I just left one open question: #553 (comment)

pacman100

Thank you!

pacman100 · 2023-06-07T12:55:10Z

src/peft/peft_model.py

    def active_peft_config(self):
        return self.peft_config[self.active_adapter]

+    def push_to_hub(


let's remove it as it is taken care in transformers and the next release is in couple of days

* add v1 * clean up * more improvements * add device * final adjustements * use `EntryNotFoundError` * better checks * add tests and final fixes * make style && make quality * remove `push_to_hub` because of the release

younesbelkada added 2 commits June 6, 2023 13:24

add v1

826b441

clean up

abb66f8

Narsil reviewed Jun 6, 2023

View reviewed changes

src/peft/peft_model.py Outdated Show resolved Hide resolved

Narsil reviewed Jun 6, 2023

View reviewed changes

more improvements

c072b7c

younesbelkada mentioned this pull request Jun 6, 2023

Support safetensors for PEFT adapters or raise an exception when safe_serialization=True is specified #546

Closed

younesbelkada added 2 commits June 6, 2023 15:39

add device

f999cc9

final adjustements

95b4f6c

younesbelkada commented Jun 6, 2023

View reviewed changes

younesbelkada requested a review from pacman100 June 6, 2023 15:52

use EntryNotFoundError

1622e6b

Narsil reviewed Jun 7, 2023

View reviewed changes

pacman100 reviewed Jun 7, 2023

View reviewed changes

src/peft/peft_model.py Show resolved Hide resolved

pacman100 reviewed Jun 7, 2023

View reviewed changes

tests/testing_common.py Show resolved Hide resolved

younesbelkada added 3 commits June 7, 2023 12:06

better checks

2817988

add tests and final fixes

9796cad

make style && make quality

a9ffa4c

younesbelkada mentioned this pull request Jun 7, 2023

[Hub] Add safe_serialization in push_to_hub huggingface/transformers#24074

Merged

pacman100 approved these changes Jun 7, 2023

View reviewed changes

younesbelkada commented Jun 7, 2023

View reviewed changes

remove push_to_hub because of the release

d6c8458

pacman100 approved these changes Jun 9, 2023

View reviewed changes

younesbelkada merged commit 189a6b8 into huggingface:main Jun 9, 2023

younesbelkada deleted the add-st branch June 9, 2023 10:33

NanoCode012 mentioned this pull request Jun 13, 2023

optionally save as safetensors. axolotl-ai-cloud/axolotl#31

Closed

	cls, model, model_id, adapter_name="default", is_trainable=False, load_safetensors=False, **kwargs
	cls, model, model_id, adapter_name="default", is_trainable=False, use_safetensors=False, **kwargs

[core] Add safetensors integration #553

[core] Add safetensors integration #553

Uh oh!

Conversation

younesbelkada commented Jun 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Uh oh!

HuggingFaceDocBuilderDev commented Jun 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Narsil left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pacman100 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

younesbelkada commented Jun 7, 2023

Uh oh!

younesbelkada commented Jun 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pacman100 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

younesbelkada Jun 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

younesbelkada commented Jun 7, 2023

Uh oh!

pacman100 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[`core`] Add safetensors integration #553

[`core`] Add safetensors integration #553

younesbelkada commented Jun 6, 2023 •

edited

Loading

HuggingFaceDocBuilderDev commented Jun 6, 2023 •

edited

Loading

pacman100 left a comment •

edited

Loading

younesbelkada commented Jun 7, 2023 •

edited

Loading

younesbelkada Jun 7, 2023 •

edited

Loading