Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 23131

Accelerate and bitsandbytes is needed to install but I did

$
0
0

I'm trying to load quantization like

from transformers import LlamaForCausalLMfrom transformers import BitsAndBytesConfigmodel = '/model/'model = LlamaForCausalLM.from_pretrained(model, quantization_config=BitsAndBytesConfig(load_in_8bit=True))

but I get the error

ImportError: Using `load_in_8bit=True` requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes `pip install -i https://test.pypi.org/simple/ bitsandbytes` or pip install bitsandbytes` 

But I've installed both, and I get the same error. I shut down and restarted the jupyter kernel I was using this on.


Viewing all articles
Browse latest Browse all 23131

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>