Cannot load tokenizer for llama2

saireddy
Thu, 14 Sep 2023 17:12:26 +0000

Hi,
I am trying to load tokenizer for llama2 using AutoTokenizer but I am facing this issue

“”“OSError: Can’t load tokenizer for ‘meta-llama/Llama-2-7b-hf’. If you were trying to load it from ‘Models - Hugging Face’, make sure you don’t have a local directory with the same name. Otherwise, make sure ‘meta-llama/Llama-2-7b-hf’ is the correct path to a directory containing all relevant files for a LlamaTokenizer tokenizer.
“””

any help is appreciated!

Thanks