Failed to create SentenceTransformer embedding function with model 'all-MiniLM-L6-v2'

[zhangjy859]

Hi,

I am trying to create a RAG with memory, but I seem to have encountered an unclear issue.

rag_memory = ChromaDBVectorMemory(
        config=PersistentChromaDBVectorMemoryConfig(
            collection_name="autogen_docs",
            persistence_path=os.path.join(str(Path.home()), ".chromadb_autogen"),
            k=3,  # Return top 3 results
            score_threshold=0.4,  # Minimum similarity score
            embedding_function_config=SentenceTransformerEmbeddingFunctionConfig(
                model_name="all-MiniLM-L6-v2"  # Use default model for testing
            ),
        )
    )

## update memeory -> chroma_user_memory 

chroma_user_memory = ...

assistant_agent = AssistantAgent(
        name="assistant_agent",
        model_client=custom_model_client,
        system_message = "You are a helpful AI assistant. .",
        memory=[chroma_user_memory],
    )

Log output is:

DEBUG:chromadb.config:Starting component System
DEBUG:chromadb.config:Starting component Posthog
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): us.i.posthog.com:443
DEBUG:urllib3.connectionpool:https://us.i.posthog.com:443 "POST /batch/ HTTP/1.1" 200 15
INFO:sentence_transformers.SentenceTransformer:Load pretrained SentenceTransformer: all-MiniLM-L6-v2
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): huggingface.co:443
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /sentence-transformers/all-MiniLM-L6-v2/resolve/main/modules.json HTTP/1.1" 307 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /api/resolve-cache/models/sentence-transformers/all-MiniLM-L6-v2/[REDACTED]/modules.json HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /sentence-transformers/all-MiniLM-L6-v2/resolve/main/config_sentence_transformers.json HTTP/1.1" 307 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /api/resolve-cache/models/sentence-transformers/all-MiniLM-L6-v2/[REDACTED]/config_sentence_transformers.json HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /sentence-transformers/all-MiniLM-L6-v2/resolve/main/config_sentence_transformers.json HTTP/1.1" 307 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /api/resolve-cache/models/sentence-transformers/all-MiniLM-L6-v2/[REDACTED]/config_sentence_transformers.json HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /sentence-transformers/all-MiniLM-L6-v2/resolve/main/README.md HTTP/1.1" 307 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /api/resolve-cache/models/sentence-transformers/all-MiniLM-L6-v2/[REDACTED]/README.md HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /sentence-transformers/all-MiniLM-L6-v2/resolve/main/modules.json HTTP/1.1" 307 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /api/resolve-cache/models/sentence-transformers/all-MiniLM-L6-v2/[REDACTED]/modules.json HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /sentence-transformers/all-MiniLM-L6-v2/resolve/main/sentence_bert_config.json HTTP/1.1" 307 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /api/resolve-cache/models/sentence-transformers/all-MiniLM-L6-v2/[REDACTED]/sentence_bert_config.json HTTP/1.1" 200 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /sentence-transformers/all-MiniLM-L6-v2/resolve/main/adapter_config.json HTTP/1.1" 404 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /sentence-transformers/all-MiniLM-L6-v2/resolve/main/config.json HTTP/1.1" 307 0
DEBUG:urllib3.connectionpool:https://huggingface.co:443 "HEAD /api/resolve-cache/models/sentence-transformers/all-MiniLM-L6-v2/[REDACTED]/config.json HTTP/1.1" 200 0
ERROR:autogen_ext.memory.chromadb._chromadb:Failed to get/create collection: Failed to create SentenceTransformer embedding function with model 'all-MiniLM-L6-v2'. Ensure sentence-transformers is installed and the model is available. Error: Error while deserializing header: MetadataIncompleteBuffer
Traceback (most recent call last):
  File "[REDACTED]/site-packages/autogen_ext/memory/chromadb/_chromadb.py", line 213, in _create_embedding_function
    return embedding_functions.SentenceTransformerEmbeddingFunction(model_name=config.model_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/chromadb/utils/embedding_functions/sentence_transformer_embedding_function.py", line 44, in __init__
    self.models[model_name] = SentenceTransformer(
                              ^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/sentence_transformers/SentenceTransformer.py", line 327, in __init__
    modules, self.module_kwargs = self._load_sbert_model(
                                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/sentence_transformers/SentenceTransformer.py", line 2254, in _load_sbert_model
    module = module_class.load(
             ^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/sentence_transformers/models/Transformer.py", line 541, in load
    return cls(model_name_or_path=model_name_or_path, **init_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/sentence_transformers/models/Transformer.py", line 88, in __init__
    self._load_model(model_name_or_path, config, cache_dir, backend, is_peft_model, **model_args)
  File "[REDACTED]/site-packages/sentence_transformers/models/Transformer.py", line 186, in _load_model
    self.auto_model = AutoModel.from_pretrained(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/transformers/models/auto/auto_factory.py", line 600, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/transformers/modeling_utils.py", line 311, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/transformers/modeling_utils.py", line 4710, in from_pretrained
    with safe_open(checkpoint_files[0], framework="pt") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "[REDACTED]/article_rag.py", line 252, in <module>
    asyncio.run(Console(stream))
  File "[REDACTED]/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "[REDACTED]/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/autogen_agentchat/ui/_console.py", line 117, in Console
    async for message in stream:
  File "[REDACTED]/site-packages/autogen_agentchat/agents/_base_chat_agent.py", line 202, in run_stream
    async for message in self.on_messages_stream(input_messages, cancellation_token):
  File "[REDACTED]/site-packages/autogen_agentchat/agents/_assistant_agent.py", line 940, in on_messages_stream
    for event_msg in await self._update_model_context_with_memory(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/autogen_agentchat/agents/_assistant_agent.py", line 1044, in _update_model_context_with_memory
    update_context_result = await mem.update_context(model_context)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/autogen_ext/memory/chromadb/_chromadb.py", line 323, in update_context
    query_results = await self.query(query_text)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/autogen_ext/memory/chromadb/_chromadb.py", line 361, in query
    self._ensure_initialized()
  File "[REDACTED]/site-packages/autogen_ext/memory/chromadb/_chromadb.py", line 272, in _ensure_initialized
    embedding_function = self._create_embedding_function()
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED]/site-packages/autogen_ext/memory/chromadb/_chromadb.py", line 215, in _create_embedding_function
    raise ImportError(
ImportError: Failed to create SentenceTransformer embedding function with model 'all-MiniLM-L6-v2'. Ensure sentence-transformers is installed and the model is available. Error: Error while deserializing header: MetadataIncompleteBuffer
ERROR conda.cli.main_run:execute(49): `conda run python [REDACTED]/article_rag.py` failed. (See above for error)

[junmiron]

1- Do you have sentencetrasnformers installed?
2- Have you tried using sentencetransformers directly? I use so I can use my GPU, but it works both ways for me
3- I use embedding_function not embedding_function_config
4- you can uncomment embedding_function to use the autogen chroma extension version and change model -> embedding_function

model = SentenceTransformer('all-MiniLM-L6-v2')
model = model.to('cuda') # This moves the model to GPU

embedding_function = (

embedding_functions.SentenceTransformerEmbeddingFunction(

model_name="paraphrase-multilingual-mpnet-base-v2"

)

)

Using a specific Sentence Transformer model

memory = ChromaDBVectorMemory(
config=PersistentChromaDBVectorMemoryConfig(
collection_name="experiences",
persistence_path="/.experiences",
k=3,
score_threshold=0.4,
embedding_function=model,
)
)

[zhangjy859]

Hi, @junmiron
Thanks for your suggestion.

1. Yes
2. I don't quite understand this point.
3. I am not try that
4. I tried, but not work

In fact, after some time, I found a way to bypass this error. Simply deleting the .chroma.db file works. Although it seems unrelated to the error, it work well.