Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 14011

How to create a Chromadb after VectorstoreIndexCreator for your CSV?

$
0
0

I have successfully created a chatbot that can answer question by referencing to the csv. My code is as below,

loader = CSVLoader(file_path='data.csv') # load the csvindex_creator = VectorstoreIndexCreator() # initiationdocsearch = index_creator.from_loaders([loader]) # embedding

My chain is as follow,

chain = RetrievalQA.from_chain_type(llm=OpenAI(model_name="gpt-3.5-turbo", temperature=0), chain_type="stuff", retriever=docsearch.vectorstore.as_retriever(), input_key="question")

Everything is perfect so far.

But I am trying to create an app which will solve problems by referencing to this csv, therefore I would like to store the vectorized data into a chromadb which can be retrieved without embedding again.

I understand that (please tell me if I am wrong) docsearch.vectorstore is already a chromadb. When I print(docsearch.vectorstore), it returns,

<langchain_community.vectorstores.chroma.Chroma object at 0x13e079130>

But how do it store it as a file? Like that you would do after embedding a txt or pdf file, you persist it in a folder.

embedding = OpenAIEmbeddings(openai_api_key=openai_api_key)# embedding = OpenAIEmbeddings(openai_api_key=openai_api_key, model_name='text-embedding-3-small')persist_directory = "embedding/chroma"# Create a Chroma vector database for the current states after embeddingvectordb = Chroma(    persist_directory = persist_directory,    embedding_function = embedding)vectordb.persist() database = Chroma.from_documents(doc, embedding=embedding, persist_directory = persist_directory)

Then you will be able find the database file in the persist_directory.

So, my question is, how do I achieve a similar process with my csv data? I have googled, e.g. Is there any way to load an index created through VectorstoreIndexCreator in langchain? How does it work?But it seems not the answer. Because the answer calls the "docs" and "embeddings", which I can't seem to find those in the op,.

Any help is much appreciated.


Viewing all articles
Browse latest Browse all 14011

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>