Is there a way to ONLY look at the source documents?

[langejay]

When i ask a simple question involving my source documents, many of my questions take a long time to respond and seems to answer independant of my documents. Then it will quickly generate an answer regarding my source documents. It would be good to bypass the (seemingly) first process.

[Lutios]

I have the same question. Is there a way to skip the LLM step and just return the sources?

[mechanicmuthu]

You need to make some changes in program to do that. You can change in privateGPT.py to call

docs = db.as_retriever(search_type="similarity", search_kwargs={"k":4}).get_relevant_documents(query)

and totally comment out the part where call is made to retrievalQA

qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents= not args.hide_source)

It needs some coding though.

[Lutios]

It worked, thank you! And now I can control the number of sources returned using search_kwargs={"k":5}. It is also much faster without the LLM inference step.

    ####qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents= not args.hide_source)
    # Interactive questions and answers
    while True:
        query = input("\nEnter a query: ")
        if query == "exit":
            break

        # Get the answer from the chain
        ####res = qa(query)
        ####answer, ####
        docs = db.as_retriever(search_type="similarity", search_kwargs={"k":5}).get_relevant_documents(query)

        # # Print the result
        # print("\n\n> Question:")
        # print(query)
        # print("\n> Answer:")
        # print(answer)

        # Print the relevant sources used for the answer
        for document in docs:
            print("\n> " + document.metadata["source"] + ":")
            print(document.page_content)

[chitteshwari]

@Lutios
How would we get & print the answer in this case since with this improved code, the answer portion is commented out?

[nicholasqa]

This seems to only result in the sections of the documents found that are related to the question, but not an actual answer from the LLM. It would be helpful to have an actual answer from the LLM but basing only on the ingested db. Is that possible at all?

For example, I'm ingesting dozens of pdfs but when I ask a specific question that I know to be within those pdfs, I get a generic answer from the LLM that seems to be based on it's "general" knowledge and not on the ingested documents.

[milincheta]

this is, as I understand, to point out the relevant documents and parts of them, but not to articulate an answer.

[NPap0]

In layman terms,
what you want is to vectorize the database (turn the documents/chunks into numbers/already takes place in privateGPT) then have your question transformed into numbers as well and then you'll use the same similarity function used in privateGPT to fetch the K top similar chunks. Yes it is possible, it shouldn't be too much work // start reading the code and you'll see the part where the top K chunks are returned, you'd like to stop there and return those instead of the answer of the LLM. (I just wanted to rephrase the question so maybe one that has some free time is able to work on it, cheers!)

[pseudotensor]

FYI https://github.com/h2oai/h2ogpt supports this now. Just specify "Only" when selecting subset of documents, and it'll return just the sources it would have used:

[pseudotensor]

The main readme has this: https://github.com/h2oai/h2ogpt/tree/main#cpu and links to the FAQ.md#cpu. Can you say a bit about what was confusing? I'm happy to improve anything. Thanks!

[pseudotensor]

Thanks. I reorganized the CPU installation instructions and put in main readme page. I think should be less confusing: https://github.com/h2oai/h2ogpt#cpu

[NPap0]

FYI https://github.com/h2oai/h2ogpt supports this now. Just specify "Only" when selecting subset of documents, and it'll return just the sources it would have used:

Hey, checked out your work and I'm quite amazed. Got 2 questions if you don't mind:

1. How easy would the implementation be for to go from GUI to python? I would like to use this return specific chunks process as part of a pipeline.
2. What are the differences between PrivateGPT and H2O at this moment? Why should one choose H2O over the other? Just trying to get some pros and cons here. For starters, I think PrivateGPT doesn't get updated as much, maybe the contributors don't have as much time, but I see you are pushing commits on the daily and that is great for further development

[pseudotensor]

1. I think you mean a little CLI version like privateGPT instead of GUI or client API. I think that's great point, will do it today.
2. h2oGPT has a UI, supports image captions, one can select which documents out of collection to ask question of, can save any chats, export them, import them, etc. I should add a list of current differences in our FAQ. Thanks!

[NPap0]

1. I think you mean a little CLI version like privateGPT instead of GUI or client API. I think that's great point, will do it today.

   2. h2oGPT has a UI, supports image captions, one can select which documents out of collection to ask question of, can save any chats, export them, import them, etc. I should add a list of current differences in our FAQ. Thanks!

Hey, just saw your reply. Thank you for continuing your contribution to this! I'll read over the newest commits and definitely give this a try if its available!

[dgloeppky]

I have a few more random thoughts on source documents:

A way to delete an ingested file directly from the GUI.

A way to delete multiple document ids from an API endpoint.

A way to quickly clear all embeddings from the database.

A pre-configured watch folder (source_documents) as part of the installation and no need to
run an ingest script. Any documents placed in the folder would automatically be ingested,
even changes to an already ingested file if it has been modified.