Welcome to PDF-Assistant-RAG Discussions!

[param20h]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[param20h]

Hello everyone! 👋

@blinkerbit
@HirenGajjar
@PranavAgarkar07
@vaishnavijha12
@Jiya3177
@Yuvraj-Sarathe
@akshy-yy
@algojogacor
@SatyamPrakash09
@saurabhhhcodes
@nancy-verma780
@GLNPardhiv
@Srushti-Kamble14

Feel free to use this discussion thread for any project-related questions, doubts, suggestions, feature requests, bug reports, or general guidance. If you're facing an issue or need clarification on anything, don't hesitate to start a discussion here.

Let's collaborate, help each other, and make the project better together! 🚀

[GLNPardhiv]

@param20h This is a doubt regarding the feature #47 (Allow users to configure chunk size and chunk overlap via settings)

Basically there are two ways in which this feature can work

1. When a user uploads a file due to _ingest_document function running in background the document first gets chunked, embedded and summary is generated with default chunk_size and default chunk_overlap, then the users can view the output and input their own chunk_size and chunk_overlap through settings then the _injest_document function runs again with new chunk_size and new chunk_overlap.
2. We can remove the _injest_document function running in the background immediately after file upload and let users decide chunk_size and chunk_overlap after file upload and then run the _injest_document function to chunk, embed the document and generate summary.

Which approach do you prefer for this repo? I'll implement accordingly.

[param20h]

@param20h This is a doubt regarding the feature #47 (Allow users to configure chunk size and chunk overlap via settings)

Basically there are two ways in which this feature can work

1. When a user uploads a file due to _ingest_document function running in background the document first gets chunked, embedded and summary is generated with default chunk_size and default chunk_overlap, then the users can view the output and input their own chunk_size and chunk_overlap through settings then the _injest_document function runs again with new chunk_size and new chunk_overlap.
2. We can remove the _injest_document function running in the background immediately after file upload and let users decide chunk_size and chunk_overlap after file upload and then run the _injest_document function to chunk, embed the document and generate summary.

Which approach do you prefer for this repo? I'll implement accordingly.

Go with approach 1—run ingestion on upload with sane defaults, and add an optional re-ingest after user customizes chunking settings. This matches most users’ expectations and established SaaS usability patterns

[GLNPardhiv]

Sure, I will implement approach 1.Thank you for clarifying.

[saurabhhhcodes]

Sure @param20h

[param20h]

Star the repo donot forgot

[SatyamPrakash09]

@param20h I have raised the pr #204 for the issue #117 . Can you review it ?

[param20h]

Guys check out your prs there are merge conflicts

[GLNPardhiv]

@param20h This is a doubt regarding issue #113 (Implement Semantic Re-ranking for search results)

This is a doubt regarding the implementation.

1. At present, reranking is done by retrieving top_k chunks and given scores by the reranker model.
   Then these top_k chunks are sorted in decreasing order and only some of top (5 or 8 chunks) among the top_k chunks are feed to LLM for required answer generation.
   This is simple and calls LLM only once per query but (sometimes when context of questions are similar) some queries may receive similar retrieved sources due to reranking similar results higher.

2. But, LLM has a feature if we set task="conversational" LLM sees the feeded chunks and decides whether the required answer is present in the chunks feeded to it, if yes then it generates answer, if no it restarts the retrieval process by reframing the question to match the similarity of embedded vectors and again chunks are retrieved, reranked and then feed to LLM, this process repeates until LLM finds relevant answer or most close answer (we fix certain number of tries).

• There is a catch in second method, this method increases LLM calls per minute and uses more tokens very quickly, free models cant replicate this conversation as their rate limits gets reached very quickly.
• Sometimes LLM might not get satisfied with the chunks it receives and keeps restarting the chain which leads to a long loop wasting time for users and increasing LLM calls and cost (This can be solved by restricting LLM to some number of maximum chain restarts ex. 5).

(Current model in config.py file is able to perform the second task)

which approach would you prefer for this repo? I'll implement accordingly.

[GLNPardhiv]

@param20h, I have implemented and raised PR #396 according to method 1 as it was the method mentioned in the issue and it is good for the repo with good test results. But If you need me to implement method 2, please feel free to comment I will definitely complete your requirement.

[param20h]

i have started merging prs it will be done today

[param20h]

Hey i am on day off so, pending issue assign and pull request be review after 7 June 5PM