Compare 2 long texts

alexcoch
Tue, 02 May 2023 10:29:31 +0000

Hello,
How can I compare for long texts using LLM?The questions are of interest to the following:

1. What is new in this article compared to the articles previously marked as read, by points,

2. What of this article has already been in previously read articles, point by point.
I guess that the solution may lie through a bundle of Langchain, Vector DB, and LLM, but it is not entirely clear how to formulate the above questions. I am glad to any ideas.
What I did before:

1. I broke the text of the first article into pieces,

2. I got embedding with the LLM API to them,

3. Saved it in Vector DB,

4. I sent requests to Vector DB and then, together with the results obtained, to the LLM API and received human-like responses.
Perhaps, it’s worth storing chunks in the view of embeddings to Vector DB and then try compare this article with previous one. Is it possible?
Best,
Alex

Read full topic