added more content

Author: programmerraja

notes/2024/Generative-AI.html

@@ -58,8 +58,8 @@ <h3 id="chunking-methods">Chunking Methods<a role="anchor" aria-hidden="true" ta
 <h3 id="evaluating-chunking-strategies"><strong>Evaluating Chunking Strategies</strong><a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#evaluating-chunking-strategies" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
 <p>Recall is a crucial metric for evaluating the effectiveness of a chunking strategy. It measures the proportion of relevant chunks retrieved in response to a query.</p>
 <p>A high recall rate indicates that the chunking strategy is effectively capturing and representing the information in a way that allows for accurate retrieval.</p>
-<p>Example:
-Imagine a document has been chunked, and a query results in three relevant chunks. The retriever returns five chunks, but only two of those are the relevant ones. In this case:</p>
+<p>Example:</p>
+<p>Imagine a document has been chunked, and a query results in three relevant chunks. The retriever returns five chunks, but only two of those are the relevant ones. In this case:</p>
 <ul>
 <li>Relevant elements = 3</li>
 <li>Retrieved elements that are also relevant = 2</li>
@@ -92,7 +92,6 @@ <h3 id="retrevial-algorithm">Retrevial Algorithm<a role="anchor" aria-hidden="tr
 <li><strong>Locality-Sensitive Hashing (LSH)</strong>: This algorithm hashes input vectors so that similar vectors map to the same hash, allowing fast lookup based on hash values rather than full comparison</li>
 <li><strong>BM25 (Best Matching 25)</strong> Unlike vector-based search, which relies on embeddings, BM25 is a term-based algorithm that ranks documents based on the presence and frequency of query terms in the documents.</li>
 </ul>
-<p>BM25,ADA-002</p>
 <h3 id="rag-evaluation">RAG Evaluation<a role="anchor" aria-hidden="true" tabindex="-1" data-no-popover="true" href="#rag-evaluation" class="internal"><svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M10 13a5 5 0 0 0 7.54.54l3-3a5 5 0 0 0-7.07-7.07l-1.72 1.71"></path><path d="M14 11a5 5 0 0 0-7.54-.54l-3 3a5 5 0 0 0 7.07 7.07l1.71-1.71"></path></svg></a></h3>
 <p>RAG Triad of metrics</p>
 <ul>