Bat ai/log analysis (#275)

* log_analysis using self corrective RAG

* updated to ChatNVIDIA

* removed chatopenai

* updated with the minor changes

* minor changes

* minor changes readme

* minor fixes

* minor changes in readme

* updated readme

Author: Santhoshcharugulla001

community/log_analysis_multi_agent_rag/.env

@@ -0,0 +1 @@
+API_KEY = ADD_YOUR_API_KEY

community/log_analysis_multi_agent_rag/README.md

@@ -8,14 +8,26 @@ We are calling this tool as BAT.AI (Bug Automation Tool)
 # Target Audience
 Devlopers : This tool is designed for developers who need to quickly analyze log files and gain actionable insights using large language model (LLM). The system automatically refines prompts to ensure optimal results, offering developers an intuitive way to interact with log data and streamline their debugging process.  
 
+# How to Use
+
+This repository provides a sample code to demonstrate how you can use the log analysis tool for your logs. Follow the instructions below to set up and integrate the tool into your workflow.
+
+### Set up your API Key:
+
+- **Generate your API key** by following the steps in the link below:
+  [Click here to view the steps for generating an API Key](https://docs.nvidia.com/nim/large-language-models/latest/getting-started.html#generate-an-api-key)
+
+- **Store your API key** : You can securely store your API key by creating a `.env` file in the root directory of your project
+- **example.py** : The sample script showcases how to integrate log analysis into your workflow. It demonstrates how to pass your log data through the system, generate insights, and manage the output.
+
 # Components
 - bat_ai.py: Defines the main workflow graph using LangGraph.
 - graphnodes.py: Contains the node implementations for the workflow graph.
 - multiagent.py: Implements the HybridRetriever class for document retrieval.
 - graphedges.py: Contains the implementation of the edges for decision making 
 - binaryscroes.py: Contains the formatted output information
-- utils.py : It helps to implement the queries, retrieve relevant documents, grade their relevance, and generate responses      using a multi-agent RAG system.
-- example.py: The script that analyzes a specified log file for errors based on a user-provided question, leveraging the workflow module to process and generate relevant insights.
+- utils.py : It helps to implement the queries, retrieve relevant documents, grade their relevance, and generate responses using a multi-agent RAG system.
+
 
 ![SW Architecture](<BAT.AI SW Architecture Diagram.drawio.png>)
 
@@ -30,8 +42,9 @@ Devlopers : This tool is designed for developers who need to quickly analyze log
     Set up the NVIDIA API key in your environment.
     Prepare your document corpus and update the file path in the code.
 
+
 # Code
-`python main.py path/to/your/logfile.txt --question "What are the critical errors in the log file?"`
+`python example.py path/to/your/logfile.txt --question "What are the critical errors in the log file?"`
 
 # Software Components
 NVIDIA NIM Microservices

community/log_analysis_multi_agent_rag/graphnodes.py

@@ -1,10 +1,12 @@
 from langchain_nvidia_ai_endpoints import NVIDIARerank
-api_key = "<add your api key>"
+import os
 from multiagent import HybridRetriever
 import io
 from contextlib import redirect_stdout, redirect_stderr
 from utils import automation
-
+from dotenv import load_dotenv
+load_dotenv()
+api_key = os.getenv('API_KEY')
 
 class Nodes:
     @staticmethod

community/log_analysis_multi_agent_rag/multiagent.py

@@ -5,7 +5,9 @@
 from langchain.retrievers import EnsembleRetriever
 from langchain_community.retrievers import BM25Retriever
 from langchain_community.vectorstores.faiss import FAISS
-import argparse
+from dotenv import load_dotenv
+load_dotenv()
+api_key = os.getenv('API_KEY')
 class HybridRetriever:
     def __init__(self, file_path, api_key):
         self.file_path = file_path

community/log_analysis_multi_agent_rag/utils.py

@@ -3,13 +3,14 @@
 from langchain_core.output_parsers import StrOutputParser
 from binary_score_models import GradeAnswer,GradeDocuments,GradeHallucinations
 import os
-
+from dotenv import load_dotenv
+load_dotenv()
 import json
 
 class Nodeoutputs:
-    def __init__(self, api_key, base_url, model, prompts_file):
+    def __init__(self, api_key, model, prompts_file):
         os.environ["NVIDIA_API_KEY"] = api_key
-        self.llm = ChatNVIDIA(base_url=base_url, api_key=api_key, model=model)
+        self.llm = ChatNVIDIA( api_key=api_key, model=model)
         self.prompts = self.load_prompts(prompts_file)
         self.setup_prompts()
 
@@ -62,8 +63,10 @@ def format_docs(self, docs):
         return "\n\n".join(doc.page_content for doc in docs)
 
 # Usage
-api_key = "<addd your api key>"
-base_url = "<add your endpoint>"
+
+
+# Access the API key from environment variables
+api_key = os.getenv('API_KEY')
 model = "meta/llama-3.1-70b-instruct"
 prompts_file = "prompt.json"
-automation = Nodeoutputs(api_key, base_url, model, prompts_file)
+automation = Nodeoutputs(api_key, model, prompts_file)