Summarize content from any website or a YouTube video - with a click
Automatically detects whether the input URL is a YouTube video or a regular web page, then pulls data with
youtube-transcript-apior LangChain’sUnstructuredURLLoaderto build a clean Document object pipeline.
Uses a custom
count_tokenshelper (Hugging Face tokenizer) to measure prompt size and switch between stuff, map-reduce, or refine chains on the fly, keeping every request under Groq’s 12 k TPM ceiling while squeezing maximum context into Llama-3.3-70B.
Implements
RecursiveCharacterTextSplitterwith adjustable chunk/overlap sizes, then iteratively trims excess segments to stay within a configurable token budget, this provides ability to have granular control over memory, latency, and cost for large-scale summarization workflows.
- The Python script leverages functionalities from LangChain and accesses Meta's llama-3.3-70b-versatile through Groq API
- Validates URLs and handles them based on their source (youtube, generic websites, etc)
- Utilizes chain summarization methods from LangChain to chain prompts and inputs
- Provides a Streamlit page for web based interaction
- Install the requirements in your python environment using
pip install -r requirements.txt - Run
app.pywithstreamlit run app.py - Provide a
Groq API Key(for more : Groq API) - Provide a URL and click the summarize button
-
A thoughtful Medium article by Debbie Levitt on thinking critically about perspectives and information presented in a satirical fashion.
-
One of the most popular API Documentation - Stripe API Docs
-
An insightful YouTube video lecture : Stanford CS229 I Machine Learning I Building Large Language Models (LLMs) by Yann Dubois(PhD Student at Stanford)
