feat: Integrate Toonify library for token-efficient responses #67
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🎨 TOON Format Integration
This PR integrates the Toonify library to enable token-efficient responses using TOON (Token-Oriented Object Notation) format, achieving 30-60% reduction in token usage compared to JSON.
📋 Changes
Core Implementation
toonify>=1.0.0dependency topyproject.tomltoon_converter.pyutility module for TOON conversionreturn_toonparameter to all scraping methods (sync & async)Supported Methods
All the following methods now support
return_toon=True:smartscraper()/get_smartscraper()searchscraper()/get_searchscraper()crawl()/get_crawl()agenticscraper()/get_agenticscraper()markdownify()/get_markdownify()scrape()/get_scrape()Documentation & Examples
TOON_INTEGRATION.mddocumentationexamples/toon_example.pyexamples/toon_async_example.py💡 Benefits
📊 Example Comparison
JSON Format (verbose):
{ "request_id": "f424487d-6e2b-4361-824f-9c54f8fe0d8e", "status": "completed", "website_url": "https://example.com", "result": { "page_title": "Example Domain", "main_heading": "Example Domain" } }TOON Format (compact):
🚀 Usage
✅ Testing
📁 Files Changed
pyproject.toml,client.py,async_client.pytoon_converter.py,TOON_INTEGRATION.md, example filesTotal: 7 files changed, 764 insertions(+), 58 deletions(-)
🔗 Related