Refactor Processing Pipeline

[FranciscoLozCoding]

This pull request introduces a major refactor and enhancement to the weavloader service in the HybridSearch example, transitioning it to a more robust, scalable, and maintainable architecture. The changes include containerization updates, a switch to a Celery/Redis-based job system for image processing, modularization of inference/model code, and improved logging and monitoring capabilities.

Job System & Task Management

• Added a new job system using Celery and Redis, with configuration in job_system/celery_config.py. Periodic and retry settings are included for robust task handling.

Inference & Model Modularization

• Moved model inference functions to a new inference package, with a clear __init__.py and improved logging for error/debug messages. The old model.py is renamed and reorganized as inference/model.py.

Image Processing Improvements

• Added a new process_single_image_data function to data.py for handling individual image ingestion and metadata extraction, improving error handling and modularity.
• Updated imports in data.py to use the new inference package structure.

Monitoring & Metrics

• Introduced a metrics package with methods to start a metrics server and retrieve metrics, supporting observability for the service.

These changes lay the foundation for scalable, parallelized image processing and monitoring, and make the codebase easier to maintain and extend.