Much better Pro docs #4263
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Contributor
There was a problem hiding this comment. This could use a simple diagram explaining each deployment scenario that looks like the ones here: https://zenml-io.gitbook.io/alexej/zenml-pro/core-concepts/hierarchy |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,187 @@ | ||
| --- | ||
| description: Compare ZenML Pro deployment scenarios to find the right fit for your organization. | ||
| icon: code-merge | ||
| layout: | ||
| title: | ||
| visible: true | ||
| description: | ||
| visible: true | ||
| tableOfContents: | ||
| visible: true | ||
| outline: | ||
| visible: true | ||
| pagination: | ||
| visible: true | ||
| --- | ||
|
|
||
| # Deployment Scenarios | ||
AlexejPenner marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ZenML Pro offers three flexible deployment options to match your organization's security, compliance, and operational needs. This page helps you understand the differences and choose the right scenario for your use case. | ||
|
|
||
| ## Quick Comparison | ||
AlexejPenner marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| | Deployment Aspect | Purpose | SaaS | Hybrid SaaS | Self-hosted | | ||
| |-------------------|---------|------|-------------|-------------| | ||
| | **ZenML Server** | Stores pipeline metadata and serves the API that your SDK and UI connect to | ZenML infrastructure | Your infrastructure | Your infrastructure | | ||
| | **Pipeline/ Artifact Metadata** | Records of your pipeline runs, step executions, and artifact locations | ZenML infrastructure | Your infrastructure | Your infrastructure | | ||
| | **ZenML Control Plane** | Manages authentication, RBAC, and organization-level settings across workspaces | ZenML infrastructure | ZenML infrastructure | Your infrastructure | | ||
| | **ZenML Pro UI** | Web UI for visualizing pipelines, artifacts, and managing your ML workflows | ZenML infrastructure | ZenML infrastructure | Your infrastructure | | ||
| | **Compute & Data** | Your ML training infrastructure, models, datasets, and artifacts | Your infrastructure | Your infrastructure | Your infrastructure | | ||
| | **Setup Time** | Time to get your first pipeline running | ⚡ ~1 hour | ~4 hours | ~8 hours | | ||
| | **Maintenance** | Ongoing operational responsibility | Fully managed | Partially managed (workspace maintenance required) | Customer managed | | ||
| | **Best For** | Recommended use case | Teams wanting minimal infrastructure overhead and fastest time-to-value | Organizations with security/compliance requirements but wanting simplified user management | Organizations requiring complete data isolation and on-premises control | | ||
|
|
||
| {% hint style="info" %} | ||
| In all of these cases the client sdk that you pip install into your development environment is the same one found here: https://pypi.org/project/zenml/ | ||
| {% endhint %} | ||
|
|
||
| ## Which Scenario is Right for You? | ||
|
|
||
| ### SaaS Deployment | ||
|
|
||
| Choose **SaaS** if you want to get started immediately with zero infrastructure overhead. | ||
|
|
||
| **What runs where:** | ||
| - ZenML Server: ZenML infrastructure | ||
| - Metadata and RBAC: ZenML infrastructure | ||
| - Compute and Data: Your infrastructure | ||
|
|
||
| **Key Benefits:** | ||
| - ⚡ Fastest setup (minutes) | ||
| - ✅ Fully managed by ZenML | ||
| - 🚀 Immediate production readiness | ||
| - 💰 Minimal operational overhead | ||
|
|
||
| **Ideal for:** Startups, teams prioritizing time-to-value and operational simplicity, organizations comfortable leveraging managed cloud services. | ||
|
|
||
| [Learn more about SaaS deployment →](saas-deployment.md) | ||
|
|
||
| ### Hybrid SaaS Deployment | ||
|
|
||
| Choose **Hybrid** if you need to keep sensitive metadata in your infrastructure while benefiting from centralized user management. | ||
AlexejPenner marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| **What runs where:** | ||
| - ZenML Control Plane: ZenML infrastructure | ||
| - ZenML Pro UI: ZenML infrastructure | ||
| - ZenML Pro Server: Your infrastructure | ||
| - Run metadata: Your infrastructure | ||
| - Compute and Data: Your infrastructure | ||
|
|
||
| **Key Benefits:** | ||
| - 🔐 Metadata stays in your infrastructure | ||
| - 👥 Centralized user management | ||
| - ⚖️ Balance of control and convenience | ||
| - 🏢 Control plane and UI fully maintained and patched by ZenML | ||
| - ✅ Day 1 production ready | ||
|
|
||
| **Ideal for:** Organizations with security policies requiring metadata sovereignty, teams wanting simplified identity management without full infrastructure control. | ||
|
|
||
| [Learn more about Hybrid deployment →](hybrid-deployment.md) | ||
|
|
||
| ### Self-hosted Deployment | ||
|
|
||
| Choose **Self-hosted** if you need complete control with no external dependencies. | ||
|
|
||
| **What runs where:** | ||
| - All components: Your infrastructure (completely isolated) | ||
|
|
||
| **Key Benefits:** | ||
| - 🔒 Complete data sovereignty | ||
| - 🚫 No external network dependencies | ||
| - 🛡️ Maximum security posture | ||
| - 📋 Full audit trail control | ||
|
|
||
| **Ideal for:** Regulated industries (healthcare, finance, defense), government organizations, enterprises with strict data residency requirements, environments requiring offline operation. | ||
|
|
||
| [Learn more about Self-hosted deployment →](self-hosted-deployment.md) | ||
|
|
||
| ## Common Pipeline Execution Data Flow | ||
|
|
||
| All three deployment scenarios follow a similar pipeline execution pattern, with differences in where authentication happens and where data resides: | ||
|
|
||
| ### Standard Data Flow Steps | ||
|
Contributor
There was a problem hiding this comment. This definitely needs a diagram
Contributor
Author
There was a problem hiding this comment. Agreed - we might even have one laying around somewhere |
||
|
|
||
| 1. **Code Execution**: You write code and run pipelines with your client SDK using Python | ||
|
|
||
| 2. **Token Acquisition**: The ZenML client fetches short-lived tokens from your ZenML workspace for: | ||
|
Contributor
There was a problem hiding this comment. note: this only happens if you use service connectors
Contributor
Author
There was a problem hiding this comment. so what would be a more generally correct statement |
||
| - Pushing Docker images to your container registry | ||
| - Communicating with your artifact store | ||
| - Submitting workloads to your orchestrator | ||
| - *Note: Your local Python environment needs the client libraries for your stack components* | ||
|
|
||
| 3. **Image & Workload Submission**: The client automatically builds and pushes Docker images (and optionally code if no code repository is configured) to your container registry, then submits the workload to your orchestrator | ||
|
|
||
| 4. **Orchestrator Execution**: In the orchestrator environment: | ||
| - The Docker image is pulled from your container registry | ||
| - The necessary code is pulled in | ||
| - A connection to your ZenML workspace is established | ||
| - The relevant pipeline/step code is executed | ||
|
|
||
| 5. **Runtime Data Flow**: During execution: | ||
| - Pipeline and step run metadata is logged to your ZenML workspace | ||
| - Logs are streamed to your log backend | ||
| - Artifacts are written to your artifact store | ||
| - Metadata pointing to these artifacts is persisted | ||
|
|
||
| 6. **Observability**: The ZenML UI connects to your workspace and uses all persisted metadata to provide you with a complete observability plane | ||
|
|
||
| ### Deployment-Specific Differences | ||
|
|
||
| **SaaS**: Metadata is stored in ZenML infrastructure. Your ML data and compute remain in your infrastructure. | ||
|
|
||
| **Hybrid**: Metadata and control plane are split — authentication/RBAC happens at ZenML control plane, but all run metadata, artifacts, and compute stay in your infrastructure. | ||
|
Contributor
There was a problem hiding this comment. I thnk the authentication bit is the most important here and isnt really elaborated but maybe it is later?
Contributor
Author
There was a problem hiding this comment. What more would you like to know about this at this stage? |
||
|
|
||
| **Self-hosted**: All components (control plane, metadata, authentication, compute) run entirely within your infrastructure with zero external dependencies. | ||
|
|
||
| ## Making Your Choice | ||
|
|
||
| Consider these factors when deciding: | ||
|
|
||
| 1. **Data Location Requirements**: Where must your ML metadata and run data reside? | ||
| - Cloud-hosted is acceptable → **SaaS** | ||
| - Must stay in your infrastructure → **Hybrid** | ||
| - Must be completely isolated on-premises → **Self-hosted** | ||
|
|
||
| 2. **Infrastructure Complexity**: How much infrastructure control do you want? | ||
| - Minimal → **SaaS** | ||
| - Moderate → **Hybrid** | ||
| - Full control → **Self-hosted** | ||
|
|
||
| 3. **Time to Value**: How quickly do you need to be productive? | ||
| - Within 1 hour → **SaaS** | ||
| - Within 4 hours → **Hybrid** | ||
| - Within 8 hours (or longer planning period) → **Self-hosted** | ||
|
|
||
| 4. **Compliance Requirements**: What regulations apply to your organization? | ||
| - General business → **SaaS** | ||
| - Data residency rules → **Hybrid** | ||
| - Strict isolation requirements → **Self-hosted** | ||
|
|
||
| ## Security & Compliance | ||
|
|
||
| All ZenML Pro deployments include: | ||
|
|
||
| - ✅ **SOC 2 Type II** certification | ||
| - ✅ **ISO 27001** certification | ||
| - ✅ **Vulnerability Assessment Reports** available on request | ||
| - ✅ **Software Bill of Materials (SBOM)** available on request | ||
|
|
||
| For software deployed on your infrastructure (Hybrid and Self-hosted scenarios), ZenML provides comprehensive security documentation to support your compliance requirements. | ||
|
|
||
| ## Running Pipelines from the web UI | ||
|
|
||
| All deployment scenarios support running pipeline snapshots from the UI through [workload managers](workload-managers.md). Workload managers are built into the ZenML Pro workspace and can be configured to orchestrate pipeline execution on your Kubernetes cluster, AWS ECS, or GCP infrastructure. | ||
|
|
||
| Learn more: [Understanding Workload Managers](workload-managers.md) | ||
|
|
||
| ## Next Steps | ||
|
|
||
| - **Ready to start?** [Choose SaaS Deployment](saas-deployment.md) | ||
| - **Need metadata control?** [Set up Hybrid Deployment](hybrid-deployment.md) | ||
| - **Require complete isolation?** [Configure Self-hosted Deployment](self-hosted-deployment.md) | ||
| - **Deploying on your own infrastructure?** [See Self-hosted Deployment Guide](self-hosted.md) | ||
| - **Want to run pipelines from the UI?** [Configure Workload Managers](workload-managers.md) | ||
|
|
||
| {% hint style="info" %} | ||
| Not sure which option is right for you? [Book a call](https://www.zenml.io/book-your-demo) with our team to discuss your specific requirements. | ||
| {% endhint %} | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.