diff --git a/src/docs.json b/src/docs.json index 301e804c9e..4ef803f833 100644 --- a/src/docs.json +++ b/src/docs.json @@ -867,8 +867,15 @@ "langsmith/data-purging-compliance" ] }, + { + "group": "Access control & Authentication", + "pages": [ + "langsmith/rbac", + "langsmith/organization-workspace-operations", + "langsmith/authentication-methods" + ] + }, "langsmith/scalability-and-resilience", - "langsmith/authentication-methods", "langsmith/faq", "langsmith/regions-faq", "langsmith/pricing-faq" diff --git a/src/langsmith/administration-overview.mdx b/src/langsmith/administration-overview.mdx index 72ac481700..e7919703a4 100644 --- a/src/langsmith/administration-overview.mdx +++ b/src/langsmith/administration-overview.mdx @@ -3,6 +3,9 @@ title: Overview sidebarTitle: Overview --- +import OrgWorkspaceRole from '/snippets/langsmith/multi-workspace-org-roles.mdx'; +import PermissionReference from '/snippets/langsmith/permissions-reference.mdx'; + This overview covers topics related to managing users, organizations, and workspaces within LangSmith. ## Resource Hierarchy @@ -117,21 +120,25 @@ To see how to create a service key or Personal Access Token, see the [setup guid ### Organization roles -Organization roles are distinct from the Enterprise feature (RBAC) below and are used in the context of multiple [workspaces](#workspaces). Your organization role determines your workspace membership characteristics and your organization-level permissions. See the [organization setup guide](/langsmith/set-up-a-workspace#organization-roles) for more information. +Organization roles are distinct from the [Enterprise feature workspace RBAC](#workspace-roles-rbac) and are used in the context of multiple [workspaces](#workspaces). Your organization role determines your workspace membership characteristics and your [organization-level permissions](/langsmith/organization-workspace-operations). The organization role selected also impacts workspace membership as described here: -* `Organization Admin` grants full access to manage all organization configuration, users, billing, and workspaces. **An `Organization Admin` has `Admin` access to all workspaces in an organization** -* `Organization User` may read organization information but cannot execute any write actions at the organization level. An `Organization User` may create Personal Access Tokens. **An `Organization User` can be added to a subset of workspaces and assigned workspace roles as usual (if RBAC is enabled), which specify permissions at the workspace level.** -* `Organization Viewer` is equivalent to `Organization User`, but **cannot** create Personal Access Tokens. (for self-hosted, available in Helm chart version 0.11.25+) +- [Organization Admin](/langsmith/rbac#organization-admin) grants full access to manage all organization configuration, users, billing, and workspaces. + - An Organization Admin has `Admin` access to all workspaces in an organization. +- [Organization User](/langsmith/rbac#organization-user) may read organization information but cannot execute any write actions at the organization level. An Organization User may create [Personal Access Tokens](#personal-access-tokens-pats). + - An Organization User can be added to a subset of workspaces and assigned workspace roles as usual (if RBAC is enabled), which specify permissions at the workspace level. +- [Organization Viewer](/langsmith/rbac#organization-viewer) is equivalent to Organization User, but **cannot** create Personal Access Tokens. (for self-hosted, available in Helm chart version 0.11.25+). -The `Organization User` and `Organization Viewer` roles are only available in organizations on plans with multiple workspaces. In organizations limited to a single workspace, all users are `Organization Admins`. Custom organization-scoped roles are not available yet. + See [security settings](/langsmith/manage-organization-by-api#security-settings) for instructions on how to disable PAT creation for the entire organization. -See the table below for all organization permissions: +For more information on setting up organizations and workspaces, refer to the [organization setup guide](/langsmith/set-up-a-workspace#organization-roles) for more information. + +The following table provdies an overview of organization level permissions: | | Organization Viewer | Organization User | Organization Admin | | ------------------------------------------- | ------------------- | ----------------- | ------------------ | @@ -151,6 +158,7 @@ See the table below for all organization permissions: | Update data retention settings | ❌ | ❌ | ✅ | | Update usage limits | ❌ | ❌ | ✅ | + ### Workspace roles (RBAC) @@ -160,17 +168,19 @@ RBAC (Role-Based Access Control) is a feature that is only available to Enterpri Roles are used to define the set of permissions that a user has within a workspace. There are three built-in system roles that cannot be edited: -* `Admin` - has full access to all resources within the workspace -* `Viewer` - has read-only access to all resources within the workspace -* `Editor` - has full permissions except for workspace management (adding/removing users, changing roles, configuring service keys) +- [Workspace Admin](/langsmith/rbac#workspace-admin) has full access to all resources within the workspace. +- [Workspace Editor](/langsmith/rbac#workspace-editor) has full permissions except for workspace management (adding/removing users, changing roles, configuring service keys). +- [Workspace Viewer](/langsmith/rbac#workspace-viewer) has read-only access to all resources within the workspace. -Organization admins can also create/edit custom roles with specific permissions for different resources. +[Organization admins](/langsmith/rbac#organization-admin) can also create/edit custom roles with specific permissions for different resources. -Roles can be managed in organization settings under the `Roles` tab: +Roles can be managed in **Organization Settings** under the **Roles** tab: -![Roles](/langsmith/images/roles-tab-rbac.png) +![The Organization members and roles view showing a list of the roles.](/langsmith/images/roles-tab-rbac.png) -For more details on assigning and creating roles, see the [access control setup guide](/langsmith/user-management). +- For comprehensive documentation on roles and permissions, refer to the [Role-based access control](/langsmith/rbac) guide. +- For more details on assigning and creating roles, refer to the [User Management](/langsmith/user-management) guide. +- ## Best Practices diff --git a/src/langsmith/organization-workspace-operations.mdx b/src/langsmith/organization-workspace-operations.mdx new file mode 100644 index 0000000000..bf7ae2e149 --- /dev/null +++ b/src/langsmith/organization-workspace-operations.mdx @@ -0,0 +1,567 @@ +--- +title: Organization and workspace operations reference +sidebarTitle: Organization and workspace operations +mode: wide +--- + +This page provides a comprehensive reference table of [workspace](/langsmith/administration-overview#workspaces) and [organization](/langsmith/administration-overview#organizations) operations and which roles can perform them. + +The list includes API operations in LangSmith along with: + +- Which system roles can perform each operation. +- The specific permission string required. +- Notes about partial access or special cases. + + +For an overview of LangSmith's RBAC system, role definitions, and permission concepts, refer to [Role-based access control](/langsmith/rbac). + + +## Contents + +| Organization-level operations | Workspace-level operations | +|-------------------------------|---------------------------| +| **Core management:**
• [Organization settings](#organization-settings): Org info and configuration
• [Workspaces](#workspaces): Workspace management
• [Organization members](#organization-members): Member management
• [Roles and permissions](#roles-and-permissions): Custom roles | **Core resources:**
• [Projects](#projects): Organize traces and runs
• [Runs](#runs): Individual execution traces
• [Datasets](#datasets): Test datasets for evaluation
• [Examples](#examples): Individual dataset examples
• [Experiments](#experiments): Comparative experiments | +| **Security and authentication:**
• [SSO and authentication](#sso-and-authentication): Single sign-on setup
• [SCIM](#scim): Identity provisioning
• [Access policies](#access-policies): Attribute-based access control | **Monitoring and analysis:**
• [Rules](#rules): Automated run rules
• [Alerts](#alerts): Alert rules for monitoring
• [Feedback](#feedback): Scores and labels on outputs
• [Annotation Queues](#annotation-queues): Human review queues
• [Charts](#charts): Custom visualizations | +| **Billing and accounts:**
• [Billing and payments](#billing-and-payments): Subscription management
• [API keys](#api-keys): Org-level keys | **Development and configuration:**
• [Prompts](#prompts): Prompt templates (LangChain Hub)
• [Deployments](#deployments): Deployment configurations
• [MCP Servers](#mcp-servers): Model Context Protocol servers | +| **Analytics:**
• [Charts and dashboards](#organization-charts-and-dashboards): Org-level visualizations
• [Usage and analytics](#usage-and-analytics): Usage tracking and TTL settings | **Workspace management:**
• [Workspace settings](#workspace-settings-and-management): Members, settings
• [Tags](#tags): Metadata tagging system
• [Bulk Exports](#bulk-exports): Data export operations | + +**Additional information:** + +- [User-level operations](#user-level-operations): Operations for all authenticated users +- [Permission inheritance](#permission-inheritance): How roles inherit across org/workspaces + +## Legend + +- ✓ **Allowed**: User with this role can perform this action +- ✗ **Not Allowed**: User with this role cannot perform this action +- ⚠ **Partial**: User has limited access (see notes) + +## Organization-level operations + + +Organization-level operations are controlled by organization roles, which are separate from the RBAC feature. Learn more in the [Role-based access control](/langsmith/rbac#organization-roles) guide. + + +### Organization settings + +| Operation | Org Admin | Org User | Org Viewer | Required Permission | +|-----------|:---------:|:--------:|:----------:|---------------------| +| View organization info | ✓ | ✓ | ✓ | `organization:read` | +| View organization dashboard | ✓ | ✓ | ✓ | `organization:read` | +| Update organization info | ✓ | ✗ | ✗ | `organization:manage` | +| View billing info | ✓ | ✓ | ✓ | `organization:read` | +| View company info | ✓ | ✓ | ✓ | `organization:read` | +| Set company info | ✓ | ✗ | ✗ | `organization:manage` | + +### Workspaces + +Organization-level workspace management operations. + +| Operation | Org Admin | Org User | Org Viewer | Required Permission | +|-----------|:---------:|:--------:|:----------:|---------------------| +| List all workspaces | ✓ | ✓ | ✓ | `organization:read` | +| Create workspace | ✓ | ✗ | ✗ | `organization:manage` | + +### Organization members + +| Operation | Org Admin | Org User | Org Viewer | Required Permission | +|-----------|:---------:|:--------:|:----------:|---------------------| +| View organization members | ✓ | ✓ | ✓ | `organization:read` | +| View active org members | ✓ | ✓ | ✓ | `organization:read` | +| View pending org members | ✓ | ✓ | ✓ | `organization:read` | +| Invite member to organization | ✓ | ✗ | ✗ | `organization:manage` | +| Invite members (batch) | ✓ | ✗ | ✗ | `organization:manage` | +| Add basic auth members | ✓ | ✗ | ✗ | `organization:manage` | +| Remove organization member | ✓ | ✗ | ✗ | `organization:manage` | +| Update organization member role | ✓ | ✗ | ✗ | `organization:manage` | +| Delete pending org member | ✓ | ✗ | ✗ | `organization:manage` | + +### Roles and permissions + +| Operation | Org Admin | Org User | Org Viewer | Required Permission | +|-----------|:---------:|:--------:|:----------:|---------------------| +| List organization roles | ✓ | ✓ | ✓ | `organization:read` | +| List available permissions | ✓ | ✓ | ✓ | N/A (user-level) | +| Create custom role | ✓ | ✗ | ✗ | `organization:manage` | +| Update custom role | ✓ | ✗ | ✗ | `organization:manage` | +| Delete custom role | ✓ | ✗ | ✗ | `organization:manage` | + +### SSO and authentication + +| Operation | Org Admin | Org User | Org Viewer | Required Permission | +|-----------|:---------:|:--------:|:----------:|---------------------| +| View SSO settings | ✓ | ✓ | ✓ | `organization:read` | +| Create SSO settings | ✓ | ✗ | ✗ | `organization:manage` | +| Update SSO settings | ✓ | ✗ | ✗ | `organization:manage` | +| Delete SSO settings | ✓ | ✗ | ✗ | `organization:manage` | +| View login methods | ✓ | ✓ | ✓ | `organization:read` | +| Update allowed login methods | ✓ | ✗ | ✗ | `organization:manage` | +| Set default SSO provision | ✓ | ✗ | ✗ | `organization:manage` | + +### SCIM + +System for Cross-domain Identity Management for user provisioning. + +| Operation | Org Admin | Org User | Org Viewer | Required Permission | +|-----------|:---------:|:--------:|:----------:|---------------------| +| List SCIM tokens | ✓ | ✓ | ✓ | `organization:read` | +| Get SCIM token | ✓ | ✓ | ✓ | `organization:read` | +| Create SCIM token | ✓ | ✗ | ✗ | `organization:manage` | +| Update SCIM token | ✓ | ✗ | ✗ | `organization:manage` | +| Delete SCIM token | ✓ | ✗ | ✗ | `organization:manage` | + +### Access policies + +Attribute-based access control (ABAC) policies for fine-grained permissions. + + +ABAC is in private preview. + + +| Operation | Org Admin | Org User | Org Viewer | Required Permission | +|-----------|:---------:|:--------:|:----------:|---------------------| +| List access policies | ✓ | ✓ | ✓ | `organization:read` | +| Get access policy | ✓ | ✓ | ✓ | `organization:read` | +| Create access policy | ✓ | ✗ | ✗ | `organization:manage` | +| Delete access policy | ✓ | ✗ | ✗ | `organization:manage` | +| Attach access policy to role | ✓ | ✗ | ✗ | `organization:manage` | + +### Billing and payments + +| Operation | Org Admin | Org User | Org Viewer | Required Permission | +|-----------|:---------:|:--------:|:----------:|---------------------| +| Create Stripe setup intent | ✓ | ✗ | ✗ | `organization:manage` | +| Handle payment method creation | ✓ | ✗ | ✗ | `organization:manage` | +| Change payment plan | ✓ | ✗ | ✗ | `organization:manage` | +| Create Stripe checkout session | ✓ | ✗ | ✗ | `organization:manage` | +| Confirm checkout completion | ✓ | ✗ | ✗ | `organization:manage` | +| Create Stripe account links | ✓ | ✗ | ✗ | `organization:manage` | + +### API keys + +| Operation | Org Admin | Org User | Org Viewer | Required Permission | +|-----------|:---------:|:--------:|:----------:|---------------------| +| List org-scoped API keys | ✓ | ✓ | ✓ | `organization:read` | +| Create org-scoped API key (workspace-scoped)* | ✓ | ⚠ | ✗ | `organization:pats:create` | +| Create org-scoped API key (org-wide)* | ✓ | ✗ | ✗ | `organization:pats:create` + `organization:manage` | +| List personal access tokens | ✓ | ✓ | ✗ | `organization:read` | +| Create personal access token | ✓ | ✓ | ✗ | `organization:pats:create` | +| Delete personal access token | ✓ | ✓ | ✗ | `organization:read` | + + +\* Organization Users can create workspace-scoped API keys only for workspaces where they are a Workspace Admin. Org-wide API keys require the Organization Admin role. + + +### Organization charts and dashboards + +| Operation | Org Admin | Org User | Org Viewer | Required Permission | +|-----------|:---------:|:--------:|:----------:|---------------------| +| List org charts | ✓ | ✓ | ✓ | `organization:read` | +| Get org chart by ID | ✓ | ✓ | ✓ | `organization:read` | +| Create org chart | ✓ | ✗ | ✗ | `organization:manage` | +| Update org chart | ✓ | ✗ | ✗ | `organization:manage` | +| Delete org chart | ✓ | ✗ | ✗ | `organization:manage` | +| Render org chart | ✓ | ✓ | ✓ | `organization:read` | +| Get org chart section | ✓ | ✓ | ✓ | `organization:read` | +| Create org chart section | ✓ | ✗ | ✗ | `organization:manage` | +| Update org chart section | ✓ | ✗ | ✗ | `organization:manage` | +| Delete org chart section | ✓ | ✗ | ✗ | `organization:manage` | +| Render org chart section | ✓ | ✓ | ✓ | `organization:read` | + +### Usage and analytics + +| Operation | Org Admin | Org User | Org Viewer | Required Permission | +|-----------|:---------:|:--------:|:----------:|---------------------| +| View organization usage | ✓ | ✓ | ✓ | `organization:read` | +| View TTL settings | ✓ | ✓ | ✓ | `organization:read` | +| Upsert TTL settings | ✓ | ✗ | ✗ | `organization:manage` | + +## Workspace-level operations + +These operations are controlled by [workspace-level roles and permissions](/langsmith/rbac#workspace-roles). + + +To understand what each role means and their overall capabilities, refer to the [Role-based access control](/langsmith/rbac) guide. + + +### Projects + +Projects organize traces and runs from your LLM applications. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| Create a new project | ✓ | ✗ | ✗ | `projects:create` | +| View project list | ✓ | ✓ | ✓ | `projects:read` | +| View project details | ✓ | ✓ | ✓ | `projects:read` | +| View prebuilt dashboard | ✓ | ✓ | ✓ | `projects:read` | +| View project metadata (top K values) | ✓ | ✓ | ✓ | `projects:read` | +| Update project metadata (name, description, tags) | ✓ | ✓ | ✗ | `projects:update` | +| Create filter view | ✓ | ✗ | ✗ | `projects:create` | +| View filter views | ✓ | ✓ | ✓ | `projects:read` | +| View specific filter view | ✓ | ✓ | ✓ | `projects:read` | +| Update filter view | ✓ | ✓ | ✗ | `projects:update` | +| Delete filter view | ✓ | ✗ | ✗ | `projects:delete` | +| Delete a project | ✓ | ✗ | ✗ | `projects:delete` | +| Delete multiple projects | ✓ | ✗ | ✗ | `projects:delete` | +| Get insights jobs (Beta) | ✓ | ✓ | ✓ | `projects:read` | +| Get specific insights job (Beta) | ✓ | ✓ | ✓ | `projects:read` | +| Create insights job (Beta) | ✓ | ✓ | ✓ | `projects:read` + `rules:create` | +| Update insights job (Beta) | ✓ | ✓ | ✗ | `projects:update` | +| Delete insights job (Beta) | ✓ | ✗ | ✗ | `projects:delete` | +| Get insights job configs (Beta) | ✓ | ✓ | ✓ | `rules:read` | +| Create insights job config (Beta) | ✓ | ✓ | ✗ | `rules:create` | +| Auto-generate insights job config (Beta) | ✓ | ✓ | ✗ | `rules:create` | +| Update insights job config (Beta) | ✓ | ✓ | ✗ | `rules:update` | +| Delete insights job config (Beta) | ✓ | ✓ | ✗ | `rules:delete` | +| Get run cluster from insights job (Beta) | ✓ | ✓ | ✓ | `projects:read` | +| Get runs from insights job (Beta) | ✓ | ✓ | ✓ | `projects:read` | + +### Runs + +Individual execution traces and spans from your LLM applications. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| Send traces from SDK (includes single run, batch, multipart, and OTEL) | ✓ | ✓ | ✗ | `runs:create` | +| View a specific run | ✓ | ✓ | ✓ | `runs:read` | +| View thread preview | ✓ | ✓ | ✓ | `runs:read` | +| Query/list runs | ✓ | ✓ | ✓ | `runs:read` | +| View run statistics | ✓ | ✓ | ✓ | `runs:read` | +| View grouped run statistics | ✓ | ✓ | ✓ | `runs:read` | +| Group runs by expression | ✓ | ✓ | ✓ | `runs:read` | +| Generate filter query from natural language | ✓ | ✓ | ✓ | `runs:read` | +| Prefetch runs | ✓ | ✓ | ✓ | `runs:read` | +| Update a run (PATCH) | ✓ | ✓ | ✗ | `runs:create` | +| View run sharing state | ✓ | ✓ | ✓ | `runs:read` | +| Share a run publicly | ✓ | ✓ | ✗ | `runs:share` | +| Unshare a run | ✓ | ✓ | ✗ | `runs:share` | +| Delete runs by trace ID or metadata | ✓ | ✗ | ✗ | `runs:delete` | + +### Rules + +Automated run rules that trigger actions based on run conditions. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| List all run rules | ✓ | ✓ | ✓ | `rules:read` | +| Create a run rule | ✓ | ✓ | ✗ | `rules:create` | +| Update a run rule | ✓ | ✓ | ✗ | `rules:update` | +| Delete a run rule | ✓ | ✓ | ✗ | `rules:delete` | +| View rule logs | ✓ | ✓ | ✓ | `rules:read` | +| Get last applied rule | ✓ | ✓ | ✓ | `rules:read` | +| Manually trigger a rule | ✓ | ✓ | ✗ | `rules:update` | +| Trigger multiple rules | ✓ | ✓ | ✗ | `rules:update` | + +### Alerts + +Alert rules for monitoring run conditions. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| Create alert rule | ✓ | ✓ | ✓ | `runs:read` | +| Update alert rule | ✓ | ✓ | ✓ | `runs:read` | +| Delete alert rule | ✓ | ✓ | ✓ | `runs:read` | +| Get alert rule | ✓ | ✓ | ✓ | `runs:read` | +| List alert rules | ✓ | ✓ | ✓ | `runs:read` | +| Test alert action | ✓ | ✓ | ✓ | `runs:read` | + +### Datasets + +Test datasets with examples for evaluation. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| Create a dataset | ✓ | ✓ | ✗ | `datasets:create` | +| List datasets | ✓ | ✓ | ✓ | `datasets:read` | +| View dataset details | ✓ | ✓ | ✓ | `datasets:read` | +| Update dataset metadata | ✓ | ✓ | ✗ | `datasets:update` | +| Delete a dataset | ✓ | ✗ | ✗ | `datasets:delete` | +| Upload CSV dataset | ✓ | ✓ | ✗ | `datasets:create` | +| Clone dataset | ✓ | ✓ | ✗ | `datasets:update` | +| Get dataset version | ✓ | ✓ | ✓ | `datasets:read` | +| Get dataset versions | ✓ | ✓ | ✓ | `datasets:read` | +| Diff dataset versions | ✓ | ✓ | ✓ | `datasets:read` | +| Update dataset version (tags) | ✓ | ✓ | ✗ | `datasets:update` | +| Download dataset (OpenAI format) | ✓ | ✓ | ✓ | `datasets:read` | +| Download dataset (OpenAI fine-tuning format) | ✓ | ✓ | ✓ | `datasets:read` | +| Download dataset (CSV) | ✓ | ✓ | ✓ | `datasets:read` | +| Download dataset (JSONL) | ✓ | ✓ | ✓ | `datasets:read` | +| View dataset sharing state | ✓ | ✓ | ✓ | `datasets:read` | +| Share dataset publicly | ✓ | ✗ | ✗ | `datasets:share` | +| Unshare dataset | ✓ | ✗ | ✗ | `datasets:share` | +| Get index info | ✓ | ✓ | ✓ | `datasets:read` | +| Index dataset | ✓ | ✓ | ✗ | `datasets:update` | +| Sync dataset index | ✓ | ✓ | ✗ | `datasets:update` | +| Remove dataset index | ✓ | ✓ | ✗ | `datasets:update` | +| Search dataset | ✓ | ✓ | ✓ | `datasets:read` | +| Generate synthetic examples | ✓ | ✓ | ✗ | `datasets:update` | +| Get dataset splits | ✓ | ✓ | ✓ | `datasets:read` | +| Update dataset splits | ✓ | ✓ | ✓ | `datasets:read` | +| Run playground experiment (batch) | ✓ | ⚠ | ✗ | `prompts:read` + `datasets:read` + `projects:create` | +| Run playground experiment (stream) | ✓ | ⚠ | ✗ | `prompts:read` + `datasets:read` + `projects:create` | +| Run studio experiment | ✓ | ⚠ | ✗ | `datasets:read` + `projects:create` | + + +Workspace Editors have partial access because they cannot create projects, which limits their ability to create new experiments. + + +### Examples + +Individual examples within datasets. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| Count examples | ✓ | ✓ | ✓ | `datasets:read` | +| View a specific example | ✓ | ✓ | ✓ | `datasets:read` | +| List examples | ✓ | ✓ | ✓ | `datasets:read` | +| Create a new example | ✓ | ✓ | ✗ | `datasets:update` | +| Create examples (bulk) | ✓ | ✓ | ✗ | `datasets:update` | +| Update a single example | ✓ | ✓ | ✗ | `datasets:update` | +| Update examples (bulk) | ✓ | ✓ | ✗ | `datasets:update` | +| Update examples (multipart) | ✓ | ✓ | ✗ | `datasets:update` | +| Upload examples from CSV | ✓ | ✓ | ✗ | `datasets:update` | +| Upload examples from JSONL | ✓ | ✓ | ✗ | `datasets:update` | +| Delete a single example | ✓ | ✓ | ✗ | `datasets:update` | +| Delete examples (bulk) | ✓ | ✓ | ✗ | `datasets:update` | +| View examples with runs | ✓ | ✓ | ✓ | `datasets:read` | +| View grouped examples with runs | ✓ | ✓ | ✓ | `datasets:read` | +| Validate a single example | ✓ | ✓ | ✓ | `datasets:read` | +| Validate examples (bulk) | ✓ | ✓ | ✓ | `datasets:read` | + +### Experiments + +Comparative experiments for evaluating LLM outputs. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| View comparative experiments | ✓ | ✓ | ✓ | `projects:read` | +| Create comparative experiment | ✓ | ⚠ | ✗ | `projects:create` | +| Delete comparative experiment | ✓ | ✗ | ✗ | `projects:delete` | +| View examples with runs | ✓ | ✓ | ✓ | `datasets:read` | +| View grouped examples with runs | ✓ | ✓ | ✓ | `datasets:read` | +| View grouped experiments | ✓ | ✓ | ✓ | `datasets:read` | +| View feedback delta | ✓ | ✓ | ✓ | `datasets:read` | +| Upload experiment results | ✓ | ⚠ | ✗ | `datasets:create` + `datasets:update` + `projects:create` + `runs:create` | +| Get experiment view overrides | ✓ | ✓ | ✗ | `datasets:update` | +| Create experiment view override | ✓ | ✓ | ✗ | `datasets:update` | +| Update experiment view override | ✓ | ✓ | ✗ | `datasets:update` | +| Delete experiment view override | ✓ | ✓ | ✗ | `datasets:update` | + + +Workspace Editors have partial access because they cannot create projects, which limits their ability to create new experiments. + + +### Feedback + +Scores, labels, and corrections on LLM outputs. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| List feedback formulas | ✓ | ✓ | ✓ | `feedback:read` | +| Get feedback formula | ✓ | ✓ | ✓ | `feedback:read` | +| Create feedback formula | ✓ | ✓ | ✗ | `feedback:create` | +| Update feedback formula | ✓ | ✓ | ✗ | `feedback:update` | +| Delete feedback formula | ✓ | ✓ | ✗ | `feedback:delete` | +| View specific feedback | ✓ | ✓ | ✓ | `feedback:read` | +| List feedbacks | ✓ | ✓ | ✓ | `feedback:read` | +| Create feedback | ✓ | ✓ | ✗ | `feedback:create` | +| Eagerly create feedback | ✓ | ✓ | ✗ | `feedback:create` | +| Update feedback | ✓ | ✓ | ✗ | `feedback:update` | +| Delete feedback | ✓ | ✓ | ✗ | `feedback:delete` | +| Batch ingest feedback | ✓ | ✓ | ✗ | `feedback:create` | +| Create feedback ingest token | ✓ | ✓ | ✗ | `feedback:create` | +| List feedback ingest tokens | ✓ | ✓ | ✗ | `feedback:create` | +| Create feedback with token (no auth required) | ✓ | ✓ | ✓ | N/A (token-based) | +| List feedback configs | ✓ | ✓ | ✓ | `feedback:read` | +| Create feedback config | ✓ | ✓ | ✗ | `feedback:create` | +| Update feedback config | ✓ | ✓ | ✗ | `feedback:update` | + +### Annotation Queues + +Human review queues for LLM outputs. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| List annotation queues | ✓ | ✓ | ✓ | `annotation-queues:read` | +| Get annotation queue | ✓ | ✓ | ✓ | `annotation-queues:read` | +| Create annotation queue | ✓ | ✓ | ✗ | `annotation-queues:create` | +| Update annotation queue | ✓ | ✓ | ✗ | `annotation-queues:update` | +| Delete annotation queue | ✓ | ✗ | ✗ | `annotation-queues:delete` | +| Populate annotation queue | ✓ | ✓ | ✗ | `annotation-queues:update` | +| Get runs from queue | ✓ | ✓ | ✓ | `annotation-queues:read` | +| Get run from queue (by index) | ✓ | ✓ | ✓ | `annotation-queues:read` | +| Get queues for run | ✓ | ✓ | ✓ | `annotation-queues:read` | +| Get queue total size | ✓ | ✓ | ✓ | `annotation-queues:read` | +| Get queue total archived | ✓ | ✓ | ✓ | `annotation-queues:read` | +| Get queue size | ✓ | ✓ | ✓ | `annotation-queues:read` | +| Add runs to queue | ✓ | ✓ | ✗ | `annotation-queues:update` | +| Update run in queue | ✓ | ✓ | ✗ | `annotation-queues:update` | +| Delete run from queue | ✓ | ✓ | ✗ | `annotation-queues:update` | +| Delete runs from queue (bulk) | ✓ | ✓ | ✗ | `annotation-queues:update` | +| Create identity annotation queue run status | ✓ | ✓ | ✗ | `annotation-queues:update` | +| Export archived runs | ✓ | ✓ | ✓ | `annotation-queues:read` | + +### Prompts + +Prompt templates and chains in the LangChain Hub. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| List prompt repos | ✓ | ✓ | ✓ | `prompts:read` | +| View prompt repo | ✓ | ✓ | ✓ | `prompts:read` | +| Create prompt repo | ✓ | ✓ | ✗ | `prompts:create` | +| Fork prompt repo | ✓ | ✓ | ✗ | `prompts:create` | +| Update prompt repo | ✓ | ✓ | ✗ | `prompts:update` | +| Delete prompt repo | ✓ | ✓ | ✗ | `prompts:delete` | +| List commits | ✓ | ✓ | ✓ | `prompts:read` | +| View commit | ✓ | ✓ | ✓ | `prompts:read` | +| Push commit | ✓ | ✓ | ✗ | `prompts:update` | +| List repo tags | ✓ | ✓ | ✓ | `prompts:read` | +| Get all tags | ✓ | ✓ | ✓ | `prompts:read` | +| Create tag | ✓ | ✓ | ✗ | `prompts:create` | +| Update tag | ✓ | ✓ | ✗ | `prompts:update` | +| Delete tag | ✓ | ✓ | ✗ | `prompts:delete` | +| View events | ✓ | ✓ | ✓ | `prompts:read` | +| List comments | ✓ | ✓ | ✓ | `prompts:read` | +| Create comment | ✓ | ✓ | ✗ | `prompts:read` | +| Delete comment | ✓ | ✓ | ✗ | `prompts:read` | +| Toggle like | ✓ | ✓ | ✗ | `prompts:read` | +| Optimize prompt | ✓ | ✓ | ✗ | `prompts:update` | +| List optimization jobs | ✓ | ✓ | ✓ | `prompts:read` | +| Create optimization job | ✓ | ✓ | ✗ | `prompts:create` | +| Update optimization job | ✓ | ✓ | ✗ | `prompts:update` | +| Delete optimization job | ✓ | ✓ | ✗ | `prompts:delete` | +| Invoke prompt canvas | ✓ | ✓ | ✗ | `prompts:update` | +| List quick actions | ✓ | ✓ | ✓ | `prompts:read` | +| Create quick action | ✓ | ✓ | ✓ | `prompts:read` | +| Delete quick action | ✓ | ✓ | ✓ | `prompts:read` | +| Update quick action | ✓ | ✓ | ✓ | `prompts:read` | + + +Some prompt operations support public access for shared prompts. + + +### Charts + +Custom visualizations and dashboards. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| List charts | ✓ | ✓ | ✓ | `charts:read` | +| Get chart by ID | ✓ | ✓ | ✓ | `charts:read` | +| Create chart | ✓ | ✓ | ✗ | `charts:create` | +| Update chart | ✓ | ✓ | ✗ | `charts:update` | +| Delete chart | ✓ | ✓ | ✗ | `charts:delete` | +| Render chart | ✓ | ✓ | ✓ | `charts:read` | +| List chart sections | ✓ | ✓ | ✓ | `charts:read` | +| Get chart section by ID | ✓ | ✓ | ✓ | `charts:read` | +| Create chart section | ✓ | ✓ | ✗ | `charts:create` | +| Update chart section | ✓ | ✓ | ✗ | `charts:update` | +| Delete chart section | ✓ | ✓ | ✗ | `charts:delete` | +| Render chart section | ✓ | ✓ | ✓ | `charts:read` | + +### Deployments + +[LangSmith Deployment](/langsmith/deployments) configurations. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| Create deployment | ✓ | ✓ | ✗ | `deployments:create` | +| View deployment | ✓ | ✓ | ✓ | `deployments:read` | +| Update deployment | ✓ | ✓ | ✗ | `deployments:update` | +| Delete deployment | ✓ | ✗ | ✗ | `deployments:delete` | + +### Workspace settings and management + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| View workspace info | ✓ | ✓ | ✓ | `workspaces:read` | +| View workspace statistics | ✓ | ✓ | ✓ | `workspaces:read` | +| Update workspace (name, description) | ✓ | ✗ | ✗ | `workspaces:manage` | +| Delete workspace | ✓ | ✗ | ✗ | `workspaces:manage` | +| View workspace members | ✓ | ✓ | ✓ | `workspaces:read` | +| View active workspace members | ✓ | ✓ | ✓ | `workspaces:read` | +| View pending workspace members | ✓ | ✓ | ✓ | `workspaces:read` | +| Add member to workspace | ✓ | ✗ | ✗ | `workspaces:manage` | +| Add members (batch) | ✓ | ✗ | ✗ | `workspaces:manage` | +| Update workspace member role | ✓ | ✗ | ✗ | `workspaces:manage` | +| Remove workspace member | ✓ | ✗ | ✗ | `workspaces:manage` | +| Delete pending workspace member | ✓ | ✗ | ✗ | `workspaces:manage` | +| View usage limits | ✓ | ✓ | ✓ | `workspaces:read` | +| View shared entities | ✓ | ✓ | ✓ | `workspaces:read` | +| Bulk unshare entities | ✓ | ✗ | ✗ | `workspaces:manage` | + +### Tags + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| List tag keys | ✓ | ✓ | ✓ | `workspaces:read` | +| Get tag key | ✓ | ✓ | ✓ | `workspaces:read` | +| Create tag key | ✓ | ✗ | ✗ | `workspaces:manage` | +| Update tag key | ✓ | ✗ | ✗ | `workspaces:manage` | +| Delete tag key | ✓ | ✗ | ✗ | `workspaces:manage` | +| List tag values | ✓ | ✓ | ✓ | `workspaces:read` | +| Get tag value | ✓ | ✓ | ✓ | `workspaces:read` | +| Create tag value | ✓ | ✗ | ✗ | `workspaces:manage` | +| Update tag value | ✓ | ✗ | ✗ | `workspaces:manage` | +| Delete tag value | ✓ | ✗ | ✗ | `workspaces:manage` | +| List tags | ✓ | ✓ | ✓ | `workspaces:read` | +| List tags for resource | ✓ | ✓ | ✓ | `workspaces:read` | +| List tags for resources (batch) | ✓ | ✓ | ✓ | `workspaces:read` | +| List taggings | ✓ | ✓ | ✓ | `workspaces:read` | +| Create tagging | ✓ | ✗ | ✗ | `workspaces:manage` | +| Delete tagging | ✓ | ✗ | ✗ | `workspaces:manage` | + +### Bulk exports + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| List bulk exports | ✓ | ✓ | ✓ | `workspaces:read` | +| Get bulk export | ✓ | ✓ | ✓ | `workspaces:read` | +| Create bulk export | ✓ | ✗ | ✗ | `workspaces:manage` | +| Cancel bulk export | ✓ | ✗ | ✗ | `workspaces:manage` | +| Get bulk export destinations | ✓ | ✓ | ✓ | `workspaces:read` | +| Get bulk export destination | ✓ | ✓ | ✓ | `workspaces:read` | +| Create bulk export destination | ✓ | ✗ | ✗ | `workspaces:manage` | +| Get filtered export runs | ✓ | ✓ | ✓ | `workspaces:read` | + +### MCP servers + +Model Context Protocol servers for extended functionality. + +| Operation | Workspace Admin | Workspace Editor | Workspace Viewer | Required Permission | +|-----------|:---------------:|:--------------:|:----------------:|---------------------| +| List MCP servers | ✓ | ✓ | ✓ | `workspaces:read` | +| Get MCP server | ✓ | ✓ | ✓ | `workspaces:read` | +| Create MCP server | ✓ | ✓ | ✓ | `workspaces:read` | +| Update MCP server | ✓ | ✓ | ✓ | `workspaces:read` | +| Delete MCP server | ✓ | ✓ | ✓ | `workspaces:read` | + +## User-level operations + +These operations are available to all authenticated users and don't require specific workspace or organization permissions: + +- View own user profile +- Update own user profile +- List organizations for user +- Create new organization +- List pending workspace invites +- Delete pending workspace invite +- Claim pending workspace invite +- List pending organization invites +- Delete pending organization invite +- Claim pending organization invite + +## Permission inheritance + +### Organization to workspace + +- [Organization Admin](/langsmith/rbac#organization-admin) automatically has full permissions in all workspaces. +- [Organization User](/langsmith/rbac#organization-user) and [Organization Viewer](/langsmith/rbac#organization-viewer) only get workspace access when explicitly added to workspaces with workspace-level roles. + +For detailed role definitions, refer to [Organization roles](/langsmith/rbac#organization-roles) and [Workspace roles](/langsmith/rbac#workspace-roles). + +### Workspace role independence + +- Users can have different workspace roles in different workspaces. +- A user might be a [Workspace Admin](/langsmith/rbac#workspace-admin) in one workspace and a [Workspace Viewer](/langsmith/rbac#workspace-viewer) in another. diff --git a/src/langsmith/rbac.mdx b/src/langsmith/rbac.mdx new file mode 100644 index 0000000000..58a75ddea2 --- /dev/null +++ b/src/langsmith/rbac.mdx @@ -0,0 +1,181 @@ +--- +title: Role-based access control +sidebarTitle: Role-based access control +--- + +import OrgWorkspaceRole from '/snippets/langsmith/multi-workspace-org-roles.mdx'; +import PermissionReference from '/snippets/langsmith/permissions-reference.mdx'; + +This reference explains LangSmith's Role-Based Access Control (RBAC) system for managing organization-level and workspace-level permissions. + + +RBAC (Role-Based Access Control) is an Enterprise feature for managing workspace-level permissions. If you are interested in this feature, [contact our sales team](https://www.langchain.com/contact-sales). Other plans default to using the Admin role for all users. + + +LangSmith's RBAC system manages user permissions within workspaces. RBAC allows you to control who can access your LangSmith [workspace](/langsmith/administration-overview#workspaces) and what they can do within it. + +In LangSmith, each user has: +- One [**organization role**](#organization-roles) that applies across the entire organization (separate from workspace RBAC). + - +- One [**workspace role**](#workspace-roles) per workspace they're a member of (requires Enterprise RBAC feature). + +On Enterprise plans, organizations can create [custom workspace roles](#custom-roles) with granular permission combinations. + +To learn how to set up RBAC and assign roles to users, refer to the [User Management guide](/langsmith/user-management#set-up-access-control). + + + + + +## Role types + +### Organization roles + +Organization roles are **distinct from the workspace RBAC feature** and are used to manage organization-wide capabilities. The roles are system-defined and cannot be modified or extended. These roles are available in multi-workspace organizations on [Plus and Enterprise plans](https://langchain.com/pricing). + +| Role | Description | +|------|-------------| +| [Organization Admin](#organization-admin) | Full permissions to manage organization configuration, users, billing, and workspaces | +| [Organization User](#organization-user) | Read access to organization information and ability to create personal access tokens | +| [Organization Viewer](#organization-viewer) | Read-only access to organization information | + + +In organizations limited to a single workspace, all users are [Organization Admins](#organization-admin). + + +#### Organization Admin + +**Description**: Full permissions to manage all organization configuration, users, billing, and workspaces. + +**Permissions**: +- `organization:manage` - Full control over organization settings, SSO, security, billing +- `organization:read` - Read access to all organization information +- `organization:pats:create` - Create organization-level [personal access tokens](/langsmith/administration-overview#personal-access-tokens-pats) + + + +**Key Capabilities**: +- Manage [organization settings](/langsmith/set-up-a-workspace#set-up-an-organization) and branding +- Configure [SSO and authentication methods](/langsmith/user-management#set-up-saml-sso-for-your-organization) +- Manage [billing](/langsmith/billing) and subscription plans +- Create and delete [workspaces](/langsmith/set-up-a-workspace) +- Invite and remove organization members +- Assign organization and workspace roles to members +- Create and manage [custom roles](#custom-roles) +- Configure RBAC and ABAC (Attribute-Based Access Control) policies (Note that ABAC is in private preview) +- View organization [usage](/langsmith/administration-overview#usage-limits) and analytics + +For details on setting up and managing your organization, refer to the [Administration Overview](/langsmith/administration-overview#organizations). + +#### Organization User + +**Description**: Read access to organization information and ability to create personal access tokens. + +**Permissions**: +- `organization:read` - Read access to organization information +- `organization:pats:create` - Create personal access tokens + + + +**Key Capabilities**: +- View organization members and workspaces +- View organization settings (but not modify) +- Create [personal access tokens](/langsmith/administration-overview#personal-access-tokens-pats) for API access +- Join workspaces they're invited to + +**Restrictions**: +- Cannot modify organization settings +- Cannot manage billing or subscriptions +- Cannot create or delete workspaces +- Cannot invite or remove organization members +- Cannot manage roles or permissions + +You can add an Organization User to a subset of workspaces and assigned workspace roles (if RBAC is enabled), which specify permissions at the workspace level. + +#### Organization Viewer + +**Description**: Read-only access to organization information. + +**Permissions**: +- `organization:read` - Read access to organization information + + + +**Key Capabilities**: +- View organization members and workspaces +- View organization settings + +**Restrictions**: +- Cannot modify anything at the organization level +- Cannot create personal access tokens +- Cannot manage billing, workspaces, or members + +### Workspace roles + +Workspace roles are part of the **Enterprise RBAC feature** and control what users can do with resources inside a workspace: + +| Role | Description | +|------|-------------| +| [Workspace Admin](#workspace-admin) | Full permissions for all resources and ability to manage workspace | +| [Workspace Editor](#workspace-editor) | Full permissions for most resources, cannot manage workspace settings or delete certain resources | +| [Workspace Viewer](#workspace-viewer) | Read-only access to all workspace resources | + + +RBAC (Role-Based Access Control) is a feature that is only available to [Enterprise](https://langchain.com/pricing) customers. If you are interested in this feature, [contact our sales team](https://www.langchain.com/contact-sales). Other plans default to using the Admin role for all users. + + +#### Workspace Admin + +**Description**: Role with full permissions for all resources and ability to manage workspace. + +**Permissions**: +- All create, read, update, delete, and share permissions for all resource types +- Workspace management capabilities + + + +#### Workspace Editor + +**Description**: Role with full permissions for most resources. Cannot manage workspace settings or delete certain critical resources. + +**Key Differences from Admin**: +- Cannot delete [runs](/langsmith/observability#runs) +- Cannot manage workspace settings (add/remove members, change workspace name, etc.) + +#### Workspace Viewer + +**Description**: Read-only access to all workspace resources. + +**Permissions**: Read-only access to all resource types. + + + + +For step-by-step instructions on assigning workspace roles to users, refer to the [User Management guide](/langsmith/user-management#assign-a-role-to-a-user). + + +## Custom roles + +Creating custom roles is available for organizations on the Enterprise plan. + +[Organization Admins](#organization-admin) can create custom roles with specific combinations of permissions tailored to their organization's needs. + +### Creating custom roles + +Custom roles are created at the [organization](/langsmith/administration-overview#organizations) level and can be assigned to users in any [workspace](/langsmith/administration-overview#workspaces) within that organization. + +**Steps**: +1. Navigate to Organization **Settings** > **Roles**. +2. Click **Create Custom Role**. +3. Select the permissions to include in the role. +4. Assign the custom role to users in specific workspaces. + +For details on which specific permissions are required for each operation, refer to the [Organization and workspace operations reference](/langsmith/organization-workspace-operations). + +Note the following details on custom roles: + +- Custom roles can only be created and managed by Organization Admins. +- Custom roles are organization-specific (not transferable between organizations). +- Each custom role can have any combination of workspace-level permissions. +- Custom roles cannot have organization-level permissions. +- Users can have different roles (including custom roles) in different workspaces. diff --git a/src/langsmith/user-management.mdx b/src/langsmith/user-management.mdx index d8594f7841..54187c06ce 100644 --- a/src/langsmith/user-management.mdx +++ b/src/langsmith/user-management.mdx @@ -21,6 +21,8 @@ You may find it helpful to read the [Administration overview](/langsmith/adminis LangSmith relies on RBAC to manage user permissions within a [workspace](/langsmith/administration-overview#workspaces). This allows you to control who can access your LangSmith workspace and what they can do within it. Only users with the `workspace:manage` permission can manage access control settings for a workspace. +For a complete reference of workspace roles and their permissions, refer to the [Role-based access control](/langsmith/rbac#workspace-roles) guide. For specific operations each role can perform, refer to the [Organization and workspace operations reference](/langsmith/organization-workspace-operations). + ### Create a role By default, LangSmith comes with a set of system roles: diff --git a/src/snippets/langsmith/multi-workspace-org-roles.mdx b/src/snippets/langsmith/multi-workspace-org-roles.mdx new file mode 100644 index 0000000000..760327f40c --- /dev/null +++ b/src/snippets/langsmith/multi-workspace-org-roles.mdx @@ -0,0 +1 @@ +The Organization User and Organization Viewer roles are only available in organizations on [plans](https://langchain.com/pricing) with multiple workspaces. In organizations limited to a single workspace, all users have the Organization Admin role. diff --git a/src/snippets/langsmith/permissions-reference.mdx b/src/snippets/langsmith/permissions-reference.mdx new file mode 100644 index 0000000000..9d09795d66 --- /dev/null +++ b/src/snippets/langsmith/permissions-reference.mdx @@ -0,0 +1 @@ +For a comprehensive list of required permissions along with the operations and roles that can perform them, refer to the [Organization and workspace reference](/langsmith/organization-workspace-operations).