-
Notifications
You must be signed in to change notification settings - Fork 4
Commit b41af0f
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?v=4&size=40){kind=avatar}
Add Deepnote Environments kernel management (#152)
* feat: implement Phase 1 of Deepnote kernel configuration management
Add core infrastructure for managing Deepnote kernel configurations,
enabling future user-controlled kernel lifecycle management.
## What's New
### Core Models & Storage
- Create type definitions for kernel configurations with UUID-based identity
- Implement persistent storage layer using VS Code globalState
- Build configuration manager with full CRUD operations
- Add event system for configuration change notifications
### Components Added
- `deepnoteKernelConfiguration.ts`: Type definitions and interfaces
- `deepnoteConfigurationStorage.ts`: Serialization and persistence
- `deepnoteConfigurationManager.ts`: Business logic and lifecycle management
### API Updates
- Extend `IDeepnoteToolkitInstaller` with configuration-based methods
- Extend `IDeepnoteServerStarter` with configuration-based methods
- Maintain backward compatibility with file-based APIs
### Service Registration
- Register DeepnoteConfigurationStorage as singleton
- Register DeepnoteConfigurationManager with auto-activation
- Integrate with existing dependency injection system
## Testing
- Add comprehensive unit tests for storage layer (11 tests, all passing)
- Add unit tests for configuration manager (29 tests, 22 passing)
- 7 tests intentionally failing pending Phase 2 service refactoring
## Documentation
- Create KERNEL_MANAGEMENT_VIEW_IMPLEMENTATION.md with complete architecture
- Document 8-phase implementation plan
- Define migration strategy from file-based to configuration-based system
## Dependencies
- Add uuid package for configuration ID generation
## Status
Phase 1 complete. Ready for Phase 2 (refactoring existing services).
Related: #4913
* feat: add configuration-based APIs to DeepnoteToolkitInstaller and DeepnoteServerStarter
Refactor DeepnoteToolkitInstaller and DeepnoteServerStarter to support
configuration-based kernel management alongside existing file-based workflow.
DeepnoteToolkitInstaller changes:
- Add ensureVenvAndToolkit() method for direct venv path management
- Add installAdditionalPackages() for installing packages in existing venvs
- Add getVenvInterpreterByPath() helper for venv path-based interpreter lookup
- Add getKernelSpecName() and getKernelDisplayName() for venv-based naming
- Refactor installImpl() to installVenvAndToolkit() using venv paths
- Update kernel spec installation to use venv directory names instead of file hashes
- Mark ensureInstalled() as deprecated, now delegates to ensureVenvAndToolkit()
DeepnoteServerStarter changes:
- Add startServer() method accepting venv path and configuration ID
- Add stopServer() method accepting configuration ID instead of file URI
- Add startServerForConfiguration() implementation for config-based lifecycle
- Add stopServerForConfiguration() implementation for config-based cleanup
- Mark getOrStartServer() as deprecated for backward compatibility
- Update server process tracking to work with configuration IDs as keys
These changes enable the kernel configuration manager to create and manage
isolated Python environments without requiring .deepnote file associations.
Legacy file-based methods are preserved for backward compatibility.
Part of Phase 2: Refactoring Existing Services
* feat: implement Phase 3 - Tree View UI for kernel configurations
Add VS Code tree view interface for managing Deepnote kernel configurations.
Changes:
- Created DeepnoteConfigurationTreeItem with status-based icons and context values
- Created DeepnoteConfigurationTreeDataProvider implementing TreeDataProvider
- Created DeepnoteConfigurationsView handling all UI commands:
* create: Multi-step wizard for new configurations
* start/stop/restart: Server lifecycle management
* delete: Configuration removal with confirmation
* editName: Rename configurations
* managePackages: Update package lists
* refresh: Manual tree refresh
- Created DeepnoteConfigurationsActivationService for initialization
- Registered all services in serviceRegistry.node.ts
- Added deepnoteKernelConfigurations view to package.json
- Added 8 commands with icons and context menus
- Added view/title and view/item/context menu contributions
Technical details:
- Uses Python API to enumerate available interpreters
- Implements progress notifications for long-running operations
- Provides input validation for names and package lists
- Shows status indicators (Running/Starting/Stopped) with appropriate colors
- Displays configuration details (Python path, venv, packages, timestamps)
Fixes:
- Made DeepnoteConfigurationManager.initialize() public to match interface
- Removed unused getDisplayName() method from DeepnoteToolkitInstaller
- Added type annotations to all lambda parameters
* feat: implement Phase 3 - Tree View UI for kernel configurations
Add VS Code tree view interface for managing Deepnote kernel configurations.
Changes:
- Created DeepnoteConfigurationTreeItem with status-based icons and context values
- Created DeepnoteConfigurationTreeDataProvider implementing TreeDataProvider
- Created DeepnoteConfigurationsView handling all UI commands:
* create: Multi-step wizard for new configurations
* start/stop/restart: Server lifecycle management
* delete: Configuration removal with confirmation
* editName: Rename configurations
* managePackages: Update package lists
* refresh: Manual tree refresh
- Created DeepnoteConfigurationsActivationService for initialization
- Registered all services in serviceRegistry.node.ts
- Added deepnoteKernelConfigurations view to package.json
- Added 8 commands with icons and context menus
- Added view/title and view/item/context menu contributions
Technical details:
- Uses Python API to enumerate available interpreters
- Implements progress notifications for long-running operations
- Provides input validation for names and package lists
- Shows status indicators (Running/Starting/Stopped) with appropriate colors
- Displays configuration details (Python path, venv, packages, timestamps)
Fixes:
- Made DeepnoteConfigurationManager.initialize() public to match interface
- Removed unused getDisplayName() method from DeepnoteToolkitInstaller
- Added type annotations to all lambda parameters
* feat: add Phase 7 Part 1 - configuration picker infrastructure
Add infrastructure for notebook-to-configuration mapping and selection UI.
Changes:
- Created DeepnoteConfigurationPicker for showing configuration selection UI
- Created DeepnoteNotebookConfigurationMapper for tracking notebook→config mappings
- Added interfaces to types.ts for new services
- Registered new services in serviceRegistry.node.ts
- Updated KERNEL_MANAGEMENT_VIEW_IMPLEMENTATION.md with:
* New components documentation (picker and mapper)
* Updated file structure with implementation status
* Comprehensive implementation status section
Technical details:
- Picker shows configurations with status indicators (running/stopped)
- Picker includes "Create new" option
- Mapper stores selections in workspace state (per-workspace persistence)
- Mapper provides bidirectional lookups (notebook→config and config→notebooks)
Next steps:
- Integrate picker and mapper with DeepnoteKernelAutoSelector
- Show picker when notebook opens without selected configuration
- Use selected configuration's venv and server instead of auto-creating
* refactor: rename "Configuration" to "Environment" for clarity
This refactoring improves terminology clarity across the Deepnote kernel
management system by renaming "Kernel Configuration" to "Environment".
Rationale:
- "Configuration" implies kernel settings, but the system manages Python
virtual environments (venv + packages + Jupyter server)
- "Environment" is more accurate and familiar to developers
- Reduces confusion between "configuration" (settings) and "environment"
(Python venv)
Changes:
- Renamed directory: configurations/ → environments/
- Renamed 15 files (classes, interfaces, tests)
- Updated types.ts: 6 interface names, 6 symbol constants
- Updated package.json: 9 commands, 1 view ID, titles/labels
- Updated all import paths and references across codebase
- Updated documentation in KERNEL_MANAGEMENT_VIEW_IMPLEMENTATION.md
Terminology mapping:
- Kernel Configuration → Environment
- IDeepnoteConfigurationManager → IDeepnoteEnvironmentManager
- DeepnoteConfigurationsView → DeepnoteEnvironmentsView
- deepnote.configurations.* → deepnote.environments.*
- deepnoteKernelConfigurations view → deepnoteEnvironments view
All tests pass, TypeScript compilation successful.
* Fixed tests
* Ensure propsed APIs are loaded properly
this was leading to many error logs in the "Jupyter Outputs"
* Improve DX by auto showing "Deepnote" outputs
* feat: add environment selection UI for notebooks
Added comprehensive UI for selecting and switching environments for Deepnote
notebooks, making it easy for users to choose which kernel environment to use.
New Features:
- Added "Select Environment for Notebook" command in notebook toolbar that
shows a quick pick with all available environments
- Environment picker now directly triggers the create environment dialog when
"Create New" is selected, then automatically re-shows the picker
- Environment names are now displayed in kernel connection labels for better
visibility (e.g., "Deepnote: Python 3.10")
- Shows current environment selection with checkmark in the picker
- Displays environment status (Running/Stopped) with icons in picker
- Warns users before switching if cells are currently executing
Improvements:
- Environment picker integration: Clicking "Create New" now launches the full
creation dialog instead of just showing an info message
- Added rebuildController() to IDeepnoteKernelAutoSelector interface for
switching environments
- Updated test mocks to include new dependencies
UI/UX:
- Added notebook toolbar button with server-environment icon
- Quick pick shows environment details: interpreter path, packages, status
- Graceful handling of edge cases (no environments, same environment selected)
- Clear progress notifications during environment switching
* fix: resolve environment switching issues with stale server connections
Fixed three critical bugs that prevented proper environment switching:
1. Controller disposal race condition: Old controller was disposed before
the new controller was ready, causing "DISPOSED" errors during cell
execution. Fixed by deferring disposal until after new controller is
fully registered.
2. Stale configuration caching: Configuration object wasn't refreshed after
startServer(), so we connected with outdated serverInfo. Fixed by always
calling getEnvironment() after startServer() to get current server info.
3. Environment manager early return: startServer() had an early return when
config.serverInfo was set, preventing verification that the server was
actually running. This caused connections to wrong/stale servers when
switching TO a previously-used environment. Fixed by always calling
serverStarter.startServer() (which is idempotent) to ensure current info.
Additional improvements:
- Made kernel spec installation idempotent and ensured it runs when reusing
existing venvs
- Removed legacy auto-create fallback path that's no longer needed
- Added proper TypeScript non-null assertions after server info validation
* WIP: Environment switching with controller disposal workaround
This commit implements environment switching but with a known limitation
regarding controller disposal and queued cell executions.
## Changes
### Port Allocation Refactoring
- Refactored DeepnoteServerProvider to handle both jupyterPort and lspPort
- Updated DeepnoteServerStarter to allocate both ports
- Updated DeepnoteEnvironmentManager to store complete serverInfo
- All port-related code now consistently uses the dual-port model
### Kernel Selection Logic Extraction
- Extracted kernel selection logic into public `selectKernelSpec()` method
- Added 4 unit tests for kernel selection (all passing)
- Method is now testable and validates environment-specific kernel preference
- Falls back to generic Python kernels when env-specific kernel not found
### Environment Switching Implementation
- Added environment switching via tree view context menu
- Switching calls `rebuildController()` to create new controller
- Shows warning dialog if cells are currently executing
- Updates notebook-to-environment mapping on switch
## Known Issue: Controller Disposal Problem
When switching environments, we encountered a critical "notebook controller is
DISPOSED" error. The issue occurs because:
1. User queues cell execution (VS Code references current controller)
2. User switches environments
3. New controller created and marked as preferred
4. Old controller disposed
5. Queued execution tries to run 5+ seconds later → DISPOSED error
### Current Workaround (Implemented)
We do NOT dispose old controllers at all. Instead:
- Old controllers stay alive to handle queued executions
- New controller is marked as "Preferred" for new executions
- Garbage collection cleans up eventually
### Why This Is Not Ideal
- Potential memory leak with many switches
- No guarantee new executions use new controller
- Users might execute on wrong environment after switch
- VS Code controller selection not fully deterministic
### What We Tried
1. Adding delay before disposal → Failed (timing unpredictable)
2. Disposing after setting preference → Failed
3. Never disposing → Prevents error but suboptimal
### Proper Solution Needed
May require VS Code API changes to:
- Force controller selection immediately
- Cancel/migrate queued executions
- Query if controller has pending executions
See TODO.md for full analysis and discussion.
## Testing
- ✅ All 40+ unit tests passing
- ✅ Kernel selection tests passing (4 new tests)
- ✅ Port allocation working correctly
- 1 parent 07f9044 commit b41af0fCopy full SHA for b41af0f
File tree
Expand file treeCollapse file tree
54 files changed
+7390
-902
lines changedOpen diff view settings
Filter options
- .vscode
- src
- kernels
- deepnote
- environments
- notebooks
- controllers
- deepnote
- integrations
- platform
- common
- application
- utils
- deepnote
- interpreter/display
- logging
- standalone
- context
- import-export
- test
- webviews/extension-side/variablesView
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Expand file treeCollapse file tree
54 files changed
+7390
-902
lines changedOpen diff view settings
Collapse file
+1-1Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
| 12 | + | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| |||
Collapse file
+12-1Lines changed: 12 additions & 1 deletion
- Display the source diff
- Display the rich diff
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
| 2 | + | |
2 | 3 | | |
3 | 4 | | |
4 | 5 | | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
5 | 12 | | |
6 | 13 | | |
| 14 | + | |
7 | 15 | | |
8 | 16 | | |
9 | 17 | | |
10 | 18 | | |
11 | 19 | | |
12 | 20 | | |
13 | 21 | | |
| 22 | + | |
14 | 23 | | |
| 24 | + | |
15 | 25 | | |
16 | 26 | | |
17 | 27 | | |
18 | 28 | | |
19 | 29 | | |
20 | 30 | | |
| 31 | + | |
21 | 32 | | |
22 | 33 | | |
23 | 34 | | |
| |||
28 | 39 | | |
29 | 40 | | |
30 | 41 | | |
31 | | - | |
| 42 | + | |
0 commit comments