-
Notifications
You must be signed in to change notification settings - Fork 2.6k
feat: enable MCP tools to handle base64 image uploads #9806
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
- Add image file path detection in UseMcpToolTool - Convert image file paths to base64 data URLs automatically - Support nested image paths in complex argument objects - Add comprehensive tests for image handling scenarios - Preserve non-image paths and already-encoded base64 data Fixes #9805
Review complete. Found 3 issues that need attention:
Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues. |
| // Check if the file exists | ||
| try { | ||
| const stats = await fs.stat(value) | ||
| return stats.isFile() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Relative image paths won't be resolved correctly. The fs.stat(value) call on line 61 uses the path as-is without resolving it relative to task.cwd. This means relative paths like "./images/photo.png" will fail the file existence check even if the file exists, preventing conversion to base64. Other tools like ReadFileTool resolve paths with path.resolve(task.cwd, relPath) before checking file existence.
Fix it with Roo Code or mention @roomote and request a fix.
| // Convert any image file paths in arguments to base64 data URLs | ||
| const processedArguments = parsedArguments | ||
| ? ((await this.convertImagePathsToBase64(parsedArguments)) as Record<string, unknown>) | ||
| : parsedArguments |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The isImageFilePath and convertImagePathsToBase64 methods need access to task.cwd to properly resolve relative paths, but they don't receive it as a parameter. To fix the path resolution issue, these methods should accept cwd: string as a parameter so they can resolve paths like path.resolve(cwd, value) before calling fs.stat().
Fix it with Roo Code or mention @roomote and request a fix.
| } catch (error) { | ||
| // If we can't read the image, return the original value | ||
| console.error(`Failed to convert image file to base64: ${error}`) | ||
| return obj | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Image conversion errors are only logged to console and silently fall back to the original value. Users won't be notified when image conversion fails. Consider using await task.say("error", ...) to report conversion failures to the user, similar to how ReadFileTool handles image errors on line 382.
Fix it with Roo Code or mention @roomote and request a fix.
This PR attempts to address Issue #9805. Feedback and guidance are welcome!
Problem
When invoking an MCP tool with an image parameter, the system was passing file paths instead of base64-encoded image data, preventing MCP servers from properly receiving and processing image inputs.
Solution
Modified
UseMcpToolToolto automatically detect and convert image file paths to base64 data URLs before passing arguments to MCP servers.Changes
UseMcpToolTool.tsImplementation Details
readImageAsDataUrlWithBufferhelper fromimageHelpers.tsTesting
Fixes #9805