projects: update tools to support pagination, improve prompts

[tmelliottjr]

This PR updates the tool/argument descriptions for projects related tools to better enable tool selection and project item filter.

Additionally, this PR introduces pagination for list_* operations to enable effective querying.

[Copilot]

Pull Request Overview

This PR enhances the GitHub Projects V2 integration by adding advanced pagination support, improving response structures, refining query capabilities, and updating test coverage to match the new implementation. The changes focus on providing more robust pagination handling and clearer field filtering capabilities.

Key changes:

• Added cursor-based pagination support (after, before parameters) with pageInfo in responses
• Changed response format to include pageInfo metadata alongside data arrays
• Updated field filtering to exclude special/system data types from responses
• Refactored field ID handling from array to comma-separated string format

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

File	Description
pkg/github/projects.go	Core implementation: adds pagination options extraction, pageInfo builder, filterSpecialTypes function, updated response structures with pageInfo, and comprehensive query documentation
pkg/github/projects_test.go	Test updates: modified mock data to include required fields (node_id), updated assertions to validate new response structure with pageInfo, simplified field query parameter handling
pkg/github/toolsnaps/*.snap	Snapshot updates: documents new pagination parameters and updated descriptions for tool schemas
README.md	Documentation updates: adds pagination parameters and extensive query syntax documentation for projects tools

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kerobbi]

The technical improvements here lgtm! The only issues I can see are related to descriptions/prompts being quite verbose, and overly focused on technical details.

I have added a couple of comments just for better clarity, although they are all related to the same root concern.

[kerobbi]

Left some comments mainly regarding reducing/optimising context usage and avoiding duplicate instructions.

I also suggested removing a lot of the query related instructions, as in general we've been finding that the model is able to construct pretty good queries using the GitHub syntax. I am curious, have you found that the model has not been able to construct queries well? If so, was it happening even for simple queries or for more complex projects ones?

[kerobbi]

This section (lines 123 - 156) on query construction is quite long. What I would suggest doing here is just keeping anything which is projects specific or that we think that the model does not know already, and removing the rest.

We do have a query parameter in other tools (e.g., search_pull_requests) that could be used as a reference in case you want to update the list_project_items query param. In general the model is quite good at constructing queries so it probably won't need this level of guidance.

If we were to consider adding guidance on search syntax in the future, that would apply across the entire server as a whole, not just projects tools, so it wouldn't live in these projects specific instructions.

[tmelliottjr]

I think this is worth validating again, but I did not have much success at all without introducing these query rules for projects. Projects search is also a combination of some issues search capabilities and project specific rules for interacting with items and their fields.

I also found, while working through enabling this for chat, that in its current form, chat failed spectacularly at using the tools at the right time and effectively didn't apply any filters.

[kerobbi]

chat failed spectacularly at using the tools at the right time

Having some server instructions to cover the ways the tools can/should be used together, i.e., workflows, should definitely help with this, although without running evals it's pretty much impossible to concretely determine if these changes would have any significant impact. We already have a couple of examples for other toolsets.

effectively didn't apply any filters

A bit hard to pinpoint which descriptions/instructions could be leading to this, as even a couple of words could make a huge difference, but just trying to take a step back here. Could the following base server instructions be a contributing factor? I'm wondering whether adding one or two sentences as part of the server instructions for projects, specific to the list_projects and list_projects_items, highlighting that we do not just want "basic filtering" can improve query building.

Another thought

If the somewhat right tool calls do happen, but the provided query filters are irrelevant or wrong, could it be that the model(s) is understanding or trying to do something entirely different? Is the difference between Projects (i.e., (the project management feature) and, for instance, repositories, clear enough to the models given the current descriptions and server instructions?

This is just me assuming that everything needed to query/filter projects is available as part of this documentation, and if that is the case, the model should already be aware of how to build queries.

[tmelliottjr]

These instructions effectively only exist as part of the user/assistant flow which seems to work well enough for say, chat in VSCode. They do not appear to have much effect in dotcom chat.

I have tried running evals locally, but without success. I'll need to try again next week. I also tried to see if there were any ways to run evals with custom branches, but that didn't seem possible either.

In regards to the docs availability, how can I confirm that it is using those docs in context during any given tool call?

[kerobbi]

Out of curiosity, have you found that the model used these forbidden filters or are you adding them here as a precaution?

[tmelliottjr]

It didn't happen all the time, but it certainly has tried crafting queries that aren't valid in this context.

[tmelliottjr]

I've removed the forbidden params and left only those permitted and examples.

[kerobbi]

Suggested change

	mcp.Description(`Query string for advanced filtering of project items. See Projects server instructions (list_project_items query rules) for full construction heuristics, syntax essentials, qualifier glossary, pagination mandate, recovery guidance, and prohibited behaviors.`),
	mcp.Description(`Query string for advanced filtering of project items using GitHub search syntax.`),

[kerobbi]

Just a suggestion based on my comment above.

[kerobbi]

Suggested change

	mcp.Description("Field IDs to include (e.g. [\"102589\", \"985201\"]). CRITICAL: Always provide to get field values. Without this, only titles returned. Get IDs from list_project_fields first."),
	mcp.Description("Field IDs to include (e.g. [\"102589\", \"985201\"]). CRITICAL: Always provide to get field values. Without this, only titles returned."),

[kerobbi]

I'd reconsider if Get IDs from list_project_fields first. is really needed, as this is something we already say in the server instructions (i.e., Field usage section).

[stephenotalora]

Thanks, @kerobbi! I appreciate the feedback, @tmelliottjr is out until Monday, but I’m reviewing your comments in the meantime and will address them accordingly 🙇🏼

[stephenotalora]

@tmelliottjr just an fyi, recent push is to address a number of conflicts with main