You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
description: Learn how to implement an AI-powered triage system that automatically evaluates Jira tickets, enriches them with missing context, and ensures only well-defined tasks reach your coding agents.
4
4
---
5
5
6
-
# Triage tickets to coding agents
6
+
import MCPCapabilitiesHint from '/docs/guides/templates/ai/_mcp_capabilities_hint.mdx'
7
+
8
+
9
+
# Improve specifications with Port AI
7
10
8
11
Coding agents often fail or produce suboptimal results when given incomplete or poorly defined tasks. Developers waste time and resources on unsuccessful runs caused by missing context, unclear success criteria, or insufficient requirements.
9
12
10
13
This guide demonstrates how to implement an AI-powered triage system in Port that automatically evaluates incoming Jira tickets, enriches them with missing details, and ensures only well-defined tasks reach your coding agents.
@@ -33,9 +36,71 @@ While this guide uses Jira for ticket management, you can adapt it for other pro
33
36
34
37
We will configure the necessary blueprints and scorecards to support our AI triage workflow.
35
38
36
-
### Create Jira ticket blueprint
39
+
### Create Feature blueprint
40
+
41
+
To implement the AI triage system, we need a blueprint to store different features (tickets) from your project management tool. This blueprint will track the AI readiness status, triage stages, and enrichment suggestions for each feature.
37
42
38
-
First, we need to create a Jira ticket blueprint that includes properties for tracking AI readiness and triage status.
43
+
:::info If you already have Jira integration installed
44
+
Port's Jira integration creates a `jiraIssue` blueprint by default. You can extend this existing blueprint by adding the following properties to enable AI triage functionality:
45
+
46
+
1. Go to the [builder](https://app.getport.io/settings/data-model) page of your portal.
47
+
2. Select the `Jira Issue` blueprint.
48
+
3. Click on the `{...} Edit JSON` button.
49
+
4. Copy and paste the following JSON configuration under the `properties` schema:
50
+
51
+
<details>
52
+
<summary><b>Extended Jira ticket blueprint (Click to expand)</b></summary>
53
+
54
+
```json showLineNumbers
55
+
"description": {
56
+
"type": "string",
57
+
"title": "Description",
58
+
"description": "Detailed description of the issue",
59
+
"format": "markdown"
60
+
},
61
+
"current_stage": {
62
+
"icon": "DefaultProperty",
63
+
"title": "Triage Stage",
64
+
"description": "Current triage stage of the issue",
65
+
"type": "string",
66
+
"default": "Not started",
67
+
"enum": [
68
+
"Awaiting approval",
69
+
"Not started",
70
+
"Draft",
71
+
"Approved"
72
+
],
73
+
"enumColors": {
74
+
"Awaiting approval": "yellow",
75
+
"Not started": "lightGray",
76
+
"Draft": "orange",
77
+
"Approved": "green"
78
+
}
79
+
},
80
+
"confidence_score": {
81
+
"type": "number",
82
+
"title": "Confidence Score",
83
+
"description": "AI's assessment of readiness for coding agent (0-100)",
84
+
"minimum": 0,
85
+
"maximum": 100
86
+
},
87
+
"ai_suggested_description": {
88
+
"type": "string",
89
+
"title": "AI Suggested Description",
90
+
"description": "AI-generated suggestions for improving the ticket",
91
+
"format": "markdown"
92
+
},
93
+
"prioritized": {
94
+
"type": "boolean",
95
+
"title": "Prioritized"
96
+
}
97
+
```
98
+
</details>
99
+
100
+
The rest of this guide will reference `jira_ticket` as the blueprint identifier.
101
+
:::
102
+
103
+
If you don't have Jira integration or want to create a custom blueprint, follow these steps:
39
104
40
105
1. Go to the [builder](https://app.getport.io/settings/data-model) page of your portal.
41
106
2. Click on `+ Blueprint`.
@@ -48,7 +113,7 @@ First, we need to create a Jira ticket blueprint that includes properties for tr
48
113
```json showLineNumbers
49
114
{
50
115
"identifier": "jira_ticket",
51
-
"title": "Ticket",
116
+
"title": "Feature",
52
117
"icon": "Jira",
53
118
"schema": {
54
119
"properties": {
@@ -161,15 +226,20 @@ First, we need to create a Jira ticket blueprint that includes properties for tr
161
226
```
162
227
</details>
163
228
164
-
5. Click `Create` to save the blueprint
229
+
5. Click `Create` to save the blueprint.
165
230
166
231
### Create AI readiness scorecard
167
232
168
-
Next, we will create a scorecard that evaluates whether tickets are ready for coding agents based on completeness criteria.
233
+
Next, we will create a scorecard that evaluates whether tickets are ready for coding agents based on completeness criteria. This ensures that only well-defined and complete tasks reach your AI coding assistants, preventing wasted compute resources and failed runs.
169
234
170
-
1. Open the newly created `Ticket`
171
-
2. Click on `+ Scorecard`
172
-
3. Click on the `{...} Edit JSON` button
235
+
The scorecard evaluates tickets based on three key readiness criteria:
236
+
- Whether the ticket has been prioritized for development.
237
+
- Current stage status (must be approved).
238
+
- Assignment status (requires an assignee).
239
+
240
+
1. Open the newly created `Feature` blueprint.
241
+
2. Click on `+ Scorecard`.
242
+
3. Click on the `{...} Edit JSON` button.
173
243
4. Copy and paste the following JSON configuration:
174
244
175
245
<details>
@@ -222,6 +292,9 @@ Next, we will create a scorecard that evaluates whether tickets are ready for co
We will create three self-service actions to handle the triage workflow: improving tickets with AI suggestions, approving suggestions, and requesting refinements.
@@ -230,9 +303,9 @@ We will create three self-service actions to handle the triage workflow: improvi
230
303
231
304
This action allows the AI agent to suggest improvements to ticket descriptions.
232
305
233
-
1. Go to the [self-service](https://app.getport.io/self-serve) page of your portal
234
-
2. Click on `+ New Action`
235
-
3. Click on the `{...} Edit JSON` button
306
+
1. Go to the [self-service](https://app.getport.io/self-serve) page of your portal.
307
+
2. Click on `+ New Action`.
308
+
3. Click on the `{...} Edit JSON` button.
236
309
4. Copy and paste the following JSON configuration:
237
310
238
311
<details>
@@ -315,15 +388,15 @@ This action allows the AI agent to suggest improvements to ticket descriptions.
315
388
```
316
389
</details>
317
390
318
-
5. Click `Save` to create the action
391
+
5. Click `Save` to create the action.
319
392
320
393
### Create approval action
321
394
322
395
This action allows human reviewers to approve or reject AI suggestions.
323
396
324
-
1. Go back to the [self-service](https://app.getport.io/self-serve) page of your portal
325
-
2. Click on `+ New Action`
326
-
3. Click on the `{...} Edit JSON` button
397
+
1. Go back to the [self-service](https://app.getport.io/self-serve) page of your portal.
398
+
2. Click on `+ New Action`.
399
+
3. Click on the `{...} Edit JSON` button.
327
400
4. Copy and paste the following JSON configuration:
328
401
329
402
<details>
@@ -385,15 +458,15 @@ This action allows human reviewers to approve or reject AI suggestions.
385
458
```
386
459
</details>
387
460
388
-
5. Click `Save` to create the action
461
+
5. Click `Save` to create the action.
389
462
390
463
### Create retry action
391
464
392
465
This action allows users to provide feedback and request AI to refine its suggestions.
393
466
394
-
1. Go back to the [self-service](https://app.getport.io/self-serve) page of your portal
395
-
2. Click on `+ New Action`
396
-
3. Click on the `{...} Edit JSON` button
467
+
1. Go back to the [self-service](https://app.getport.io/self-serve) page of your portal.
468
+
2. Click on `+ New Action`.
469
+
3. Click on the `{...} Edit JSON` button.
397
470
4. Copy and paste the following JSON configuration:
398
471
399
472
<details>
@@ -455,15 +528,15 @@ This action allows users to provide feedback and request AI to refine its sugges
455
528
```
456
529
</details>
457
530
458
-
5. Click `Save` to create the action
531
+
5. Click `Save` to create the action.
459
532
460
533
## Create AI agent
461
534
462
-
Now we'll create the AI agent that evaluates tickets and suggests improvements.
535
+
Now we will create the AI agent that evaluates tickets and suggests improvements.
463
536
464
-
1. Go to the [AI Agents](https://app.getport.io/_ai_agents) page of your portal
465
-
2. Click on `+ AI Agent`
466
-
3. Toggle `Json mode` on
537
+
1. Go to the [AI Agents](https://app.getport.io/_ai_agents) page of your portal.
538
+
2. Click on `+ AI Agent`.
539
+
3. Toggle `Json mode` on.
467
540
4. Copy and paste the following JSON schema:
468
541
469
542
<details>
@@ -490,25 +563,23 @@ Now we'll create the AI agent that evaluates tickets and suggests improvements.
490
563
```
491
564
</details>
492
565
493
-
5. Click `Create` to save the agent
566
+
5. Click `Create` to save the agent.
494
567
495
-
:::tip MCP Enhanced Capabilities
496
-
The AI agent uses MCP (Model Context Protocol) enhanced capabilities to automatically discover important and relevant blueprint entities via its tools. The `^(list|get|search|track|describe)_.*` pattern allows the agent to access and analyze related entities in your software catalog, providing richer context for ticket evaluation. Additionally, we explicitly add `run_ask_ai_to_improve_on_ticket` to the tools, which instructs the AI agent to call this specific action to update tickets with AI suggestions.
497
-
:::
568
+
<MCPCapabilitiesHint/>
498
569
499
570
## Set up automations
500
571
501
572
We'll create two automations to orchestrate the AI triage workflow:
502
573
503
-
1. Trigger the AI agent when new tickets are created or synced
504
-
2. Automatically assign coding agents to approved tickets
574
+
1. Trigger the AI agent when new tickets are created or synced.
575
+
2. Automatically assign coding agents to approved tickets.
505
576
506
577
### Create ticket sync trigger automation
507
578
508
579
This automation triggers the AI triage agent whenever a new Jira ticket is created or synced to the Catalog.
509
580
510
-
1. Go to the [automations](https://app.getport.io/settings/automations) page of your portal
511
-
2. Click on `+ Automation`
581
+
1. Go to the [automations](https://app.getport.io/settings/automations) page of your portal.
582
+
2. Click on `+ Automation`.
512
583
3. Copy and paste the following JSON schema:
513
584
514
585
<details>
@@ -555,14 +626,14 @@ This automation triggers the AI triage agent whenever a new Jira ticket is creat
555
626
```
556
627
</details>
557
628
558
-
4. Click `Create` to save the automation
629
+
4. Click `Create` to save the automation.
559
630
560
631
### Create coding agent assignment automation
561
632
562
633
This automation automatically assigns coding agents to tickets once they're approved and ready.
563
634
564
-
1. Go back to the [automations](https://app.getport.io/settings/automations) page of your portal
565
-
2. Click on `+ Automation`
635
+
1. Go back to the [automations](https://app.getport.io/settings/automations) page of your portal.
636
+
2. Click on `+ Automation`.
566
637
3. Copy and paste the following JSON schema:
567
638
568
639
<details>
@@ -590,7 +661,7 @@ This automation automatically assigns coding agents to tickets once they're appr
@@ -614,11 +685,11 @@ This automation automatically assigns coding agents to tickets once they're appr
614
685
Replace the **service** with the name of the Github repository. Also, replace the webhook URL and payload structure in the automation to match your preferred coding agent's API. You can integrate with Claude Code, GitHub Copilot, Devin, or any other AI coding assistant that accepts webhook triggers.
615
686
:::
616
687
617
-
4. Click `Create` to save the automation
688
+
4. Click `Create` to save the automation.
618
689
619
690
## Test the workflow
620
691
621
-
Now let's test the complete AI triage workflow to ensure everything works correctly.
692
+
Now let us test the complete AI triage workflow to ensure everything works correctly.
The AI agent uses MCP (Model Context Protocol) enhanced capabilities to automatically discover important and relevant blueprint entities via its tools. The `^(list|get|search|track|describe)_.*` pattern allows the agent to access and analyze related entities in your software catalog, providing richer context for ticket evaluation.
description: "Learn how to implement an AI-powered triage system that automatically evaluates tickets, enriches them with missing context, and ensures only well-defined tasks reach your coding agents",
0 commit comments