Improve specifications with Port AI

Coding agents often fail or produce suboptimal results when given incomplete or poorly defined tasks. Developers waste time and resources on unsuccessful runs caused by missing context, unclear success criteria, or insufficient requirements.

This guide demonstrates how to implement an AI-powered triage system in Port that automatically evaluates incoming Jira tickets, enriches them with missing details, and ensures only well-defined tasks reach your coding agents.

Common use cases

Prevent coding agent failures by ensuring tickets have sufficient context before assignment.
Streamline human review process with clear approval workflows for AI suggestions.
Reduce development bottlenecks by catching poorly defined tasks early in the process.

Prerequisites

This guide assumes the following:

You have a Port account and have completed the onboarding process.
Port's Jira integration is installed in your account.
You have access to create and configure AI agents in Port.
You have a coding agent setup (like Claude Code, GitHub Copilot, or similar) that can be triggered via webhook.

Alternative integrations and coding agents

While this guide uses Jira for ticket management, you can adapt it for other project management tools like Linear, Asana, or Azure DevOps. Additionally, you can integrate with various coding agents including Claude Code, GitHub Copilot, Devin, or any other AI coding assistant that accepts webhook triggers.

Set up data model

We will configure the necessary blueprints and scorecards to support our AI triage workflow.

Create Feature blueprint

To implement the AI triage system, we need a blueprint to store different features (tickets) from your project management tool. This blueprint will track the AI readiness status, triage stages, and enrichment suggestions for each feature.

If you already have Jira integration installed

Port's Jira integration creates a jiraIssue blueprint by default. You can extend this existing blueprint by adding the following properties to enable AI triage functionality:

Go to the builder page of your portal.
Select the Jira Issue blueprint.
Click on the {...} Edit JSON button.

Copy and paste the following JSON configuration under the properties schema:

Extended Jira ticket blueprint (Click to expand)

      "description": {
        "type": "string",
        "title": "Description",
        "description": "Detailed description of the issue",
        "format": "markdown"
      },
      "current_stage": {
        "icon": "DefaultProperty",
        "title": "Triage Stage",
        "description": "Current triage stage of the issue",
        "type": "string",
        "default": "Not started",
        "enum": [
          "Awaiting approval",
          "Not started",
          "Draft",
          "Approved"
        ],
        "enumColors": {
          "Awaiting approval": "yellow",
          "Not started": "lightGray",
          "Draft": "orange",
          "Approved": "green"
        }
      },
      "confidence_score": {
        "type": "number",
        "title": "Confidence Score",
        "description": "AI's assessment of readiness for coding agent (0-100)",
        "minimum": 0,
        "maximum": 100
      },
      "ai_suggested_description": {
        "type": "string",
        "title": "AI Suggested Description",
        "description": "AI-generated suggestions for improving the ticket",
        "format": "markdown"
      },
      "prioritized": {
        "type": "boolean",
        "title": "Prioritized"
      }

The rest of this guide will reference jira_ticket as the blueprint identifier.

If you don't have Jira integration or want to create a custom blueprint, follow these steps:

Go to the builder page of your portal.
Click on + Blueprint.
Click on the {...} Edit JSON button.

Copy and paste the following JSON configuration:

Jira ticket blueprint (Click to expand)

{
  "identifier": "jira_ticket",
  "title": "Feature",
  "icon": "Jira",
  "schema": {
    "properties": {
      "status": {
        "type": "string",
        "title": "Status",
        "description": "Current status of the Jira issue"
      },
      "issue_type": {
        "type": "string",
        "title": "Issue Type",
        "description": "Type of the Jira issue (Bug, Story, Task, etc.)"
      },
      "priority": {
        "type": "string",
        "title": "Priority",
        "description": "Priority level of the issue"
      },
      "creator": {
        "type": "string",
        "title": "Creator",
        "description": "User who created the issue"
      },
      "assignee": {
        "type": "string",
        "title": "Assignee",
        "description": "User assigned to the issue"
      },
      "manager": {
        "type": "string",
        "title": "Manager",
        "description": "Manager responsible for the issue"
      },
      "description": {
        "type": "string",
        "title": "Description",
        "description": "Detailed description of the issue",
        "format": "markdown"
      },
      "date_created": {
        "type": "string",
        "title": "Date Created",
        "description": "Date when the issue was created",
        "format": "date-time"
      },
      "current_stage": {
        "icon": "DefaultProperty",
        "title": "Triage Stage",
        "description": "Current triage stage of the issue",
        "type": "string",
        "default": "Not started",
        "enum": [
          "Awaiting approval",
          "Not started",
          "Draft",
          "Approved"
        ],
        "enumColors": {
          "Awaiting approval": "yellow",
          "Not started": "lightGray",
          "Draft": "orange",
          "Approved": "green"
        }
      },
      "confidence_score": {
        "type": "number",
        "title": "Confidence Score",
        "description": "AI's assessment of readiness for coding agent (0-100)",
        "minimum": 0,
        "maximum": 100
      },
      "ai_suggested_description": {
        "type": "string",
        "title": "AI Suggested Description",
        "description": "AI-generated suggestions for improving the ticket",
        "format": "markdown"
      },
      "prioritized": {
        "type": "boolean",
        "title": "Prioritized"
      }
    },
    "required": [
      "status",
      "issue_type",
      "priority",
      "creator",
      "description",
      "date_created",
      "current_stage"
    ]
  },
  "mirrorProperties": {},
  "calculationProperties": {},
  "aggregationProperties": {},
  "relations": {
    "jiraProject": {
      "title": "Jira Project",
      "target": "jiraProject",
      "required": true,
      "many": false
    },
    "service": {
      "title": "Service",
      "target": "service",
      "required": false,
      "many": false
    } 
  }
}

Click Create to save the blueprint.

Create AI readiness scorecard

Next, we will create a scorecard that evaluates whether tickets are ready for coding agents based on completeness criteria. This ensures that only well-defined and complete tasks reach your AI coding assistants, preventing wasted compute resources and failed runs.

The scorecard evaluates tickets based on three key readiness criteria:

Whether the ticket has been prioritized for development.
Current stage status (must be approved).
Assignment status (requires an assignee).

Open the newly created Feature blueprint.
Click on + Scorecard.
Click on the {...} Edit JSON button.

Copy and paste the following JSON configuration:

AI readiness scorecard (Click to expand)

{
  "identifier": "ai_readiness_stage",
  "title": "AI Readiness Stage",
  "rules": [
    {
      "identifier": "readiness_status",
      "level": "Ready for coding agent",
      "query": {
        "combinator": "and",
        "conditions": [
          {
            "operator": "=",
            "property": "prioritized",
            "value": true
          },
          {
            "operator": "=",
            "property": "current_stage",
            "value": "Approved"
          },
          {
            "operator": "isNotEmpty",
            "property": "assignee"
          }
        ]
      },
      "description": "Ticket has been prioritized, approved, and assigned",
      "title": "Readiness Status"
    }
  ],
  "levels": [
    {
      "color": "red",
      "title": "Not ready for coding agent"
    },
    {
      "color": "green",
      "title": "Ready for coding agent"
    }
  ]
}

Click Create to save the scorecard

Create self-service actions

We will create three self-service actions to handle the triage workflow: improving tickets with AI suggestions, approving suggestions, and requesting refinements.

Create AI improvement action

This action allows the AI agent to suggest improvements to ticket descriptions.

Go to the self-service page of your portal.
Click on + New Action.
Click on the {...} Edit JSON button.

Copy and paste the following JSON configuration:

Ask AI to improve ticket action (Click to expand)

{
  "identifier": "ask_ai_to_improve_on_ticket",
  "title": "Ask AI to Improve Ticket",
  "description": "Self-service action that enriches tickets with additional context based on AI suggestions",
  "trigger": {
    "type": "self-service",
    "operation": "CREATE",
    "userInputs": {
      "properties": {
        "ai_suggested_description": {
          "title": "AI Suggested Description",
          "icon": "DefaultProperty",
          "type": "string",
          "format": "markdown"
        },
        "confidence_score": {
          "icon": "DefaultProperty",
          "type": "number",
          "title": "Confidence Score",
          "maximum": 100,
          "minimum": 0,
          "description": "The estimated confidence score for the current ticket description"
        },
        "current_stage": {
          "type": "string",
          "title": "Current Stage",
          "enum": [
            "Awaiting approval",
            "Not started",
            "Draft",
            "Approved"
          ],
          "enumColors": {
            "Awaiting approval": "yellow",
            "Not started": "lightGray",
            "Draft": "orange",
            "Approved": "green"
          }
        },
        "ticket": {
          "type": "string",
          "blueprint": "jira_ticket",
          "title": "Ticket",
          "format": "entity"
        }
      },
      "required": [
        "confidence_score",
        "current_stage",
        "ticket"
      ],
      "order": [
        "ticket",
        "ai_suggested_description",
        "confidence_score",
        "current_stage"
      ]
    }
  },
  "invocationMethod": {
    "type": "UPSERT_ENTITY",
    "blueprintIdentifier": "jira_ticket",
    "mapping": {
      "identifier": "{{ .inputs.ticket.identifier }}",
      "properties": {
        "ai_suggested_description": "{{ .inputs.ai_suggested_description }}",
        "confidence_score": "{{ .inputs.confidence_score }}",
        "current_stage": "{{ .inputs.current_stage }}"
      }
    }
  },
  "requiredApproval": false
}

Click Save to create the action.

Create approval action

This action allows human reviewers to approve or reject AI suggestions.

Go back to the self-service page of your portal.
Click on + New Action.
Click on the {...} Edit JSON button.

Copy and paste the following JSON configuration:

Approve AI suggestions action (Click to expand)

{
  "identifier": "approve_ai_suggested_ticket_descripted",
  "title": "Approve AI Suggestions",
  "description": "Approves the suggested ticket description from the triage agent and proceeds to the next step",
  "trigger": {
    "type": "self-service",
    "operation": "DAY-2",
    "userInputs": {
      "properties": {
        "decision": {
          "type": "string",
          "title": "Decision",
          "enum": [
            "Approve & Proceed",
            "Reject"
          ],
          "icon": "DefaultProperty",
          "enumColors": {
            "Approve & Proceed": "green",
            "Reject": "red"
          }
        }
      },
      "required": [
        "decision"
      ],
      "order": [
        "title_section",
        "decision"
      ],
      "titles": {
        "title_section": {
          "title": "Approve to apply the AI's suggested description and mark as ready, or reject to keep current state."
        }
      }
    },
    "blueprintIdentifier": "jira_ticket"
  },
  "invocationMethod": {
    "type": "UPSERT_ENTITY",
    "blueprintIdentifier": "jira_ticket",
    "mapping": {
      "identifier": "{{ .entity.identifier }}",
      "properties": {
        "description": "{{if (.inputs.decision == (\"Approve & Proceed\")) then .entity.properties.ai_suggested_description else .entity.properties.description end}}",
        "current_stage": "{{if (.inputs.decision == (\"Approve & Proceed\")) then \"Approved\" else .entity.properties.current_stage end}}",
        "confidence_score": 95
      }
    }
  },
  "requiredApproval": false
}

Click Save to create the action.

Create retry action

This action allows users to provide feedback and request AI to refine its suggestions.

Go back to the self-service page of your portal.
Click on + New Action.
Click on the {...} Edit JSON button.

Copy and paste the following JSON configuration:

Retry AI suggestions action (Click to expand)

{
  "identifier": "retry_ai_suggested_ticket_description",
  "title": "Retry AI Suggestions",
  "description": "Request AI to re-evaluate and improve the suggested ticket description based on your feedback.",
  "trigger": {
    "type": "self-service",
    "operation": "DAY-2",
    "userInputs": {
      "properties": {
        "notes": {
          "type": "string",
          "title": "Feedback Notes",
          "description": "Provide guidance or missing details to help the AI improve its next suggestion."
        }
      },
      "required": [],
      "order": [
        "notes"
      ]
    },
    "condition": {
      "type": "SEARCH",
      "rules": [
        {
          "operator": "isNotEmpty",
          "property": "ai_suggested_description"
        }
      ],
      "combinator": "and"
    },
    "blueprintIdentifier": "jira_ticket"
  },
  "invocationMethod": {
    "type": "WEBHOOK",
    "url": "https://api.getport.io/v1/agent/ticket_triage_agent/invoke",
    "agent": false,
    "synchronized": true,
    "method": "POST",
    "headers": {
      "RUN_ID": "{{ .run.id }}",
      "Content-Type": "application/json"
    },
    "body": {
      "prompt": "The Project Manager has requested changes to the AI-suggested description.\n\nFeedback:\n{{ .inputs.notes }}\n\nTicket Details:\n- Jira Ticket ID: {{ .entity.identifier }}\n- Current Description: {{ .entity.properties.description }}\n- AI Suggested Description: {{ .entity.properties.ai_suggested_description }}\n- Related Entities in the Catalog (for context): {{ .entity.relations }}",
      "labels": {
        "source": "request_changes_to_jira_task_action",
        "jira_ticket_id": "{{ .entity.identifier }}"
      }
    }
  },
  "requiredApproval": false
}

Click Save to create the action.

Create AI agent

Now we will create the AI agent that evaluates tickets and suggests improvements.

Go to the AI Agents page of your portal.
Click on + AI Agent.
Toggle Json mode on.

Copy and paste the following JSON schema:

Ticket triage AI agent (Click to expand)

{
  "identifier": "ticket_triage_agent",
  "title": "Ticket Triage Agent",
  "icon": "Details",
  "team": [],
  "properties": {
    "description": "An agent that evaluate whether a ticket has sufficient context and well-defined requirements for successful execution by a coding agent",
    "status": "active",
    "prompt": "Your task is to evaluate and improve Jira tickets before coding agent execution.\n\n### Steps\n1. **Gather Context:** If the ticket is linked to other entities (e.g. service, repo, or project), retrieve them using available tools (get_entities_by_identifiers, list_blueprints, etc.) and extract relevant info such as README, owners, dependencies, or architecture notes. Use this data to enrich your reasoning.\n2. **Evaluate Description:** Compare the ticket description with the project’s success criteria or a template inside the description. If the ticket description contains a template/PRD structure, treat that template as canonical: fill each template section with specific, actionable content.\n3. **Identify Gaps:** Point out missing or unclear specs. If you cannot find relevant details in the ticket or linked entities, explicitly write what’s missing under the section— e.g.,  \n   - “Couldn’t find database or cloud or email configuration details.”  \n   - “No documentation on API endpoints.”  \nNote that the goal is to make it clear what the PM or engineer needs to add to make the ticket complete\n4. **Score (0–100):** Rate the ticket’s completeness and clarity.\n5. **Assign Stage:** Use “Awaiting approval” if AI edits are proposed, “Approved” if the ticket is already \n\n## Response Rules\n\n1. ALWAYS respond by calling the `ask_ai_to_improve_on_ticket` self-service action with:\n\n   ```json\n   {\n     \"actionIdentifier\": \"ask_ai_to_improve_on_ticket\",\n     \"properties\": {\n       \"ticket\": \"<provided entity identifier>\",\n       \"current_stage\": \"<determined stage>\",\n       \"confidence_score\": <number 0-100>\n       \"ai_suggested_description\": \"<markdown-formatted with filled template or empty if already perfect>\"\n     }\n   }\n   ```\n\n### Rules\n* Always include `confidence_score` and `current_stage`.\n* If score < 90 → rewrite ticket with enriched context and clearly flagged missing data.\n* If score ≥ 90 → leave `ai_suggested_description` with empty string (ticket appears complete).\n* Be concise, data-grounded, and specific about what info was found vs. missing under the respective section in the template.",
    "execution_mode": "Automatic",
    "tools": [
      "^(list|get|search|track|describe)_.*",
      "run_ask_ai_to_improve_on_ticket"
    ]
  },
  "relations": {}
}

Click Create to save the agent.

MCP Enhanced Capabilities

The AI agent uses MCP (Model Context Protocol) enhanced capabilities to automatically discover important and relevant blueprint entities via its tools. The ^(list|get|search|track|describe)_.* pattern allows the agent to access and analyze related entities in your software catalog, providing richer context for ticket evaluation.

Set up automations

We'll create two automations to orchestrate the AI triage workflow:

Trigger the AI agent when new tickets are created or synced.
Automatically assign coding agents to approved tickets.

Create ticket sync trigger automation

This automation triggers the AI triage agent whenever a new Jira ticket is created or synced to the Catalog.

Go to the automations page of your portal.
Click on + Automation.

Copy and paste the following JSON schema:

Ticket sync trigger automation (Click to expand)

{
  "identifier": "trigger_triage_agent_on_sync",
  "title": "Trigger Triage Agent On Ticket Creation",
  "description": "Automation that triggers AI triage analysis when a Jira ticket is created or synced to the catalog",
  "icon": "AI",
  "trigger": {
    "type": "automation",
    "event": {
      "type": "ENTITY_CREATED",
      "blueprintIdentifier": "jira_ticket"
    },
    "condition": {
      "type": "JQ",
      "expressions": [],
      "combinator": "and"
    }
  },
  "invocationMethod": {
    "type": "WEBHOOK",
    "url": "https://api.getport.io/v1/agent/ticket_triage_agent/invoke",
    "agent": false,
    "synchronized": true,
    "method": "POST",
    "headers": {
      "RUN_ID": "{{ .run.id }}",
      "Content-Type": "application/json"
    },
    "body": {
      "prompt": "Here is the Jira ticket information:\n\nJira Ticket ID: {{.event.diff.after.identifier}}\nJira Ticket Title: {{.event.diff.after.title}}\n\nRelated Entities in the Catalog (for context): {{.event.diff.after.relations}}\n\nFull Jira Ticket Properties:\n{{.event.diff.after.properties}}",
      "labels": {
        "source": "auto_enhance_jira_task_automation",
        "jira_ticket_id": "{{ .event.diff.after.identifier }}"
      }
    }
  },
  "publish": true
}

Click Create to save the automation.

Create coding agent assignment automation

This automation automatically assigns coding agents to tickets once they're approved and ready.

Go back to the automations page of your portal.
Click on + Automation.

Copy and paste the following JSON schema:

Auto-assign coding agent automation (Click to expand)

{
  "identifier": "auto_assign_claude_on_ready_for_ai",
  "title": "Auto-Assign Claude Coding Agent on Ready for AI",
  "description": "Automatically triggers Claude to begin code generation once a Jira ticket is approved and marked as 'Ready for AI'. This ensures a seamless handoff from human triage to AI execution.",
  "trigger": {
    "type": "automation",
    "event": {
      "type": "ENTITY_UPDATED",
      "blueprintIdentifier": "jira_ticket"
    },
    "condition": {
      "type": "JQ",
      "expressions": [
        ".diff.before.scorecards.ai_readiness_stage.level != \"Ready for coding agent\"",
        ".diff.after.scorecards.ai_readiness_stage.level == \"Ready for coding agent\""
      ],
      "combinator": "and"
    }
  },
  "invocationMethod": {
    "type": "WEBHOOK",
    "url": "https://api.getport.io/v1/actions/run_claude_code_demo/runs",
    "agent": false,
    "synchronized": true,
    "method": "POST",
    "headers": {
      "RUN_ID": "{{ .run.id }}",
      "Content-Type": "application/json"
    },
    "body": {
      "properties": {
        "service": "<GITHUB-ORG>/<REPO>",
        "prompt": "You are a senior software engineer tasked with implementing the following approved ticket: '{{ .event.diff.after.title }}'.\n\nBase your implementation on the approved technical description below:\n\n{{ .event.diff.after.properties.description }}\n\nOnce complete, generate a pull request summarizing what was implemented and why."
      }
    }
  },
  "publish": true
}

Customize coding agent integration

Replace the service with the name of the Github repository. Also, replace the webhook URL and payload structure in the automation to match your preferred coding agent's API. You can integrate with Claude Code, GitHub Copilot, Devin, or any other AI coding assistant that accepts webhook triggers.

Click Create to save the automation.

Test the workflow

Now let us test the complete AI triage workflow to ensure everything works correctly.

Create a test ticket

Go to your Jira instance and create a new ticket with minimal information, for example:
- Title: "Implement Product Search and Filtering on Storefront"
- Description: "Users can't search for products on the website"
- Labels: Add any relevant labels

Verify AI triage analysis

Go to your Port catalog and find the synced Jira ticket
Check that the AI agent has analyzed the ticket and provided:
- A confidence score.
- A suggested improved description (if the original was incomplete).
- An appropriate current stage.

Test human review process

If the AI suggested improvements, use the Approve AI Suggestions action to review and approve the changes.
If you want to request refinements, use the Retry AI Suggestions action to provide feedback.

Verify coding agent assignment

Once a ticket is approved and marked as Ready for coding agent, check that your coding agent is automatically triggered.
Verify that the coding agent receives the enriched ticket description with all necessary context.

Common use cases​

Prerequisites​

Set up data model​

Create Feature blueprint​

Create AI readiness scorecard​

Create self-service actions​

Create AI improvement action​

Create approval action​

Create retry action​

Create AI agent​

Set up automations​

Create ticket sync trigger automation​

Create coding agent assignment automation​

Test the workflow​

Create a test ticket​

Verify AI triage analysis​

Test human review process​

Verify coding agent assignment​

Related guides​

Common use cases

Prerequisites

Set up data model

Create Feature blueprint

Create AI readiness scorecard

Create self-service actions

Create AI improvement action

Create approval action

Create retry action

Create AI agent

Set up automations

Create ticket sync trigger automation

Create coding agent assignment automation

Test the workflow

Create a test ticket

Verify AI triage analysis

Test human review process

Verify coding agent assignment

Related guides