Smart WebhooksQuick Start

Quick Start: Smart Webhooks

Create an AI-powered GitHub issue triager that automatically labels and assigns new issues. This takes about 3 minutes.

You need an Aerostack account and an MCP workspace with a GitHub MCP server connected. If you have not set up a workspace yet, see MCP Workspaces.

Create the Smart Webhook

From the Dashboard:

  1. Log in to the Aerostack Dashboard
  2. Navigate to Smart Webhooks in the sidebar
  3. Click Create Webhook
  4. Fill in the form:
    • Name: github-triage
    • Source Type: github
    • Workspace: Select your workspace (must have GitHub MCP connected)
    • Instructions: 
      When a new issue is created:
1. Read the issue title and body
2. Classify it as one of: bug, feature-request, question, documentation
3. Apply the appropriate label using the GitHub API
4. If it's a bug, also add the "priority:high" label and assign it to the on-call team
5. Post a comment acknowledging the issue was received and classified
    • Source Events: opened (only process new issues)
    • Signing Secret: (optional, add your GitHub webhook secret for security)
  5. Click Create

You will see your webhook URL. Copy it.

Or via the API:

curl -X POST https://api.aerostack.dev/api/smart-webhooks \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "github-triage",
    "workspace_id": "ws_your_workspace_id",
    "source_type": "github",
    "instructions": "When a new issue is created:\n1. Read the issue title and body\n2. Classify it as one of: bug, feature-request, question, documentation\n3. Apply the appropriate label using the GitHub API\n4. If it is a bug, also add the priority:high label and assign it to the on-call team\n5. Post a comment acknowledging the issue was received and classified",
    "source_events": ["opened"],
    "signing_secret": "your_github_webhook_secret",
    "llm_model": "gpt-4o-mini"
  }'

Response:

{
  "webhook": {
    "id": "swh_9a2b3c4d5e6f7g8h9i",
    "name": "github-triage",
    "slug": "github-triage",
    "source_type": "github",
    "source_events": "[\"opened\"]",
    "status": "active",
    "enabled": 1,
    "llm_provider": "openai",
    "llm_model": "gpt-4o-mini",
    "max_tool_calls": 10,
    "timeout_ms": 30000
  },
  "webhook_url": "https://api.aerostack.dev/api/webhooks/smart/github-triage"
}

Configure GitHub

  1. Go to your GitHub repository Settings > Webhooks > Add webhook
  2. Set the Payload URL to the webhook URL from the previous step: 
    https://api.aerostack.dev/api/webhooks/smart/github-triage
  3. Set Content type to application/json
  4. Set Secret to the same signing secret you used when creating the webhook
  5. Under Which events would you like to trigger this webhook?, select Let me select individual events and check Issues
  6. Click Add webhook

Test It

Create a new issue in your GitHub repository:

  • Title: “Login page returns 500 error after password reset”
  • Body: “Steps to reproduce: 1. Request a password reset. 2. Click the reset link. 3. Set new password. 4. Try to log in. Result: 500 Internal Server Error.”

Within a few seconds, the AI agent will:

  1. Read the issue content
  2. Classify it as a bug
  3. Apply the bug and priority:high labels
  4. Assign it to the on-call team
  5. Post a comment acknowledging the issue

View Execution History

Check what the AI did by viewing the run history:

curl https://api.aerostack.dev/api/smart-webhooks/swh_9a2b3c4d5e6f7g8h9i/runs \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"
{
  "runs": [
    {
      "id": "whr_1a2b3c4d5e6f7g8h9i",
      "webhook_id": "swh_9a2b3c4d5e6f7g8h9i",
      "event_type": "opened",
      "payload_summary": "{\"action\":\"opened\",\"issue\":{\"title\":\"Login page returns 500 error...\"}}",
      "actions_taken": "Classified issue #42 as 'bug'. Applied labels: bug, priority:high. Assigned to @oncall-team. Posted acknowledgment comment.",
      "tool_calls": "[{\"name\":\"github_add_labels\",\"success\":true},{\"name\":\"github_assign_issue\",\"success\":true},{\"name\":\"github_create_comment\",\"success\":true}]",
      "tokens_input": 1456,
      "tokens_output": 234,
      "cost_cents": 0.52,
      "latency_ms": 4521,
      "status": "success",
      "created_at": "2026-03-15T10:30:12Z"
    }
  ]
}

More Examples

Stripe Payment Processor

curl -X POST https://api.aerostack.dev/api/smart-webhooks \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "stripe-payments",
    "workspace_id": "ws_your_workspace_id",
    "source_type": "stripe",
    "instructions": "Handle Stripe payment events:\n- On payment_intent.succeeded: Log the payment in the database and send a confirmation email to the customer\n- On payment_intent.payment_failed: Send a notification to the customer about the failed payment\n- On customer.subscription.deleted: Update the customer access level in the database to free tier",
    "source_events": ["payment_intent.succeeded", "payment_intent.payment_failed", "customer.subscription.deleted"],
    "signing_secret": "whsec_your_stripe_signing_secret"
  }'

Deployment Monitor

curl -X POST https://api.aerostack.dev/api/smart-webhooks \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "deploy-monitor",
    "workspace_id": "ws_your_workspace_id",
    "source_type": "custom",
    "instructions": "When a deployment event arrives:\n1. Check if the deployment succeeded or failed\n2. If succeeded, post a success message to the #deployments Slack channel with the version and environment\n3. If failed, post an alert to #deployments and #oncall with the error details\n4. Log the deployment event in the database",
    "source_events": ["*"]
  }'

Debugging Tips

Since Smart Webhooks return { ok: true } for all responses (including errors), debugging requires checking the run history:

  1. No runs appearing? Check that:

    • The webhook status is active and enabled is 1
    • The event type matches your source_events filter
    • The HMAC signature is correct (if signing_secret is set)
    • The webhook is not being rate limited (30/min)

  2. Runs appearing but wrong actions? Refine your instructions. Be specific about what tools to use and what conditions trigger each action.

  3. Runs appearing with errors? Check that your MCP workspace has the required tools available and the workspace token is active.

Next Steps

OverviewHMAC Verification