Testing & Debugging

Before going live, test your bot thoroughly. Aerostack provides a test console, detailed conversation logs, and workflow execution traces that show exactly what happened at each step.

Test Console

The test console lets you send messages to your bot without going through a messaging platform. It runs the full bot engine pipeline — LLM calls, tool execution, workflow processing — and returns detailed results.

Via Dashboard

1. Navigate to Bots and open your bot
2. Click the Test button
3. Type a message and press Enter
4. View the response, tool calls, token usage, and latency

The dashboard test console shows a chat-like interface with expandable details for each message, including which tools were called, their arguments and results, and token counts.

Via API

curl -X POST https://api.aerostack.dev/api/bots/YOUR_BOT_ID/test \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "message": "What is the status of order #12345?" }'

Response:

{
  "response": "Order #12345 is currently being shipped...",
  "tool_calls": [
    {
      "toolName": "orders__get_order",
      "arguments": { "order_id": "12345" },
      "result": { "status": "shipped", "tracking": "1Z999AA10123456784" },
      "latencyMs": 230,
      "success": true
    }
  ],
  "tokens": { "input": 485, "output": 42 },
  "token_breakdown": {
    "systemPrompt": 120,
    "toolDefinitions": 200,
    "conversationHistory": 80,
    "currentMessage": 25,
    "toolResults": 40,
    "llmOutput": 42,
    "total": 507
  },
  "cost_cents": 1,
  "latency_ms": 2340,
  "loop_iterations": 1,
  "conversation_id": "bconv_abc123"
}

What to Test

Conversation Logs

Every conversation is stored and accessible through the dashboard or API.

Via Dashboard

1. Navigate to Bots and open your bot
2. Click the Conversations tab
3. Browse conversations sorted by most recent activity
4. Click a conversation to see the full message history

Each message shows:

• User message text
• Bot response text
• Tool calls made (with arguments and results)
• Token counts and cost
• Timestamps and latency

Via API

# List conversations
curl "https://api.aerostack.dev/api/bots/YOUR_BOT_ID/conversations?limit=20" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"
 
# Get a specific conversation with messages
curl "https://api.aerostack.dev/api/bots/YOUR_BOT_ID/conversations/CONV_ID?limit=200" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

Workflow Execution Logs

When workflows are enabled, each execution produces a detailed log showing what happened at each node.

Execution Log Structure

{
  "workflow_run_id": "wfr_abc123",
  "status": "completed",
  "nodes_executed": 5,
  "nodes_total": 8,
  "duration_ms": 3400,
  "execution_log": [
    {
      "node_id": "trigger",
      "node_type": "trigger",
      "status": "completed",
      "duration_ms": 1,
      "variables_set": ["message", "user_id", "platform"]
    },
    {
      "node_id": "classify",
      "node_type": "llm_call",
      "status": "completed",
      "duration_ms": 890,
      "output": { "text": "billing", "tokensInput": 45, "tokensOutput": 3 },
      "variables_set": ["intent"]
    },
    {
      "node_id": "route",
      "node_type": "logic",
      "status": "completed",
      "duration_ms": 0,
      "branch_taken": "case_billing"
    },
    {
      "node_id": "lookup_account",
      "node_type": "mcp_tool",
      "status": "completed",
      "duration_ms": 340,
      "tool_name": "database__query",
      "variables_set": ["account"]
    },
    {
      "node_id": "send_billing",
      "node_type": "send_message",
      "status": "completed",
      "duration_ms": 0,
      "message_sent": "Your current balance is..."
    }
  ]
}

Reading the Log

Common Debugging Scenarios

Workflow takes the wrong branch: Check the logic node’s condition and the actual variable values. The execution log shows what value was evaluated. Common cause: the LLM returned extra whitespace in the classification text (e.g., "billing " instead of "billing").

MCP tool returns an error: Check the tool name (typo?), arguments (missing required field?), and the MCP server status. The execution log shows the exact arguments sent and the error returned.

Workflow stops unexpectedly: Check if the 50-node execution limit was reached, or if a node errored without an error_handler. The nodes_executed vs nodes_total count shows how far execution got.

Variable not available in downstream node: Check that the upstream node’s outputVariable is set correctly. Variables are flat-scoped — make sure there are no name collisions with another node.

Insights

The insights endpoint surfaces patterns you might miss:

curl https://api.aerostack.dev/api/bots/YOUR_BOT_ID/insights \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

{
  "unanswered_questions": [
    { "question": "Can I get a refund?", "bot_response": "...", "created_at": "..." },
    { "question": "Do you offer enterprise pricing?", "bot_response": "...", "created_at": "..." }
  ],
  "resolution_rate": 78,
  "total_answered": 312,
  "total_responses": 400
}

Unanswered questions are messages where the bot responded without using any tools — a signal that the bot might not have the right tools or knowledge to answer. Review these to identify gaps in your workspace tools or system prompt.

Resolution rate is the percentage of messages where the bot used at least one tool to provide an answer.

Analytics Dashboard

Track your bot’s performance over time:

curl "https://api.aerostack.dev/api/bots/YOUR_BOT_ID/analytics?from=2026-03-01&to=2026-03-17" \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

Returns daily breakdowns of:

• Messages received and sent
• Token usage (input and output)
• Total cost
• Unique users
• Top tools used
• Average latency

Use analytics to identify:

• Cost spikes — a sudden increase might mean a tool is returning unexpectedly large results
• Latency increases — a slow MCP server or model change
• Usage patterns — peak hours, most active users, most common intents

Debugging Checklist

When your bot is not working as expected, run through this checklist:

1. Is the bot active? Check status via the dashboard or GET /api/bots/:id. Must be active, not draft or paused.

2. Is the webhook registered? For Telegram and Discord, Aerostack registers automatically on activation. For Slack and WhatsApp, check the manual setup.

3. Is the platform config correct? Verify bot tokens, signing secrets, and other credentials have not expired or been revoked.

4. Does the workspace have tools? Check that MCP servers are connected and healthy. An empty workspace means the bot can only do free-form conversation.

5. Is the system prompt appropriate? A vague system prompt leads to unpredictable behavior. Be specific about the bot’s role, tools, and boundaries.

6. Is the workflow valid? If workflow mode is enabled, check that the JSON is valid, has a trigger node, and all edges connect correctly.

7. Check the test console. Send the same message through the test endpoint. If it works there but not on the platform, the issue is with the platform adapter or webhook, not the bot engine.

8. Check conversation logs. Look at the actual messages and tool calls. The logs show exactly what the LLM decided and why.

9. Check rate limits. If the bot stops responding, you may have hit the rate limit (60 RPM for most platforms, 10 RPM for the test endpoint).

10. Check wallet balance. In wallet billing mode, the bot stops responding when the balance reaches zero.

Next Steps

• Cost Optimization — Reduce costs identified during testing
• Workflow Recipes — Copy-paste workflow examples for testing
• Platform Guides — Platform-specific troubleshooting