Building Heskala's AI Agent

Building Heskala's AI Agent: Deep Customization on Top of @convex-dev/agent

When we started adding AI to Heskala, our ERP for small and mid-size businesses in Honduras, we didn't want another chat box that says "I'm sorry, I don't understand." We wanted an agent that lives inside the system, knows the data, and can actually do work.

The @convex-dev/agent component from Convex gave us the foundation. What it didn't give us — and what we had to build — is what made the difference.

This post is a technical walkthrough of what we built, why, and how it fits into the rest of the system.

The starting point

The Convex Agent component ships with a lot: thread persistence, tool calls, streaming, message history, and integration with the AI SDK. For a single-user demo, it's enough. For a multi-tenant ERP with role-based permissions on every table, it isn't.

Our requirements were clear from day one:

• Opt-in, never permissive defaults. No company should ever wake up to find out the AI is doing things they didn't ask for.
• Per-user configuration. Two people in the same company can have different tools enabled, different instruction sets, different limits.
• Human approval for any write. The agent can read anything. It can write nothing without a human clicking "approve."
• Multi-module awareness. Invoices, quotations, clients, products, sales, operations, documents, reports — all in one conversation.
• Costs we can track. Every token, every request, per user, per company, per operation type.
• Streaming with parallel sub-agents. When the user asks "give me a sales summary AND overdue invoices AND a top-products chart," all three should run in parallel, stream live, and cancel cleanly if the user interrupts.

The default component gave us the first three bullets of "what an agent is." We had to engineer the rest.

Extending the foundation

The first thing we realized: the auth context returns null inside the agent's tool execution context. That's by design — the agent runs in an internal action, not an authenticated request. But it means every secure query we have, which depends on the user's identity to check permissions, breaks the moment an agent tool calls it.

Our fix was a bridge we call the query proxy.

Tool → publicQueryRef → proxySecureQuery (mutation)
                              └── ctx.runQuery(internal.*.*.*Internal, { ...input, companyId })

For every secure query that an agent tool needs, we extract its logic into a pure helper, keep the public version for the web path, and add a sibling internal version that takes the company ID directly. The proxy is a dispatcher that routes the call to the right internal. Result: agent tools respect the exact same permission model as the web app, with zero duplication.

The other big extension was per-user configuration. The default component assumes a single config per agent. We needed one per user. The config table now has a user field with a composite index, every query and mutation is scoped by (companyId, userId), and a first-read initialization pattern ensures the first time a user opens the settings page, they get a config with everything turned off.

The approval flow

The most architecturally interesting part is the approval flow for write operations. We split it across four layers with a clear separation of concerns:

Layer	Responsibility
The orchestrator	Lifecycle: send → generate → submit → continue. No module knowledge.
The dispatcher	Generic component that finds the right handler for each tool.
Module-specific handlers	Per-module post-approval logic (clients, quotations, etc.).
The query proxy	The only bridge between agent context and the secure layer.

When the LLM decides to call a write tool — say, createQuotation — the tool returns { pending: true, message: "Awaiting approval", input } instead of executing. The orchestrator pauses. The frontend renders an approval card. When the user clicks approve, the submit mutation lands, the dispatcher finds the registered handler, the handler calls the module's real mutation, saves a markdown message with a clickable link to the new record, and triggers the LLM to continue the conversation with the actual result.

The orchestrator never knows what a quotation is. The handler never knows about threads or messages. They meet at a thin contract.

Parallel sub-agents with cancel tokens

When the user asks for a complex answer, the orchestrator can fan out to several sub-agents in parallel. Each one writes its results to a shared live results table as it goes, so the UI can stream them in independently. A "summary" agent, a "data" agent, and a "chart" agent all run at the same time, and the user sees them populate as they finish.

But what happens when the user sends a second message mid-stream? We need to cancel the in-flight work. A separate table holds a single row per thread with the current run identifier. Every sub-agent checks before each chunk whether its identifier is still the active one. If not, it stops. The new prompt takes over without leftover noise from the previous one.

Persistent memory, opt-in

The agent has memory. Per user. It extracts facts from conversations, deduplicates them, and injects the relevant ones into every prompt. The extraction runs in the background after each turn. There's a nightly audit that consolidates redundant or stale memories.

We're building this in phases. Phase 1 is server-side, single source of truth. Phase 2-3 moves to a local-first CRDT document so memory can sync between devices. The server-side table becomes the materialized projection.

The opt-in philosophy carries through: memory is enabled by default for the user, but users can see, edit, and delete their own memories from the settings page. No black box.

What it looks like in the UI

A real conversation:

1. User asks: "Why are we low on the top 3 selling products this month?"
2. Agent streams in. Calls product summary, low-stock, and recent stock movements in parallel.
3. The three sub-agents populate the UI live.
4. The summary writes a markdown response with a small table and a suggestion to reorder.
5. If the user clicks "yes, draft a purchase order," the agent asks for approval.
6. User approves. The order is created. A link to it appears in the chat.
7. The agent continues: "Done. Anything else you want to check?"

No copy-paste. No "open another tab." No "let me log into the ERP for you." Just an answer and an action, in the same place.

Where it's going

We just shipped:

• Per-user AI settings with opt-in everything
• Real-time notification popover that auto-opens on new alerts
• correlative expiration monitoring with idempotent notifications
• A reactive read-plus-effect-plus-write pattern for resources that need server-side checks before any user-facing action

What's next:

• The local-first memory CRDT migration
• More write tools (update invoice, cancel sale, etc.) with the same approval flow
• A "what changed" digest that pulls insights from across all modules

Stack

• Backend: Convex (TypeScript end-to-end, reactive by default)
• Agent framework: @convex-dev/agent, deeply extended
• Models: DeepSeek V3/Minimax via Anthropic-compatible API
• Frontend: React + TanStack Router + the Convex reactive hooks
• Streaming: incremental delta persistence with smooth text animation
• Approval UX: streaming responses, custom approval cards for writes

The takeaway

The Convex Agent component is a great starting point. It gives you the lifecycle, the streaming, the persistence. What it doesn't give you — and what every real production agent needs — is the boring infrastructure: per-user config, permission-aware tool calls, human approval flows, cost tracking, parallel sub-agents with cancellation, and most importantly, defaults that respect the user.

If you're building an agent on Convex, build those things early. They take longer than you think, and they're the difference between a demo and a product.