AI Consultant for a Multi-Agent Workflow Builder
I helped a fintech company turn a multi-agent design into a working product. Users describe a financial workflow in plain language, and AI agents generate and execute it.
Client: Fintech (PMS)
The Challenge
Portfolio managers and analysts working with digital assets need to monitor positions across multiple exchanges, track performance metrics like time-weighted returns, detect risk signals early, and get notified when something needs attention. Traditionally, each of these workflows requires a data engineer to build and maintain custom pipelines.
The client, a fintech company, had built a portfolio management system (PMS) and wanted users to generate executable workflows through natural language: scheduled alerts, interactive dashboards, automated reports and conditional notifications. The product applies the workflow-builder model of Zapier or n8n to financial portfolio data.
The team had initially considered migrating to n8n as the workflow engine, but decided to build a custom platform instead because the domain requirements (financial entity hierarchies, typed data operations, real-time portfolio computations) were too specific for a generic workflow tool.
Financial platforms also require auditability. Every action must be reproducible and traceable for compliance and legal reasons, so the system generates deterministic workflows instead of answering questions on the fly. Once AI agents build a workflow, it runs through Dagster with every step logged and every input and output recorded. Structured, validated data also reduces hallucination risk during execution.
The platform had been designed and partially implemented when I joined, but it could not reliably produce working workflows. Prompts broke with unexpected inputs. Agents picked the wrong tools or generated code that did not match the expected data structures. I joined to make the system generate valid workflows reliably and prepare it for production.
Solution Approach
My Role
I joined as an AI consultant to make the AI layer produce correct outputs, pass context between agents and handle real inputs reliably.
- Prompt engineering across agent tiers: different parts of the system needed different prompting strategies. I refined those prompts across hundreds of test cases.
- Agent coordination: in a multi-agent system, the quality of what one agent produces depends on the context it receives from the previous one. I refined how information flows between agents to minimize both token waste and information loss, which directly impacted the accuracy of the final output.
- Output validation and correctness: some agents generate executable code, so I implemented validation steps that verify both the structure and the outputs of what agents produce. This includes syntax checks, type validation against expected data schemas, and execution-time verification to catch issues before they reach production data.
- Model selection by agent category: A/B tests showed that lighter models were sufficient for routing and classification. Claude Opus consistently performed better on code generation and tool orchestration, where errors were more expensive.
- Error recovery and reliability: when a workflow step fails, the system feeds the error context back to the agent for a corrected attempt. I improved this feedback loop and hardened the overall state machine against edge cases that caused failures in production-like scenarios.
Results & Impact
- Brought the workflow builder from a non-functional prototype to a system that generated valid workflows reliably
- Adjusted prompts across the multi-agent system based on hundreds of test cases
- Selected Claude Opus (Anthropic) as the primary model for the agentic flow after extensive A/B testing across providers, with lighter models for routing and classification
- Improved the build-execute-retry cycle, reducing failed workflow creation
- Added validation to tool definitions and database access before workflows reached production data
- Users can create scheduled dashboards, conditional alerts, and multi-step financial reports through conversation
Technologies Used
- Python multi-agent framework with structured LLM output
- State machine for incremental, self-correcting workflow construction
- REST API for conversational interaction and workflow management
- MCP (Model Context Protocol) for tool interfaces between agents and services
- Deterministic workflow engine for scheduling, execution, and audit logging
- PostgreSQL for state persistence and portfolio data
- React-based UI for the workflow editor and interactive dashboards