feat: implement visual agent builder with node-based interface #1080

Ronit-Raj9 · 2025-11-02T22:50:23Z

🎨 Visual Agent Builder - MVP Implementation

Closes #916

📝 Summary

Implements a visual node-based interface for creating and configuring agents with drag-and-drop functionality, real-time validation, and seamless integration with the existing form-based builder.

✨ What's Implemented

Core Components

VisualAgentBuilder - Main orchestrator component managing nodes, edges, and state
Canvas - React Flow canvas with custom controls, minimap, and node selection
NodeLibrary - Sidebar with draggable node types
NodeProperties - Right panel for configuring selected nodes with real-time validation
BuilderModeToggle - Toggle between form and visual builder modes
ErrorBoundary - Error handling wrapper for visual builder

Node Types (MVP - 5 nodes)

Basic Info Node - Agent metadata (name, namespace, description, type)
LLM Node - Model configuration with namespace-aware selection
System Prompt Node - Agent instructions and behavior
Tool Node - Tool and agent selection with MCP server support
Output Node - Output formatting and streaming configuration

Property Editors

BasicInfoPropertyEditor - Name, namespace, type, description
LLMPropertyEditor - Model selection with namespace filtering
SystemPromptPropertyEditor - System prompt textarea
ToolPropertyEditor - Tool selection dialog integration
OutputPropertyEditor - Format and template configuration

Utility Libraries

graphConverter.ts - Bidirectional form ↔ graph conversion
formConverter.ts - Tool data transformations and node merging
validation.ts - Real-time graph validation with errors/warnings
utils.ts - Node positioning, default data, cycle detection, orphan detection

🎯 Key Features

✅ Drag-and-drop node creation from library
✅ Connect nodes with edges to define workflow
✅ Real-time validation with collapsible error/warning panels
✅ Delete functionality via Delete/Backspace keys or floating button
✅ Multi-select support for batch operations
✅ Default template graph (Basic Info → LLM → System Prompt → Output)
✅ Namespace-aware model filtering (only shows models from same namespace)
✅ Bidirectional sync - preserves data when switching between modes
✅ Mini-map and zoom controls for navigation
✅ Keyboard shortcuts for deletion

⚠️ Current Limitations

Scope: Declarative Agents Only

✅ Declarative agents fully supported (system prompt, model, tools, streaming)
❌ BYO agents not implemented in visual builder (form mode still works)
Users creating BYO agents should use form mode

Technical Constraints

Visual builder disabled in edit mode (only available for new agent creation)
Single LLM node per graph (enforced by validation)
Model selection restricted to same namespace as agent
Tool nodes require existing MCP servers or agents to be configured first

🔧 Technical Details

Dependencies Added

"@xyflow/react": "^12.9.2"  // Visual canvas and node management

Integration Points

Page Integration (ui/src/app/agents/new/page.tsx)
- Mode toggle between form and visual builder
- Syncs state between both modes
- Hides footer in visual mode for full-screen experience
Global Styles (ui/src/app/globals.css)
- Added .hide-footer class for visual builder full-screen mode
Type Definitions (ui/src/types/index.ts)
- Added 30+ new types for nodes, edges, validation, and props
- Exported MVP_NODE_TYPES constant for feature gating
Layout (ui/src/app/layout.tsx)
- Body uses flex flex-col h-screen for proper visual builder layout

Validation System

Node-level validation - Validates each node's data
Graph-level validation - Checks for required nodes, cycles, orphaned nodes
Real-time feedback - Debounced validation (300ms) on changes
Collapsible panels - Errors and warnings in compact side-by-side display
Create button - Disabled until all validation passes

State Management

React hooks for local state management
Memoization for expensive operations
Debounced validation to prevent excessive re-renders
Initialization guard to prevent duplicate graph creation

🧪 Testing Recommendations

Known Edge Cases

Switching to visual mode with existing form data populates graph
Empty graph shows helpful instructions
Duplicate nodes of unique types (LLM, Basic Info) trigger validation warnings
Orphaned nodes (disconnected) show warnings but don't block creation

📸 Screenshots

Add screenshots of:

Node library with all 5 node types
Connected graph with default template

Property panel for different node types

Validation errors/warnings display

Mode toggle button

🔄 Migration Notes

No breaking changes - Feature is opt-in:

Existing form builder unchanged and remains default
Visual builder only available for new agent creation (not edit mode)
All existing agents continue to work without modification
Form mode supports all features (BYO agents, advanced configs)

📚 Documentation Updates Needed

User guide for visual agent builder
Node type reference documentation
Video tutorial/GIF for basic workflow
Update DEVELOPMENT.md with visual builder architecture

🚀 Future Enhancements

See #916 for full roadmap. Next steps:

BYO agent support in visual builder
Edit mode support (convert existing agents to graph)
Advanced node types (Conditional, Memory, Resource)
Graph templates and presets
Export/import graph as JSON
Undo/redo functionality
Copy/paste nodes
Graph auto-layout algorithms

- Add visual agent builder using React Flow for drag-and-drop workflow creation - Implement 5 core node types: BasicInfo, LLM, SystemPrompt, Tool, Output - Add bidirectional conversion between form data and visual graph - Create node library, canvas, and properties panel components - Implement real-time validation with error/warning display - Add mode toggle to switch between form and visual builder - Support declarative agents with model and tool configuration - Integrate with existing agent creation workflow and validation Resolves kagent-dev#916 Signed-off-by: Ronit Raj <[email protected]>

peterj · 2025-11-03T12:29:22Z

nice! I'll take a look at the PR -- @Ronit-Raj9 it would be awesome if you could join the community meeting tomorrow and do a demo of this! https://kagent.dev/community

Ronit-Raj9 · 2025-11-03T15:35:08Z

Sure, I will show the demo in community meet tomorrow.

peterj · 2025-11-29T23:55:01Z

ui/src/app/agents/new/page.tsx

  });

+  // Builder mode state (only for non-edit mode)
+  const [builderMode, setBuilderMode] = useState<'form' | 'visual'>('form');


can you create a type for this: type BuilderMode = 'form | 'visual'`

sure, I will do that

peterj · 2025-11-29T23:55:28Z

ui/src/app/agents/new/page.tsx

+  // Hide footer when in visual builder mode
+  useEffect(() => {
+    if (builderMode === 'visual' && !isEditMode) {
+      document.body.classList.add('hide-footer');


why not a state variable here instead of directly accessing the dom?

Will fix this too

peterj · 2025-11-29T23:57:52Z

ui/src/app/agents/new/page.tsx

+      <div className={builderMode === 'visual' && !isEditMode ? "h-full flex flex-col overflow-hidden" : "min-h-screen p-8"}>
+        <div className={builderMode === 'visual' && !isEditMode ? "flex-1 flex flex-col overflow-hidden min-h-0" : "max-w-6xl mx-auto"}>
+          {/* Header with title and builder mode toggle */}
+          <div className={builderMode === 'visual' && !isEditMode ? "flex items-center justify-between px-6 py-4 border-b flex-shrink-0" : "flex items-center justify-between mb-8"}>


set a boolean outside of this return (e.g. const isVisualMode = builderMode === 'visual' && !isEditMode) and use that in the class names.

also, use clsx to make it easier to read:

<div className={clsx( isVisualMode ? "h-full flex flex-col overflow-hidden" : "min-h-screen p-8" )}>

Fine, will implement in this way.

Ronit-Raj9 added 2 commits November 3, 2025 03:54

Merge remote-tracking branch 'origin' into feat/agent_builder

e744537

Ronit-Raj9 requested a review from peterj as a code owner November 2, 2025 22:50

Ronit-Raj9 marked this pull request as draft November 2, 2025 22:50

Ronit-Raj9 marked this pull request as ready for review November 2, 2025 22:51

peterj reviewed Nov 29, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: implement visual agent builder with node-based interface #1080

feat: implement visual agent builder with node-based interface #1080

Uh oh!

Ronit-Raj9 commented Nov 2, 2025

Uh oh!

peterj commented Nov 3, 2025

Uh oh!

Ronit-Raj9 commented Nov 3, 2025

Uh oh!

peterj Nov 29, 2025

Uh oh!

Ronit-Raj9 Dec 12, 2025

Uh oh!

peterj Nov 29, 2025

Uh oh!

Ronit-Raj9 Dec 12, 2025

Uh oh!

peterj Nov 29, 2025

Uh oh!

Ronit-Raj9 Dec 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat: implement visual agent builder with node-based interface #1080

Are you sure you want to change the base?

feat: implement visual agent builder with node-based interface #1080

Uh oh!

Conversation

Ronit-Raj9 commented Nov 2, 2025

🎨 Visual Agent Builder - MVP Implementation

📝 Summary

✨ What's Implemented

Core Components

Node Types (MVP - 5 nodes)

Property Editors

Utility Libraries

🎯 Key Features

⚠️ Current Limitations

Scope: Declarative Agents Only

Technical Constraints

🔧 Technical Details

Dependencies Added

Integration Points

Validation System

State Management

🧪 Testing Recommendations

Known Edge Cases

📸 Screenshots

🔄 Migration Notes

📚 Documentation Updates Needed

🚀 Future Enhancements

Uh oh!

peterj commented Nov 3, 2025

Uh oh!

Ronit-Raj9 commented Nov 3, 2025

Uh oh!

peterj Nov 29, 2025

Choose a reason for hiding this comment

Uh oh!

Ronit-Raj9 Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

peterj Nov 29, 2025

Choose a reason for hiding this comment

Uh oh!

Ronit-Raj9 Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

peterj Nov 29, 2025

Choose a reason for hiding this comment

Uh oh!

Ronit-Raj9 Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants