Vellum

Vellum offers a comprehensive suite of tools for development and engineering in AI systems. It provides a unified prompt management dashboard with features for systematically iterating on prompts, side-by-side model comparisons, and native support for tools, structured outputs, and OpenAPI specifications. The platform includes an IDE for building steerable, agent-like systems with a visual graph builder for orchestrating complex AI systems. It supports any model, custom code, and map/reduce functions, with built-in features such as loops, parallelism, and error handling. The Workflows SDK is a full-stack SDK and GUI designed for building AI applications with precision and flexibility, featuring bi-directional sync between code and UI, fine-grained control flow, and global state management. It is customizable with Docker and streaming support. Vellum also supports test-driven development for AI through scalable test suites built via UI, API, or CSV, using pre-built or custom metrics for evaluation. It tracks quality, cost, latency, and regressions over time. For retrieval, Vellum provides powerful RAG infrastructure with simple APIs for uploading and querying unstructured data, allowing for advanced use-cases with adjustable chunking, embeddings, and search. It supports various file types, including tables and images. Deployments are streamlined with one-click deployment across any model or provider, staging environments for safe iteration, and scalable inference endpoints for production use. Finally, Vellum offers full observability into AI system performance with audit logs, debugging tools, feedback capture, evaluation loops on live traffic, and dashboards to track cost, latency, errors, and trends.