The core philosophy behind Council is to provide developers with the tools to create sophisticated agents that exhibit predictable behavior. This is achieved through a powerful approach to control flow that utilizes key components such as Controllers, Filters, Evaluators, and Budgets. These elements work in concert to allow for automated routing between agents, comparing and evaluating results, and selecting the best outcomes for specific tasks or subtasks.
One of Council's primary strengths lies in its ability to facilitate the creation of reliable agents capable of tackling complex tasks. These agents can iterate over challenges that require exploring multiple alternatives, creating and completing subgoals, and evaluating quality under specified budget constraints. This level of sophistication makes Council particularly suitable for applications where accuracy and reliability are paramount.
Council is designed with data scientists in mind, offering a Python library that integrates seamlessly with popular frameworks in the AI ecosystem. It provides connectivity to a wide variety of LLMs, both natively and through integration with established libraries such as LangChain. This connectivity ensures that developers can leverage the most appropriate language models for their specific use cases.
The framework places a strong emphasis on production readiness. It aims to simplify the process of packaging and deploying agents at scale across multiple deployment platforms. Council's integration with Kubernetes facilitates seamless production deployments, enabling easy scaling and management of AI applications in enterprise environments.
Council's architecture is built around several key concepts. At the heart of the system are Agents, which encapsulate the end-to-end application logic from prompt input to final response. Agents can be nested within other Agents recursively, allowing for the creation of complex, hierarchical AI systems.
Controllers play a crucial role in Council's operation. They determine user intent based on prompts and prompt history, routing inputs to one or multiple registered Chains. Controllers also leverage Evaluators to score returned results and determine the next course of action, including whether revisions are needed.
Chains in Council are directed graphs of Skills that are co-located and expose a single entry point for execution to Controllers. Users can define their own Chains or reuse existing implementations, providing flexibility in how AI tasks are structured and executed.
Evaluators are responsible for assessing the quality of Skills or Chains at runtime. They can rank and select the best responses or identify responses that meet specific quality thresholds. This evaluation process is crucial for maintaining the reliability and accuracy of the AI system's outputs.
Filters in Council serve to refine the responses given back to the controller, ensuring that the final output meets the desired criteria and standards.
Key features of Council include:
Council represents a significant step forward in the development of generative AI applications, offering a framework that combines flexibility, scalability, and control. Its open-source nature and focus on production readiness make it a valuable tool for organizations looking to deploy sophisticated AI systems in real-world environments.