Postman, the popular API development platform, has detailed its approach to integrating an AI-powered Agent Mode for its vast user base of 40 million developers, leveraging Amazon Bedrock for robust scalability and flexibility. The company shared insights into the architectural patterns and engineering challenges overcome in making its mature product accessible to an AI agent, moving beyond initial assumptions about prompt design and model quality.
The Chenab Times has learned that Postman’s journey with Agent Mode involved addressing deeper complexities related to embedding AI into a product with established interface conventions, a broad functional scope, and specialized concepts. Key to their success were architectural patterns focused on controlling tool sprawl, enabling schema-based data access, and prioritizing context over raw capability as the primary constraint.
Agent Mode, designed to facilitate AI-native workflows across API testing, documentation, discovery, and implementation, operates on Amazon Bedrock. This choice allows Postman to manage variable and latency-sensitive demand from its global developer community without managing its own model-serving infrastructure. The integration enables Postman to scale its production workload while retaining control over model selection, throughput, geographic processing, and cost.
Developing Agent Mode for a Mature Product
Postman’s Agent Mode represents a new paradigm for developers to interact with the platform using AI. Over its 11-year evolution, users learned to navigate Postman’s interface through visual cues and sequential actions. Adapting this user awareness for an agent, which reasons over data rather than screen interactions, required a fundamental re-evaluation of the product’s APIs, user experience, and knowledge distribution.
The architecture of Agent Mode integrates client-side tools within the Postman application, which represent the agent’s final actions, such as opening requests or modifying settings. Server-side tools are also employed for functions like web searches. A critical component is the knowledge base, which utilizes a Retrieval Augmented Generation (RAG) approach to provide relevant product information to the agent. This system-wide behavior, including agent proactivity and communication of uncertainty, is defined by generic agent instructions.
To manage the complexity of Postman’s extensive product surface—spanning multiple request protocols, mock servers, monitors, documentation, and more—the system dynamically selects relevant knowledge articles based on user queries and available context. This approach ensures the agent remains lightweight by default, while providing necessary depth when required, with the knowledge base evolving alongside the application.
Architectural Patterns for AI Integration
Postman identified several key architectural challenges and their solutions:
Handling Tool Sprawl
Initially, Postman favored highly atomic tools, which proved effective for control but led to slow user experiences for multi-step workflows and increased tool-selection errors as the toolset grew. The current architecture dynamically scopes tools based on task and context, presenting the model with only relevant options. This significantly reduces the probability of errors and improves efficiency.
A critical lesson learned is to treat the tool catalog as part of the context budget. By dynamically scoping tools per task and decoupling agent capabilities from the UI’s current state, Postman enhances agent effectiveness.
Exposing Schema-Based Reads
For structured data sources like the API Catalog, Postman consolidated multiple narrow views into single query tools. By providing the agent with schema-aware access to data, it can generate complex queries, reducing the need for numerous single-purpose tools. This shifts the engineering focus from building tools for every question to modeling data effectively.
The takeaway here is that for well-structured data, offering schema-aware read access to a query engine is more scalable than creating a proliferation of read tools. This approach trades tool count for better data modeling.
Context as the Primary Bottleneck
Postman found that incomplete or incorrect context, rather than a lack of tools, was the more frequent cause of agent failures. Context encompasses the agent’s understanding of the user’s current location within Postman, active entities, and established state. To address this, Postman developed dedicated context handlers that distill each entity into information relevant for agent reasoning, moving beyond simply serializing interface data models.
Managing the context window as a scarce resource is crucial. This involves deliberate context engineering, creating purpose-shaped handlers, and implementing strategies for truncation and expansion to prevent noise from overwhelming the signal within the model’s limited context budget.
Leveraging Amazon Bedrock for Scalability
The integration of these components culminates in inference calls to foundation models via Amazon Bedrock, which offers several key capabilities for scaling production agents:
Model Flexibility
Agent Mode’s reliance on Amazon Bedrock provides flexibility in model selection, allowing Postman to route workloads to different Anthropic Claude models based on requirements. Faster models can handle high-volume, low-latency interactions, while larger models are used for complex reasoning, offering adaptability without requiring new integrations.
Cross-Region Inference
To manage spiky developer traffic and optimize costs, Postman utilizes Amazon Bedrock’s cross-Region inference. This feature automatically routes requests among defined destination Regions, enabling efficient scaling and throughput management. Postman can select geographic profiles for processing boundaries or global profiles for maximum throughput.
Data Residency and Enterprise Controls
For enterprise customers, geographic processing boundaries are paramount. Geographic inference profiles ensure that Bedrock routing is confined to supported Regions within a specified geography, while AWS affirms that Bedrock does not use prompts or completions for model training. Postman has configured zero data retention for supported models, with availability and behavior being model-dependent.
Prompt Caching
To mitigate avoidable latency and cost associated with reprocessing stable context, Agent Mode employs Amazon Bedrock’s prompt caching. This feature reuses stable prompt prefixes, with tiered checkpoints for core instructions and more variable context, improving efficiency and reducing processing overhead.
Best Practices for Production Agents
Based on Postman’s experience, key best practices for building and scaling agents on Amazon Bedrock include budgeting tools as carefully as tokens, preferring schema-aware data reads, decoupling agent actions from interface state, engineering context deliberately, managing the context window as a scarce resource, ensuring documentation evolves with features, and optimizing inference through routing and caching.
The successful integration of Agent Mode demonstrates how mature products can be adapted for AI agents, leveraging robust cloud infrastructure like Amazon Bedrock to serve a large and dynamic user base. This approach offers a blueprint for other organizations looking to implement advanced AI capabilities within their existing product ecosystems.
The Chenab Times News Desk