Have you ever looked at all the new terminology around AI and wondered what you actually need?
RAG, embeddings, vector databases, agents, memory, MCP, guardrails, reranking, multi-agent orchestration. Every few weeks there seems to be another term added to the list.
The confusing part is that most architecture diagrams make it look as if you need all of them.
You don’t.
Each of these components solves a different problem. If you add them before you have that problem, you do not get a more production-ready platform. You get more infrastructure, more latency, more cost, and more ways for the system to fail.
So instead of starting with a finished architecture and explaining all its boxes, I want to build one progressively. We will begin with the smallest useful AI application, ask what breaks, and add a component only when a real requirement earns it.
That is how I would review this architecture in practice.
Start embarrassingly simple
Suppose I am building an internal assistant. A user types a question, the application sends it to a foundation model, and the model returns an answer.
User → Application → LLM → Response
That is not a toy architecture. For some use cases, it is the right architecture.
If the task is rewriting text, extracting fields from content supplied in the request, classifying a support ticket, summarizing a document the user has just uploaded, or brainstorming an idea, I may not need retrieval, tools, memory, or an agent loop. I need a clear prompt, a suitable model, basic safety controls, and a reliable application around it.
On AWS, I could put the application behind Amazon API Gateway, run the application logic in AWS Lambda or containers, and call a model through Amazon Bedrock. But those are implementation choices. The architectural decision came first: this request can be completed using the information already in the prompt and the model’s general capability.
Then somebody asks a perfectly reasonable question:
“What is our company’s refund policy?”
Now the simple architecture has a problem.
The model does not know my company
A foundation model may know what refund policies generally look like. It does not reliably know my current policy, which version applies in Luxembourg, whether enterprise customers have different terms, or what changed last Tuesday.
I could paste the policy into the prompt. If there are three short policies and they rarely change, that may be enough. There is no prize for using a vector database when a few pages of context solve the problem cleanly.
But what if the company has thousands of policies, product manuals, contracts, support articles, architecture decisions, and internal procedures? What if they change every day? What if different users are allowed to see different documents?
Now I need a way to find the relevant company information at request time and place it in the model’s context. This is where retrieval-augmented generation, or RAG, enters the design.
RAG does not teach the model my company permanently. It retrieves relevant evidence for this particular question and lets the model answer from that evidence. Done well, it also gives me citations, document lineage, freshness, and a place to enforce access filters.
The architecture becomes:
User → Application → Retrieve relevant knowledge → LLM → Grounded response
But I would pause here, because “the model needs company data” does not automatically mean “put everything into a vector database.”
Do I actually need RAG?
The first question is not which embedding model to use. It is what kind of question I am answering and where the correct answer lives.
Consider these five questions:
“What is our refund policy?”
“What was revenue last quarter?”
“Where is order 18423 right now?”
“Show me incidents containing error code PAY-407.”
“Which applications depend on the service affected by this incident?”
They all require company data, but they are not the same retrieval problem.
The refund policy is unstructured document knowledge. Semantic retrieval may be exactly right.
Revenue is structured and aggregatable. I want a governed SQL query against a warehouse such as Amazon Redshift, not a semantically similar paragraph from a financial report.
The current order status is live operational state. I want an API call to the order system, not an embedding created during last night’s ingestion job.
The error code is an exact token. Lexical search may be more dependable than semantic similarity.
The service-dependency question is about relationships and traversal. A graph representation may answer it more naturally than isolated text chunks.
This distinction matters because an LLM can produce a plausible answer from the wrong source. A vector search over a quarterly report might find text near the word “revenue,” but it is a poor substitute for a controlled calculation over the underlying data. An embedded order record might look relevant while already being stale.
My rule is simple: use the system that is authoritative for the shape and freshness of the answer.
Documents and narrative knowledge
Policies, manuals, contracts, research, tickets, and design documents are good candidates for document retrieval. Their meaning is mostly carried in language, users will ask questions in many different ways, and the answer often needs supporting passages.
This is the natural home of RAG and vector search, usually combined with metadata and keyword search.
Structured data and analytics
If the question asks for a count, total, trend, filter, join, or aggregation, I prefer SQL over semantic retrieval. A model can help translate an approved question into a query, but the database should compute the answer.
On AWS, this could mean Amazon Redshift, Amazon Athena, Amazon Aurora, or another governed data platform. I would expose a constrained query tool or semantic layer rather than give the model unrestricted database access.
Live operational data
Order status, account balance, inventory, shipment location, deployment status, and current permissions belong in operational systems. The assistant should call a narrowly defined API or tool that returns the current state.
This is where an AI system begins to need tool use, even if it is not yet doing anything particularly agentic.
Exact identifiers and lexical search
Product SKUs, invoice numbers, policy IDs, stack traces, people’s names, and error codes often need exact matching. Embeddings are good at meaning, not guaranteed exactness.
Traditional keyword search is still valuable. In many enterprise systems, the best retrieval is hybrid: semantic search for meaning plus lexical search for exact terms.
Relationships and graphs
Questions such as “What depends on this?”, “Who approved that?”, or “How are these customers connected?” may require multi-hop relationship traversal. A graph database such as Amazon Neptune may be the right source, or a graph query may complement retrieved documents.
This does not mean every platform needs a graph. It means I should not flatten a relationship problem into text just because the model speaks text.
Do I need a vector database?
Only after deciding that semantic retrieval is useful would I introduce embeddings and a vector-capable store.
An embedding converts content into a numerical representation. Content with similar meaning tends to land near other similar content in that numerical space. When a user asks a question, I embed the question, find nearby chunks, and give the best candidates to the model.
That is useful because the user might ask, “How long do I have to send something back?” while the policy says, “Eligible goods may be returned within 30 calendar days.” Keyword overlap is limited, but the meaning is close.
Still, a vector store is not a source of truth. It is an index. I want the original document in a governed source, a repeatable ingestion process, and enough metadata to trace every retrieved chunk back to its source and version.
On AWS, three options I would seriously consider are Amazon S3 Vectors, Amazon Aurora PostgreSQL with pgvector, and Amazon OpenSearch Serverless. I would not choose between them by asking which one is “best.” I would ask what kind of retrieval system I am operating.
S3 Vectors: when scale and economics matter more than search sophistication
S3 Vectors is attractive when I have a large vector corpus, relatively straightforward similarity retrieval, and I want an operationally simple, storage-oriented service with S3 economics.
I would consider it for large knowledge collections where vectors are the main access pattern and I do not need a broad search engine beside them. It can be a strong fit for managed RAG, especially when keeping the architecture simple matters more than advanced lexical search, complex ranking, or frequent transactional joins.
The trade-off is capability. If I know I need rich full-text search, sophisticated hybrid relevance, aggregations, or search-heavy operational tuning, I would look beyond the simplest vector layer.
Aurora PostgreSQL with pgvector: when vectors belong beside relational data
Aurora PostgreSQL is compelling when the application already depends on PostgreSQL or when vector search must live close to relational entities, transactions, and SQL filters.
Imagine product embeddings stored beside product records, tenant IDs, availability, and business attributes. A single relational platform may be easier to govern and operate than introducing a dedicated search service. The team also gets familiar SQL tooling and can combine similarity with structured conditions.
The trade-off is that I am asking a relational database to serve a search workload. That can be entirely reasonable at moderate scale, but I would test index behavior, filtering, write volume, recall, latency, and the effect on the database’s primary workload. Familiarity is not a capacity plan.
OpenSearch Serverless: when retrieval is really a search problem
I would lean toward OpenSearch Serverless when the corpus needs strong lexical and vector search together, flexible metadata filtering, search analytics, and more control over relevance.
This is often my preference for a broad enterprise knowledge experience because enterprise search contains exact codes, names, product terms, and acronyms alongside natural-language questions. Hybrid retrieval is not a future enhancement there; it is part of the core requirement.
The trade-off is more search complexity and potentially more cost than a simpler vector store. OpenSearch gives me more knobs because search relevance sometimes needs more knobs. I should choose it because I need those capabilities, not because it makes the architecture diagram look mature.
So my decision would look roughly like this:
Choose S3 Vectors for large, cost-sensitive vector collections with relatively simple retrieval needs.
Choose Aurora PostgreSQL with pgvector when vector similarity is tightly connected to relational application data and the expected scale fits the database design.
Choose OpenSearch Serverless when hybrid enterprise search, exact matching, filtering, and ranking flexibility are central requirements.
Amazon Bedrock Knowledge Bases can provide a managed ingestion and retrieval layer over supported vector stores. I like that option when it reduces undifferentiated plumbing, but it does not choose the data model, chunking strategy, security metadata, or evaluation criteria for me.
Why a good model can still produce bad answers
Once a vector store exists, it is tempting to declare RAG complete. Documents go in; answers come out.
In practice, retrieval quality becomes part of model quality.
If the system retrieves an irrelevant paragraph, omits the clause that changes the answer, or returns an old version of the policy, the model is being asked to reason from bad evidence. Replacing it with a more expensive model may produce a more fluent wrong answer.
Chunking is one of the first places I would look.
Chunking is not a token-count ritual
People often ask for the correct chunk size as if there is one number. There is not.
The question I prefer is: what is the smallest retrievable unit that still contains enough meaning to answer correctly?
A policy may divide naturally by section. A troubleshooting guide may need a symptom, cause, and resolution kept together. A contract clause may depend on definitions several pages earlier. A table may become meaningless if its headers are separated from its rows. Source code should respect functions, classes, and repository structure rather than arbitrary paragraphs.
Fixed-size chunks are simple and can be a good baseline. Overlap can preserve context across boundaries, although too much overlap creates duplicate results and wastes context. Structure-aware chunking uses headings, paragraphs, or document elements. Hierarchical retrieval can find a precise child passage while returning enough parent context to understand it. Semantic chunking can follow topic changes, but it adds processing and should earn its complexity through measured improvement.
I also want useful metadata attached during ingestion: source, document version, section, timestamp, tenant, region, product, confidentiality level, and access-control attributes. Metadata helps relevance, but more importantly it lets me filter before content reaches the model and explain later why a passage was retrieved.
If retrieval is bad, changing the foundation model is often the most expensive way to avoid fixing the real problem.
Hybrid search and reranking
Vector search answers, “What is similar in meaning?” Lexical search answers, “What contains these words or tokens?” I usually want both.
Suppose the question is, “Does PAY-407 affect delayed card captures?” Semantic search may retrieve excellent material about payment failures while missing the exact error-code page. Keyword search may find every occurrence of PAY-407 but rank an obsolete release note above the current runbook.
A hybrid approach generates candidates from both methods and combines their rankings. Then I may add a reranker that evaluates the question against each candidate more deeply and moves the most relevant evidence to the top.
The retrieval path becomes:
Question
→ identity and metadata filters
→ semantic candidates plus lexical candidates
→ merged candidate set
→ reranking
→ context assembly
→ generation with citations
Reranking is not free. It adds latency and cost. I use it when the first-stage search can retrieve the right material but does not rank it reliably enough. I would test how many candidates to retrieve, how many to rerank, and how many to send to the model rather than copying defaults from a demo.
Notice where identity filters appear in that flow. I do not retrieve confidential content and then instruct the LLM not to mention it. Unauthorized content should not enter the context in the first place.
Long context vs RAG vs fine-tuning
Modern models can accept a lot of text. Why build retrieval at all? Why not put every relevant document into the prompt?
Sometimes I would.
If the task is to review one contract, compare five proposals, analyze a bounded case file, or summarize a known set of documents, long context may be the simplest answer. The material is already identified, and the model benefits from seeing it together.
Long context becomes less attractive when the knowledge base is large, changes frequently, contains different access levels, or would be resent on every request. More context also does not guarantee more attention. Irrelevant material can dilute the important evidence, increase latency and cost, and make citations harder to control.
I see long context as a way to carry selected information, not as a replacement for deciding which information is relevant and permitted.
And what about fine-tuning?
Fine-tuning solves a different problem. I consider it when I want to change behavior: a consistent output structure, specialized classification, a particular task pattern, or performance on a repeated domain-specific behavior that prompting alone does not handle well.
I do not fine-tune a model to remember that the travel policy changed yesterday. Facts that change belong in systems that can change independently of the model.
My practical distinction is:
Use long context when the relevant information is bounded and already known for this request.
Use RAG when the relevant information must be found inside a larger, private, or changing body of knowledge.
Use fine-tuning when the problem is how the model behaves, not which current facts it knows.
A real platform may use all three. The mistake is treating them as interchangeable ways to “make the model smarter.”
When does this become an agent?
So far, the application receives a question, retrieves information, calls a model, and returns a response. It may use tools in a fixed workflow, but I would not automatically call it an agent.
For me, it becomes agentic when the model participates in deciding what to do next.
Perhaps the user says:
“Find the customer’s delayed order, check whether it qualifies for compensation, and open a support case if it does.”
The system may need to identify the customer, call the order API, retrieve the compensation policy, compare the facts with the rules, ask for missing information, create a case, and report the result. The next step depends on what the previous step returns.
That creates a loop:
Understand the goal → choose a tool → call it → observe the result → decide the next step → stop or continue
The model is no longer only composing an answer. It is helping control a workflow.
That is useful, but it is also the moment risk increases. A wrong paragraph is one thing. A wrong refund, deployment, email, or account update is another.
I want the loop to be bounded: a small tool set, clear schemas, limited credentials, timeouts, retry rules, step and cost limits, idempotency where actions may repeat, and human approval for sensitive operations. For deterministic business processes, I may keep the workflow in AWS Step Functions and use the model only for the ambiguous steps. Not every branching workflow should be handed to an agent.
One agent or many?
I would start with one agent almost every time.
One agent with five well-designed tools is easier to understand than a supervisor, researcher, planner, database agent, policy agent, and reviewer exchanging generated messages. Multiple agents add context handoffs, prompts, latency, token cost, failure modes, and another layer of evaluation.
Multi-agent architectures should solve a complexity problem, not create one.
I would consider separate agents when there is a real boundary: distinct security domains, independently owned business capabilities, very different tool sets, contexts that interfere with each other, specialist reasoning that deserves separate evaluation, or work that can genuinely run in parallel.
Even then, I prefer explicit contracts. A coordinator owns the user goal and delegates bounded tasks. Each specialist receives a defined input, has a narrow capability, and returns a structured result. I do not want an open group chat between agents and a hope that consensus emerges.
AWS gives me several possible implementations, including Amazon Bedrock Agents, custom orchestration using Bedrock model APIs, AgentCore Runtime for running agents, and Step Functions for durable workflow coordination. I would choose among them after deciding how much autonomy, durability, and control the use case requires.
Memory is not RAG, and neither is session context
The word “memory” gets used for almost everything an AI system can retrieve. I find it more useful to separate four things.
The current prompt and recent conversation are session context. They help the model understand what “it” refers to in the next sentence.
A compact summary or checkpoint that lets a conversation continue later is conversational memory.
A durable preference such as “this user prefers architecture before code” may be long-term user memory, if there is a legitimate reason and permission to retain it.
The company refund policy is not memory. It is governed organizational knowledge and belongs in RAG or another authoritative source.
The status of order 18423 is not memory either. It is operational state and belongs in the order system.
This separation prevents a subtle but serious failure: remembered content becoming treated as business truth. A user might once say, “I think this customer is on the premium plan.” That statement should not silently become a durable fact used to authorize compensation later.
I would keep short-term context scoped to the session, summarize when histories become too large, and write long-term memory only through an explicit extraction process. Durable memories need a purpose, owner, tenant and user scope, retention period, deletion path, and rules for sensitive data.
Amazon Bedrock AgentCore Memory can implement managed short- and long-term memory for agents. I would use it after deciding exactly what the application is allowed to remember. “Save every conversation forever” is not a memory strategy.
How should the agent reach enterprise systems?
As the tool set grows, direct integrations become awkward. Every agent needs to understand authentication, endpoint details, schemas, retries, and errors for every API. Tools get duplicated across teams. A change to a backend leaks into many prompts and applications.
I want a controlled tool plane between agents and enterprise systems.
Model Context Protocol, or MCP, provides a standard way to expose tools and context to AI applications. It can reduce custom integration work and make tools portable across compatible clients. But MCP is a protocol, not a security boundary by itself. A tool exposed through MCP still needs identity, authorization, validation, auditing, and operational controls.
On AWS, Amazon Bedrock AgentCore Gateway can provide a managed gateway for turning APIs, Lambda functions, and services into agent-accessible tools, including MCP-compatible access. That gives me a central place to publish tool definitions, manage connectivity, and reduce one-off integrations.
I would keep tool contracts narrow and business-oriented. “IssueEligibleRefund” is safer than “ExecuteArbitrarySQL.” “GetOrderStatus” is easier to authorize and audit than a generic HTTP tool. Descriptions should be clear enough for tool selection, but security must live in code and policy, not in the wording of the description.
For actions with side effects, I also want idempotency keys, explicit confirmation where appropriate, validation of every parameter, bounded output, sensible timeouts, and a record of the caller, decision, request, and result.
Authentication is not authorization
If the platform knows the user is Abbas, authentication has worked. That does not answer whether Abbas may read this contract, query this customer, approve this refund, or invoke this production tool.
Authorization has to survive the whole request path.
The user signs in. The application receives an identity and tenant context. Retrieval applies document-level or chunk-level access filters. Tool calls use delegated or scoped identity. The downstream system checks permission again. Sensitive actions may require approval. Every layer records enough context for an audit.
I would avoid giving the agent one powerful service role and trusting the prompt to keep users separated. The model should not become an authorization proxy.
On AWS, authentication might use Amazon Cognito or an enterprise identity provider through IAM Identity Center. Workloads can use IAM roles and temporary credentials. AgentCore Identity can help agents access resources and third-party services using managed identity mechanisms. Authorization can be enforced through IAM, application policy, API Gateway authorizers, resource policies, Amazon Verified Permissions, or the target system’s own access rules.
The exact combination depends on the organization. The invariant is more important: permission is evaluated deterministically using trusted identity and request context.
The model should be allowed to request an action. It should not be the final authority on whether that action is allowed.
Guardrails are not deterministic security
Guardrails are useful. I may use them to detect harmful content, block disallowed topics, reduce exposure of personal data, filter model input and output, or require grounding behavior. Amazon Bedrock Guardrails provides managed capabilities for several of these controls.
But a guardrail is not a replacement for authorization, schema validation, transaction limits, or workflow approval.
“Never refund more than €500” should be enforced in the refund service or policy layer, even if the prompt says the same thing. “Only HR can retrieve salary documents” should be enforced before retrieval. “Production changes require approval” should be a workflow control. “This parameter must be one of these values” should be validated against a schema.
I think of guardrails as controls around probabilistic content. I use deterministic mechanisms for deterministic rules.
I also treat retrieved documents and tool responses as untrusted input. A document may contain text telling the agent to ignore its instructions. A web page may attempt to redirect a tool call. Prompt-injection defenses therefore include source trust, content isolation, minimal privileges, output validation, and confirmation around consequential actions. A single content filter cannot carry that responsibility.
How do I debug this thing?
A traditional request might call one service and return one error. An agentic request may retrieve six chunks, choose three tools, retry one call, receive a malformed result, update its plan, and produce an answer that sounds confident.
If I record only the final text, I have almost no chance of understanding what happened.
I want a trace for the full journey: request ID, user and tenant context, selected model and configuration, prompts or prompt versions, retrieval queries, filters, retrieved chunk IDs and scores, reranking results, tool selections, validated inputs, tool outputs, latency, token usage, errors, retries, policy decisions, approvals, and the final response.
That does not mean logging secrets or every raw piece of sensitive content. Observability needs redaction, access controls, retention, and clear separation between operational telemetry and protected business data.
On AWS, I can use Amazon CloudWatch for logs, metrics, dashboards, and alarms; AWS X-Ray or OpenTelemetry-compatible tracing for distributed calls; AWS CloudTrail for relevant API activity; and AgentCore Observability for agent-focused telemetry. The product choice matters less than maintaining correlation across the whole request.
The metrics I care about are not just infrastructure metrics. I want time to first token, total latency, model and retrieval cost, tool error rate, retries, abandoned runs, human escalation, retrieval hit quality, citation coverage, and task success. A healthy server can still host a useless agent.
Evaluation is not a final test phase
How do I know whether a new model, prompt, embedding model, chunking strategy, or reranker improved the system?
Reading ten pleasant answers is not enough.
I would build an evaluation set from real, representative tasks. It should include common questions, difficult questions, exact identifiers, ambiguous requests, stale documents, permission boundaries, missing information, unsafe requests, tool failures, and cases where the correct behavior is to ask, refuse, or escalate.
Then I would evaluate the layers separately.
Did retrieval find the right evidence? Useful measures include recall at K, ranking quality, and whether the authoritative passage appeared at all.
Was the answer grounded in that evidence? I care about factual correctness, citation support, completeness, and whether the model admitted when the evidence was insufficient.
Did the agent complete the task? I want the right tools, valid parameters, correct sequence, appropriate stopping behavior, and no unauthorized side effects.
Did the security controls hold? I test cross-tenant access, prompt injection, excessive tool scope, sensitive-data leakage, and attempts to bypass approval.
Some checks can be deterministic. Some need human review. Model-based judges can help at scale, but I would calibrate them against human judgments and avoid letting one model’s opinion become the only definition of quality.
Amazon Bedrock evaluation capabilities can help assess models and RAG systems, and AWS services can support the surrounding test pipeline. Whatever tooling I choose, evaluations should run before changes reach production and continue against sampled production behavior. Every major incident should be a candidate for a new regression test.
Only now would I draw the production architecture
We began with a user and a model. Every additional box now has a reason to exist.
At the front, users authenticate through the organization’s identity system and reach a web or application interface. Amazon CloudFront, AWS WAF, Amazon API Gateway, and either Lambda, Amazon ECS, or Amazon EKS are reasonable implementation options depending on the application and operating model.
The application or agent runtime owns the request. It calls foundation models through Amazon Bedrock. I would use one primary agent or orchestrator for the first use case, with explicit limits and a durable workflow such as Step Functions where the business process needs guaranteed state transitions.
For company documents, an ingestion path takes governed sources, parses and chunks them, adds security and business metadata, creates embeddings, and indexes them. Amazon S3 remains the durable document source. Bedrock Knowledge Bases can manage much of this path. The selected vector store is S3 Vectors, Aurora PostgreSQL with pgvector, or OpenSearch Serverless based on the retrieval requirements we already discussed.
At query time, a retrieval layer applies tenant and access filters before running semantic and, where needed, lexical search. A reranker improves the candidate order. The model receives a small set of permitted evidence with citations.
Structured analytics stays in Redshift, Athena, or the relevant database behind a constrained query tool. Live state remains behind domain APIs. Relationship questions can reach a graph such as Neptune when the use case genuinely needs traversal.
The agent reaches those capabilities through a controlled tool plane. AgentCore Gateway can expose narrow tools and MCP-compatible interfaces. Identity propagates through the request, and every downstream system enforces its own authorization. Sensitive actions pass through deterministic policy and, when needed, human approval.
Session context and deliberately selected long-term memory can use AgentCore Memory. Company truth does not go there. Transaction state does not go there.
Bedrock Guardrails and application-level content controls reduce unsafe input and output. They sit alongside, not instead of, IAM, Verified Permissions, API authorization, schema validation, transaction rules, and workflow approvals.
Finally, every retrieval, model call, tool invocation, policy decision, and outcome emits correlated telemetry. Evaluation datasets and regression tests feed the delivery pipeline, while production sampling shows where the offline tests are incomplete.
What I would actually deploy
If I had to build the first production version on AWS, I would resist the urge to deploy the final diagram all at once.
I would start with one valuable use case and one agent at most. The application would call a Bedrock model through a small orchestration service, probably on Lambda for a short-lived workload or ECS for a longer-running one. I would put explicit request, step, time, and cost limits around it.
If the use case needs company documents, I would keep the originals in S3 and use Bedrock Knowledge Bases unless its managed workflow blocked a real requirement. I would begin with a simple chunking baseline, strong metadata, citations, and a test set before spending time on clever retrieval.
For the vector store, I would choose based on the workload rather than standardize prematurely. S3 Vectors would be my candidate for a very large, cost-conscious semantic corpus. Aurora PostgreSQL with pgvector would be my candidate when the application already lives in PostgreSQL and vectors need relational joins and filters. For a general enterprise knowledge assistant with exact terms, acronyms, and document search, I would probably choose OpenSearch Serverless because I expect hybrid retrieval to matter.
I would keep Redshift and operational systems outside the document index. The agent would receive a few narrow tools: perhaps GetOrderStatus, QueryApprovedSalesMetrics, SearchCompanyKnowledge, and CreateSupportCase. I would expose them through AgentCore Gateway when the shared gateway and MCP model add enough value; for a very small first release, direct internal tool calls may be simpler.
I would propagate user identity, filter retrieval before generation, and make every target system authorize the operation. Read-only tools would come first. High-impact writes would require deterministic validation and usually human confirmation.
I would use short-term session context immediately, but I would postpone long-term memory until I could name the exact information worth retaining and the deletion policy for it. I would not add multiple agents until one agent had become measurably constrained by a real domain, security, scaling, or ownership boundary.
Before launch, I would create an evaluation set with real questions and expected evidence, including questions the system must not answer. I would trace retrieval, prompts, tool calls, and policy decisions from day one. Those are much harder to bolt on after users have already learned not to trust the system.
Most importantly, I would leave parts out.
If the use case does not need actions, I would not build an agent. If the knowledge fits naturally in the request, I would not build RAG. If semantic search adds nothing, I would not create embeddings. If one agent is understandable, I would not create five. If a deterministic workflow solves the process, I would keep it deterministic.
Production-grade does not mean having the most components. It means being able to explain why each component exists, what boundary it enforces, how it fails, how it is measured, and how it can be removed when the requirement changes.
The model is only one part of that system. Often, it is not even the hardest part.
Mohamed Abbas 
Leave a comment