Building an LLM Assistant From Scratch vs Implementing AI Enable Kit
A practical, vendor-neutral field guide
Where the real time goes when a team sets out to build an AI assistant from zero — the practices that pay off, the effort most teams underestimate, a realistic timeline, and how much of that work disappears when you implement AI Enable Kit instead.
The core truth
Calling the language model is roughly 10% of the work. The other 90% is the infrastructure around it.
Across almost every production codebase the pattern is the same: the agent loop — the part that talks to the model — is a few hundred lines, while the service that surrounds it is thousands. That ratio is the lesson. A team building from scratch tends to budget as if the model call is the project, then discovers in month two that authentication, security, state, and streaming are the actual product.
Where the time actually goes
Ranked by how much they surprise a team that has only built a prototype. The model integration is deliberately not on this list — it is the part that goes quickly.
- 01Authentication & multi-tenancy
As soon as more than one customer — or more than one end user — shares the assistant, you have to answer a hard question on every single request: who is this conversation for, and what are they allowed to see? That usually grows into several distinct credentials: a key that authorises one application to the service, a short-lived token that identifies the end user a conversation is for, and a separate token for whoever administers the account.
Add per-tenant model-provider keys stored encrypted at rest, scoped so one customer can never act as another, and you are solving a genuine distributed-systems problem — not building a feature. This is the single most underestimated area, every time.
- 02Security & guardrailsOngoing
This is the code that exists in no tutorial but is the entire difference between a demo and something you can put in front of a customer’s data: filtering what the model is allowed to receive and return, allow-listing which fields of a record ever reach it, masking the rest, and sandboxing any tool the model can call so a hijacked instruction cannot reach beyond a narrow, read-mostly surface.
Treat prompt injection as a blast-radius problem rather than something you solve once. It is never "finished" — it is a posture you maintain as the surface grows.
- 03Streaming end to end
Streaming tokens one at a time is easy in isolation and genuinely miserable across a full stack — especially when a turn has to pause mid-stream for a tool to run and then resume on the same connection. Server-sent events through a relay, reconnection, and partial-message handling are where the subtle, hard-to-reproduce bugs live.
It is also nearly impossible to retrofit. A request/response design has to be rewritten to stream, so decide it on day one.
- 04Conversation state done right
Trimming history so the context stays within limits, without orphaning a tool result from the turn that produced it, is finickier than it looks. So are token and step caps, and the move from an in-memory store (fine for a prototype) to an external one that survives restarts and works across replicas.
State is where correctness bugs hide. Keep the service itself stateless and push conversation state into an external store from early on.
- 05Tool / function calling
Turning real capabilities — APIs, lookups, actions — into tool schemas the model can call, validating and encoding the parameters so a call can’t be coerced somewhere it shouldn’t go, executing them, and feeding the results back into the loop: this is a substantial build in its own right.
If live tool-calling is how your assistant stays grounded in current data, this is harder, not easier, than a document retrieval pipeline — and it is where a great deal of the engineering actually lands.
- 06The embeddable surface
A chat widget that drops into any site is its own multi-week project: style isolation so it can’t be broken by (or break) the host page, a token lifecycle that caches, renews, and retries cleanly, cross-origin handling, per-user conversation storage, and graceful reconnection.
Every channel you add — web widget, mobile, a messaging platform — repeats a version of this integration cost.
- 07ObservabilityMedium
You cannot improve, debug, or price an assistant you cannot see. Usage counters, full transcript replay, per-model token and cost accounting, and a way to capture user feedback are not nice-to-haves — they are how you find out what the assistant is actually doing in production.
Persisting complete transcripts early is cheaper than any evaluation framework and is how you actually debug.
- 08Prompt engineeringContinuous
Assembling the system prompt — base instructions, per-customer configuration, tool usage guidance — dynamically and correctly is ongoing work, not a task you close. It is a continuous tuning loop that lives for as long as the product does.
Best practices
Most of these are cheap to adopt on day one and expensive to retrofit later. That asymmetry is the whole reason to decide them up front.
Decide your grounding strategy first
Retrieval over documents (RAG) and live tool-calling against real APIs are completely different builds. Choose deliberately: tool-calling when answers must reflect live, per-user-authorised data; retrieval when the knowledge is a stable corpus. Don’t default to RAG just because it’s the fashionable pattern.
Keep the client out of the direct model path
The browser or app should never hold provider keys or talk to the model directly. Route everything through your own backend, which relays to the model server-to-server. This is the single most important security decision, and it’s painful to add later.
Abstract the provider from day one
Wrap the model behind a thin interface so you can swap or add providers without touching the rest of the system. Different providers have different quirks (some reject parameters others require); a day-one abstraction keeps those from leaking everywhere.
Stream from day one
Retrofitting streaming into a request/response design means rewriting the transport. Build the streaming path first, even when the first version just echoes tokens.
Default-deny on data exposure
Allow-list exactly which fields ever reach the model and mask the rest. Never "send the whole record and hope." New fields should be invisible to the model until someone explicitly opts them in.
Cap everything
Hard limits on reasoning steps, history length, and tokens are loop protection and cost protection in one. An uncapped agent is an uncapped bill and an infinite loop waiting to happen.
Keep the service stateless; store sessions externally
An in-memory session store is fine for a prototype and a liability in production — it loses state on restart and can’t scale across replicas. Externalise conversation state early.
Persist full transcripts
Cheaper than any evaluation framework in the early days, and the only reliable way to debug and improve. You can’t fix what you can’t replay.
Hold off on heavy evaluation
Lightweight thumbs-up/down feedback plus transcript replay delivers most of the value of a formal eval harness for a fraction of the effort. Reach for automated answer-quality evaluation once you have real traffic telling you where it hurts.
Encrypt tenant secrets at rest
Per-customer provider keys and credentials belong encrypted in storage, decrypted only in memory when a request needs them. This is table stakes the moment you hold a second party’s keys.
A realistic timeline
For a small team of two to three engineers building to production quality — not a demo. Ranges, not promises; the security and multi-tenant phase is the one that most often runs long.
- Phase 0Vertical slice1–2 weeks
One provider, one hard-coded tool, streaming, no auth. Prove the loop end to end.
- Phase 1Agent core3–4 weeks
The tool-calling loop, conversation state with persistence, history / step / token caps.
- Phase 2Multi-tenant & security4–6 weeks
The credential model, encrypted per-tenant keys, field-level redaction, tool sandboxing.
- Phase 3Surface & analytics4–6 weeks
Embeddable widget or channel integration, appearance configuration, usage and transcript views.
- OngoingHardening & tuningIndefinite
Prompt iteration, evaluation, scaling (external state, replicas), continued security work.
The bottom line
Budget roughly three to five months to reach a solid, shippable assistant at production quality — and know that the single biggest risk to that estimate is a team spending week one excited about the model and discovering in month two that authentication, security, and streaming are the real build.
The deceptive part: a flashy demo takes about three days. The three-day demo and the four-month product look identical in a screen recording. Managing that expectation gap is itself part of the job.
The alternative: implementing AI Enable Kit
An afternoon, not monthsEverything this guide calls the hard 90% — authentication and multi-tenancy, security and redaction, streaming, conversation state, tool-calling, the embeddable surface, and observability — is precisely what AI Enable Kit already ships, hardened. Adopting it is a configuration task, not an engineering project. Five steps, each one retiring a burden the pages above described:
- 1
Register your application
Replaces: Authentication & multi-tenancyCreate an app in the console and receive its API token and session endpoint. The whole credential model — an app-to-service token, short-lived per-user session tokens, a separate admin token, and encrypted per-tenant provider keys — is already built and enforced for you.
- 2
Point it at your API
Replaces: Tool-calling & groundingDeclare the REST endpoints the assistant may use; each becomes a tool automatically, with response fields allow-listed and sensitive values masked. There is no retrieval pipeline to build — the assistant stays grounded in your live data, and every call runs with the signed-in user’s own authorisation.
- 3
Choose a model and shape the prompt
Replaces: Provider integration & prompt engineeringPick your provider and model, then add instructions or skills from the console. Swap providers whenever you like — the abstraction, and the per-provider quirks, are already handled.
- 4
Theme it and embed
Replaces: Streaming, state & the embeddable surfaceCopy one script tag into your site. The widget renders inside an isolated shadow root and handles session tokens, token-by-token streaming, conversation history, and reconnection on its own — nothing for you to wire up.
- 5
Watch it work
Replaces: ObservabilityConversations, per-model usage, token and cost accounting, and user feedback are in the console from the very first message. There is nothing to instrument.
What takes a team three to five months to build and harden is, with AI Enable Kit, an afternoon of configuration. The 90% is already done — so your team spends its time on the 10% that is genuinely yours: what the assistant should know, and what it should do.