RAG-powered website assistant

01 / Challenge

Challenge

Visitors repeatedly ask questions that are already answered across a website, its FAQs, documents, policies, or service pages. Nobody reads all of it, and a search box rarely bridges the gap between how a question is phrased and how the answer is written.

A general-purpose language model on its own is not the fix. It will happily produce a confident answer that the organization never approved, quote a price that does not exist, or promise a service that is not offered.

So the problem had three parts: answers had to be grounded in approved content, the assistant had to know the edges of what it should talk about, and some conversations needed to end with a person rather than another generated reply.

02 / Design approach

Design approach

Ground first, generate second. Every answer starts with retrieval over the organization’s own content; the model is only asked to phrase what was found. If nothing relevant is retrieved, the assistant says so and points to a human channel instead of improvising.

Keep the model on a short leash. The persona is deliberately narrow, the conversation memory is bounded, and off-topic requests are detected and limited so the assistant does not drift into general chat.

Treat hand-off as a first-class path, not a failure. Whether the visitor asks to speak to someone, the question is outside the content, or the business is closed, the assistant offers the channels a person actually watches and captures the enquiry for follow-up.

Make it deployable by a small team. One script tag embeds the widget, the interface adapts to the host site’s colours, and configuration lives in an admin area rather than in code.

03 / Technical solution

Technical solution

Ingestion: a same-domain crawler collects pages within explicit limits (page count, depth, delay between requests, robots.txt respected), and uploaded PDF, Word, text, and JSON documents are parsed to plain text. Content is split into overlapping chunks sized by estimated token count.

Indexing: each chunk is embedded through the OpenAI embeddings API in batches and stored in a managed vector index keyed by document and chunk. Large documents are processed as background jobs, and document status (pending, processing, processed, failed) is visible in the admin area. An earlier iteration stored packed float vectors in MySQL and computed cosine similarity in application code — simple and dependency-free, but it does not scale past a modest number of chunks.

Retrieval: the visitor’s question is embedded and the top matching chunks above a configurable similarity threshold are collected, with a bounded fallback so the model still receives the closest passages when nothing clears the bar. Retrieved passages are labelled with their source so replies can name where an answer came from.

Generation: a server-side prompt combines the organization’s persona, scope rules, the retrieved passages, and a bounded window of recent messages. Answers are returned with the list of sources used, and an optional citation mode instructs the model to reference source names inline.

Operations: rate limiting per site and visitor, origin checks against the site’s allowed domains, deterministic intent tagging so an operator can triage conversations at a glance, usage tracking per month, and an admin area for content, conversations, enquiries, and settings.

04 / Safety and privacy decisions

Safety and privacy decisions

Scope control: the model is instructed to tag clearly unrelated questions; after a small allowance the assistant stops calling the model for off-topic messages and returns a fixed redirect. This keeps cost and behaviour predictable without a second classifier.

Data retention: chat data (conversations, messages, captured enquiries) has its own retention window with an automatic, capped purge, and a separate window for business records so a privacy setting can never silently delete operational history. A subject-erasure routine removes a visitor’s data across tables on request.

Secrets and access: provider keys are stored encrypted at rest, admin passwords use a modern memory-hard hash, admin forms carry CSRF tokens, and public endpoints validate the requesting origin.

Honest limits: there is no explicit prompt-injection filter in this build yet — mitigation relies on the narrow persona, retrieval-only context, bounded memory, and server-side controls. Content moderation of visitor messages is also not part of this build. Both are listed under extensions rather than claimed.

05 / Human hand-off

Human hand-off

A “talk to a person” request is answered with the channels the business actually staffs — a phone number, a messaging link, or a callback request that becomes an enquiry — rather than a promise that someone is watching a dashboard.

Live chat is offered only while the business is open. Outside those hours the assistant degrades to contact capture, because telling a visitor that someone will be with them shortly at two in the morning burns the trust the assistant just earned.

When retrieval finds nothing relevant, the assistant says so plainly and offers the human route. An unanswered question that leads to a real follow-up is a better outcome than a fluent guess.

Fit

Suitable use cases.

• Service businesses whose visitors ask about hours, coverage, process, and pricing already published on the site
• Organizations with policy or documentation libraries that people would rather ask than read
• Teams that want a first-line assistant but need every enquiry to reach a person
• Sites that need answers limited to approved content for accuracy or compliance reasons

Stack

Technologies.

• PHP 8
• MySQL / MariaDB
• OpenAI API (chat and embeddings)
• Managed vector index (Pinecone)
• Retrieval-augmented generation
• Background job processing
• JavaScript widget
• REST endpoints
• Apache / shared hosting

Notes

Lessons and engineering trade-offs.

Retrieval quality is a content problem first.

Chunk size, overlap, and what gets crawled matter more than model choice. Poorly structured source pages produce poorly grounded answers.

A threshold needs a fallback — and a warning.

Rejecting everything below a similarity bar leaves the model blind; always passing the closest passages risks loose answers. The build passes a bounded fallback and instructs the model to say when the content does not cover a question.

In-database vectors are fine until they are not.

Computing cosine similarity in application code over MySQL rows was the simplest possible start. Moving to a managed index was worth it once content grew.

The hand-off design is the product decision.

Most of the trust the assistant earns is decided by what happens when it cannot help. Degrading to real channels beat any wording change.

Not everything should be claimed.

Injection hardening, moderation, and a real evaluation set are the next pieces of work, and they are listed as such rather than implied.