JYnapse Knowledge Base · Free to read · ~15 min
"Build us an AI assistant" usually has three different projects hiding inside it. They cost wildly different amounts and take wildly different lengths of time — so the first job is working out which one you actually need.
The short answer, if you read nothing else
You are not training anything. You are building a searchable index of your own documents and letting a model answer from what it finds.
That distinction is worth real money: training is a months-long specialist project that teaches a model how to write, and it does not reliably teach it facts. Retrieval is a weeks-long engineering task that does exactly what you described.
And the answer to "will ChatGPT work" is: yes for a pilot, and the platform is genuinely the last decision. Content, sensitivity and volume decide it — pick those first and the platform falls out on its own.
There are four ways to give a model knowledge it did not have. They sound similar and are not remotely alike in cost.
| Approach | What it actually does | Cost & when it fits |
|---|---|---|
| Paste into the prompt | You hand the model the text at the moment you ask. | Free, instant. Fine for one document. Falls apart at scale — and you pay for every word, every time. |
| Retrieval (RAG) | Documents are indexed. The system finds the relevant few passages per question and hands only those to the model. | This is what you described. Weeks, not months. New document means re-indexing one file. Answers can cite their source. |
| Fine-tuning (including LoRA) | Adjusts the model's weights so it writes in a particular style or format, or classifies reliably. | Teaches form, not facts. Needs hundreds of worked examples. Redone whenever things change. Genuinely good for classification. |
| Pre-training | Builds a model from scratch. | Millions of dollars. Nobody in your position does this, or should. |
A note on terms you'll hear: Hugging Face is a repository — think GitHub for models, where open-weight models are downloaded free. LoRA is not a platform but a fine-tuning technique: instead of adjusting all of a model's billions of parameters, you train a small adapter beside it. Cheap, fast, and free to use. Neither is required for what you described.
If you had thousands of past emails already sorted into categories — claim notification, renewal query, complaint — and you wanted that sorting done automatically, fine-tuning would beat any amount of prompting. That is a pattern-recognition job with abundant labelled history.
Answering "what does our wording say about seepage?" is not that job. That is looking something up. Different problem, different tool, and the lookup is far cheaper.
Most organisations exploring AI have at least two separate needs here, and they are not the same project. Worth separating them before anyone quotes a price.
| Build | What it's for | Effort & shape |
|---|---|---|
| Document assistant retrieval / RAG |
Ask questions across your own documents and get an answer with a source. "Which of our wordings exclude gradual seepage?" | Weeks. Index once, query many times. Cost scales with questions asked. This is the pattern most people mean by an “AI assistant”. |
| Workflow automation Make, Zapier, Power Automate |
The same task, on a schedule, in a fixed format. Renewal falls due in 60 days → pull the details → draft the notice → route for approval. | Days. Drag-and-drop, no developer. Often needs no AI at all — and where it does, only for one step, like summarising an attachment. |
| Agentic system multiple coordinated agents |
Several judgement steps chained: one queries the database, one analyses, one drafts, one checks — coordinated by an orchestrator. | Months. Most capable, hardest to debug, and it amplifies any mess in the process underneath it. Needs a stable process first. |
If a report follows a fixed format every month, that is automation with one AI step inside it — not an agentic system, and not a reason to build one.
The distinction matters because the second build is days of work and the third is months, and from the outside they can be described in almost identical language. When you hear a proposal, the question worth asking is: how many genuine judgement calls are there in this workflow? One or two means automation with an AI step. Five or six that depend on each other means an agentic system, and a much larger bill.
When your data sits in a relational database, there is a distinct third pattern worth knowing, and a good one: instead of retrieving passages, the model writes a query against your tables, runs it, and reports the result. Useful for "how many claims did we notify last quarter by peril?" — a question no document contains, because it's a calculation.
Two cautions. Give it a read-only account, always. And have it show the query it ran, so a person can check the logic rather than trusting a number that appeared from nowhere.
Two ways to pay for the model. They are not a ladder from casual to professional — they are a straightforward trade between variable and fixed cost.
| Rent (pay per token) | Own (buy GPUs) | |
|---|---|---|
| What you pay | Per unit of text in and out. No capital outlay. | Hardware up front, then power, cooling, space and someone to maintain it. |
| Cost behaviour | Scales with every question anyone asks. Predictable per use, unpredictable per month. | Flat once bought. Heavy use costs the same as light use — which is the whole point. |
| Getting started | An afternoon. Azure OpenAI, AWS Bedrock, or direct from OpenAI, Anthropic or Google. | Weeks. Procurement, installation, an open-weight model downloaded free, and a serving setup. |
| Where the data goes | Leaves your boundary. Enterprise terms bind them not to train on it. | Never leaves the building. The simplest possible answer to a client asking. |
| Capability | The strongest models, immediately, with nothing to maintain. | Open-weight models are strong and improving, generally a step behind the frontier. |
Don't estimate this. Measure it. Run the pilot on rented tokens for a month with usage logging on, then take the monthly bill and compare it against the fully-loaded annual cost of hardware — purchase, power, cooling, and the person who looks after it — divided by twelve.
If the rented bill is a small fraction of that, keep renting and stop thinking about it. If it's approaching or passing it and still climbing, owning starts to make sense. The crossover is a number, not an opinion, and one month of real logs settles an argument that could otherwise run for a year.
The failure mode with owning is not technical, it's utilisation. Hardware bought and lightly used is worse than renting in every respect — you have paid the fixed cost and kept none of the flexibility. It pays off on usage, not on ownership.
For a firm of your size beginning to explore, renting first is almost certainly right, and I'd say that even though the alternative sounds more impressive. You learn what your real volume is, you find out whether the answers are good enough to be worth anything, and you commit no capital while both of those are still unknown.
Four separate cost lines, and the one everybody worries about is rarely the one that matters.
| Cost line | What drives it | Rough shape |
|---|---|---|
| Build | One-off. How much is custom versus off-the-shelf. | Zero for a pilot on an existing tool. Several person-weeks for a proper custom build. Usually the largest single number. |
| Running | Per question asked. Billed per token in and out. | Cents per question. Often under $100 a month at small-firm volumes — see the worked example below. |
| Storage | Where the index and documents live. | Small. Tens of dollars a month at this scale, or bundled free into an existing cloud subscription. |
| People | Someone who owns it — updates documents, watches quality, fixes it when it breaks. | Part of a role, not a headcount, but not zero. This is the line most proposals leave out. |
| Route | To start | To run |
|---|---|---|
| Off-the-shelf pilot Custom GPT, Claude Project |
Nothing. An afternoon of your own time. | Around $20–25 per person per month on business tiers, with small seat minimums. A five-person pilot lands near $100–150 a month, all in. |
| Custom build on rented tokens the usual production answer |
Several person-weeks of developer time. Multiply by your own rate — this dominates year one. | Tokens, plus modest hosting. A token is roughly three-quarters of a word. Budget models run about $0.20 per million tokens in and $1.20 out; mid-tier about $2 and $12; frontier about $5 and $30. Batch work that can wait a day is half price. |
| Self-hosted your own GPU |
A capable inference card runs roughly $4,500–5,500 new; a consumer card around $2,000–2,500; a used data-centre card $8,000 upward. Plus the server around it. | Power, cooling, space, and maintenance. Renting equivalent bare-metal instead starts around $400–700 a month for a mid-size card, which is the sensible way to test this before buying anything. |
Say ten staff ask fifteen questions each per working day: roughly 3,300 questions a month. In a retrieval system each question sends about 3,500 tokens of retrieved material — call it 2,600 words — and returns about 400.
That is around 11.5 million tokens in and 1.3 million out per month. At budget-model rates that is about $4 a month. At mid-tier, about $40. At frontier rates, about $100. Indexing the documents in the first place is a one-off charge of a couple of dollars.
Compare that against the cheapest realistic self-hosting option — around $400–700 a month in rented hardware, before anyone's time. At this volume, renting tokens is between five and a hundred times cheaper, and it is not close.
You will find a break-even number quoted confidently in a dozen places. They disagree by roughly fifty times — from a few thousand conversations a day at one end to hundreds of millions of tokens a day at the other. That spread is not carelessness. It comes from two variables that every estimate has to assume: how busy the hardware stays, and how much unbudgeted engineering time gets counted.
The part that actually decides it is which model you would be replacing. Against an expensive frontier model, owning hardware can pay off relatively early, because each call is costly. Against an already-cheap efficient model, it may never pay off at all — those are priced close to the underlying cost of the compute, so there is little margin left to capture.
Rather than a token count, published total-cost analyses tend to converge on annual spend. Roughly: under about $50,000 a year on model usage, stay on the API — your volume will not keep a GPU busy, and one engineer to run inference costs more than the bill you are trying to cut. Between roughly $50,000 and $500,000, a hybrid starts to make sense. Beyond that, owning capacity becomes the serious option.
On the worked example above, a firm of ten would be spending somewhere between $50 and $1,200 a year. That is one to two orders of magnitude below the threshold where this even becomes a question.
There is a second multiplier that rarely appears in vendor arithmetic. Once monitoring, redundancy, model updates and the engineer's time are counted, the all-in cost of self-hosting typically runs three to five times the raw hardware price. The hourly GPU rate is the easiest part of the bill to see and the smallest part to pay.
The bill everyone examines is the token bill. The bill that decides the project is the build and the person who owns it afterwards.
Match the model to the job. The spread between the cheapest and most expensive model is enormous — well over a hundredfold. Most retrieval questions do not need a frontier model, and routing simple ones to a cheaper tier is the single biggest saving available.
Batch anything that can wait. Work that can complete within a day rather than a moment is typically charged at half rate.
Send less. You pay per token going in, so never hand a whole database to a model. Retrieve the few relevant passages and send those. Good retrieval is a cost control as well as a quality control.
Everything above is order-of-magnitude, meant for sizing a decision rather than approving a budget. Prices move quickly and downward, and vary by region and contract. The arithmetic is worth redoing against live pricing pages at the point you actually commit.
MCP — Model Context Protocol — is a standard way for an AI assistant to plug into external tools and data. Published as an open specification in late 2024 and since adopted across the major providers.
The problem it solves is combinatorial. Without a shared standard, every AI application needs a bespoke connector to every system: ten applications and ten systems means a hundred integrations. With one, any assistant that speaks MCP can talk to any system that speaks MCP.
It is free. The specification is open, the toolkits are open source, and ready-made connectors already exist for databases, file storage, messaging and code repositories. If a quotation arrives with MCP as a line item, that price is somebody's integration and implementation work — which may be perfectly fair — but it is not a licence fee, because there isn't one.
Probably not, and here's the honest test. MCP is for letting an assistant reach live systems — query the database as it stands today, or take an action somewhere. If what you want is answers from a set of documents, retrieval does that and MCP adds nothing.
It becomes genuinely useful at the point where you want the assistant to pull current figures from your database inside a conversation, rather than reading about them in a file. That is a real and worthwhile step — just a later one. Get the documents working first.
The same architecture, drawn once, with the later stages marked as later. Solid boxes are the minimum viable thing; dashed boxes are what you add when the need is proven.
Worth separating what genuinely differs between tiers from what is mostly marketing, because the gap between them is where people get caught out.
| Tier | What you actually get |
|---|---|
| Personal / consumer | Terms vary and can change. Some tiers use conversations to improve the service unless you opt out. Fine for public information; not for a client's claim file. |
| Business / team | Generally a commitment not to train on your content. Better, though administrative control is still limited. |
| Enterprise / cloud platform | A contract: no training on your data, a chosen region for where it sits, retention limits, audit logging. This is a legal commitment, not a promise in a help page — which is the real difference. |
| Your own hardware | Nothing leaves. No contract needed because there is no counterparty. |
In practice, the leak is rarely the provider. It is a member of staff, under time pressure, pasting a client's document into a personal account on their own laptop — because the approved tool was slower, or because nobody told them not to.
Which means the two things that protect you most are unglamorous: a written rule about what may be pasted where, and an approved tool good enough that nobody feels the need to go around it. Both cost nothing and outperform any amount of platform selection.
Two further habits worth adopting early, while it's still easy. Keep permissions in the assistant mirroring who may already read each document, so it never becomes a way around a filing cabinet somebody deliberately locked. And run the pilot on genuinely non-sensitive material — templates, public circulars, your own procedures — so you learn whether the thing works before the question of client data ever arises.
And note what isn't in that plan: no fine-tuning, no GPUs, no agentic system, no MCP. Every one of those may be right eventually. None of them is right before you know whether the answers are any good.
Work out what you're building before you work out what to buy. The platform question answers itself once the other three are settled.
JYnapse starts every engagement with exactly this triage — what are you actually building, and is it text or scans? If you'd like to run your own list past us, book a free consultation.
Scope. The examples here are illustrative, meant to make the ideas concrete rather than to describe any particular business.
About the author. JY is the founder of JYnapse and an AI/ML engineer with a master's in data science, building production GenAI, machine-learning and automation systems for businesses.