How to Choose an AI Company (Technical + Business Checklist)
Works for any software or AI project.
Choosing the right AI partner can save months of rework and a lot of budget, and this checklist helps you do exactly that by showing what to ask before you sign anything.
Get those questions right and you move fast; skip them and the project can turn into a time and money pit.
This is the checklist we use when evaluating vendors.
Why choosing wrong is expensive
A badly run project eats through time and budget before revealing it was never going to work. Imagine two scenarios:
One company hired a vendor who proposed fine-tuning before understanding the problem. Four months later, they discovered the client needed a data pipeline, not a custom model.
In another case, the vendor locked everything into their platform. When the client wanted to leave, they had no access to the code, the prompts, or the processed data.
The cost of getting out is higher than the cost of getting in. The sooner you spot the signals, the better.
Evaluation criteria
Use these criteria to filter vendors during conversations, proposals, and reference checks.
Technical depth
Ask how they'd approach your specific problem. Not "how do you use AI". Everyone has a canned answer for that. Ask about architecture.
If a vendor suggests LLMs for everything without checking whether the problem is classification, prediction, or search, they're applying the wrong tool.
If the answer to "which model and why" is just "GPT-5" or "Claude" without mentioning cost, latency, or context window trade-offs, the vendor lacks criteria.
Ask where data lives, who has access, and what happens to it after the project. Vague answers are a red flag.
If the vendor only knows the basics in RAG architectures, they'll struggle with larger, messier document sets. Ask about hybrid search, re-ranking, and contextual retrieval.
Proof of work
Ask for examples similar to your problem. Not generic testimonials. Cases with the challenge, architecture, result, and timeline. If the portfolio is just "chatbot" and "automation" without specifics, they probably can't go deep.
Also ask for an evaluation set. A serious vendor can put together 50-100 questions with known correct answers in the first few weeks to measure quality. If nobody mentions evaluation, the project will be guided by opinion.
Process transparency
A vendor that says "we deliver the result" without showing the process is hiding something. Ask:
- What does the first week look like?
- When do you get to see the first working fix?
- How long until it runs on real data?
- What happens when things don't work?
The process should be: a diagnostic, the fix on real data, and a plan to keep it running. When that work is finished, the vendor leaves, and you keep the code.
Red flags
"We use AI" without specifying what kind, what model, or what approach is the equivalent of saying "we use technology". It means nothing.
If the proposal includes "our proprietary platform" and doesn't mention export, that's explicit vendor lock-in. You'll get stuck.
If nobody built a test set to measure output quality, "it works" is an opinion, not a fact.
If the vendor already knows what they'll deliver before understanding your data and processes, they're selling, not diagnosing.
Per-seat monthly pricing with no clear scope means you know the monthly price per user, but not what is actually included. The real cost will usually be higher.
Architecture questions for the first call
- Do you start with diagnosis, or do you jump to proposing a solution?
- Which model would you use for my case, and why?
- How do you measure quality? Do you use an evaluation set?
- Where do my data live? Who has access?
- If I end the contract, what do I walk away with? Code, prompts, pipelines, data?
- Do you propose fine-tuning, RAG, or APIs? When does each make sense?
- What's the timeline to a working fix on my data?
If the vendor answers all of these clearly, you're in good shape. If they dodge, they either don't know or the answer isn't favorable.
Pricing traps
The pricing model changes everything about what happens after the project.
Per-seat subscription works for generic SaaS. For AI, it incentivizes the vendor to keep you dependent. And the cost scales with adoption, which is the opposite of what you want.
Per-token or per-call pricing is transparent, but hard to predict. Ask for the estimated monthly cost at your expected volume.
Fixed-scope project means clear scope, fixed price, and the fix running on your data. When the work is finished, you keep the diagnosis and the code. No second phase, and no surprises.
What matters is knowing what's included and what isn't. Hosting, maintenance, prompt adjustments, training. All of it costs money and needs to be clear upfront.
What a serious process looks like
You can tell early when the process is serious. It usually looks like this:
The vendor starts by understanding the problem before proposing a solution. That sounds obvious, but many teams skip it. Diagnosis needs to cover the current workflow, available data, and constraints. If the vendor shows up with a ready-made answer, they're selling, not diagnosing.
Then comes the fix. Not a requirements document, and not a prototype to throw away. Software on the real data, reviewed while it is built, running in the systems the team already uses. A workflow that looks simple on paper shows whether it runs. If it is too big for the problem, that shows up in the work, and the scope gets cut there.
When the fix is in hand, the problem is solved. You have the result, the infrastructure cost, the code, and a plan to keep it running. There is no later conversation about the serious project.
A vendor who sells a prototype, or a pilot that needs a second contract, is leaving the problem open.
At A1 Lab, this is how we work: a diagnostic in 3 to 5 days, the fix in 1 to 3 weeks, and the technical plan in the last days, once the work is done. On the calendar that is 2 to 5 weeks when the owner and the files are available in week one. You get a write-up of the work, the fix on your data, real costs, and a plan to keep it running. Then we leave. The code is yours.
Frequently asked questions
How do I know if I need AI or just regular automation? If the problem is rule-based, repetitive, and well-defined (move data from A to B, send an email when X happens), you need automation, not AI. If the problem requires text interpretation, ambiguous decisions, or natural language responses, AI makes sense.
What's the minimum timeline to see results? A diagnostic in 3 to 5 days, then a working fix in 1 to 3 weeks, 2 to 5 weeks on the calendar once the files are in hand. Two weeks is the small case. If someone promises less than that, ask how.
Do I need to have my data organized before hiring? Not necessarily, but it helps. A good vendor will help you assess data quality as part of diagnosis. If the data is messy, that should surface during diagnosis, not after three months of development.
What do I keep if the project doesn't work out? Code, prompts, processing pipeline, and data. If the vendor won't give you that, ask why before signing.
See also: