AI is showing up in companies not because “everyone already has it”, but because it can genuinely change how work gets done.
The hard part comes earlier:
How do you choose an AI vendor that will actually solve your problem?
Knowing that a product uses AI tells you very little. Before signing a contract, it is worth asking the vendor at least these 10 questions.
In short — what is this article about?
- 🎯 a specific business problem instead of “we have AI”
- 🗄️ data: where it comes from, where it lives, and whether it trains models
- 🧪 answer quality, hallucinations, and human oversight
- 🔗 integrations, real annual cost, and a pilot
- 🚪 exit terms and the solution’s limitations
1. What specific business problem does your AI solve?
Do not start with the technology. Start with the problem. Is the AI meant to:
- automate a process
- analyze data
- create reports
- support customer service
- generate documents
- help employees make decisions
If the vendor cannot clearly connect the technology to a concrete business process, it is time to pause.
2. What data does the solution run on?
This is one of the most important questions. AI is only as useful as the data and context it receives.
- what data is used
- where it is stored
- whether data is processed by external AI models
- whether customer data can be used to train models
- how deletion works
3. Which AI model do you use?
“We have our own AI” does not always mean the company built its own model. The product may rely on models from external providers.
This also matters for cost and for how the system can evolve.
- which model is used
- whether it can be changed
- whether the solution depends on a single model vendor
- how often the model is updated
4. How do you measure AI answer quality?
This question is often skipped.
“The AI works” is not a measurable criterion. A well-designed solution should have a quality-control mechanism — not just generate answers.
Accuracy
How often does the AI give a correct answer? On what data and how many cases was this measured?
Precision and recall
Does the AI correctly identify the cases it should catch, and how often does it miss important ones?
Hallucination rate
How often does the AI generate information that is not in the data or sources? Can it say “no data” instead of guessing?
Tests on real data
Has the solution been tested on a representative sample of our data and real business cases?
Answer validation
Are results checked automatically or manually? Can you see the data source behind an answer?
Error handling
What happens when the AI is wrong? Can you report an error, verify the answer, and stop automatic action?
The most important question: “How will you prove that your AI works correctly on our problem and our data?”
5. What happens when the AI is wrong?
This question is especially important in business processes. AI can generate a wrong answer.
In many processes the best setup is not AI instead of a human, but AI + a human.
- whether the result is accepted automatically
- whether a human can review it
- whether the system reports a confidence level
- whether you can trace the source of the answer
- what happens when an error occurs
6. Can the solution integrate with our systems?
AI that runs in isolation may have limited value. It is also worth checking whether integration is standard or requires an expensive custom project.
- ERP
- CRM
- databases
- Power BI
- API
- document systems
- other tools used in the company
7. What will the solution cost in a year — not just today?
The subscription price is often only the beginning.
- licenses
- number of users
- token / API usage
- integrations
- implementation
- maintenance
- further development
- training
- extra module costs
A good question: “What will this cost when the number of users or the volume of data grows 10x?”
8. Can we start with a pilot?
Before a large rollout, test the solution on a real problem. Do not buy a promise. Test the outcome.
- a specific goal
- a defined timeline
- real data
- measurable KPIs
- a clearly stated cost
9. What happens to our data if we end the relationship?
This question should be asked before you sign. It is not only about security — it is also about vendor lock-in.
- whether we can export the data
- in what format
- whether we can transfer the configuration
- whether we can change the model provider
- how data deletion works
10. What are the limitations of your solution?
This may be the best question of all. A professional vendor should be able to name the limitations of their product.
The goal is not to find AI that “can do everything”. The goal is to find a solution that works well in our specific business case.
- “In which situations might your AI not work well?”
AI should not be chosen on technology alone
When choosing an AI vendor, look at four areas at the same time:
Business → Data → Technology → Risk
Only together do they show whether a solution has a chance of bringing the company real value.
AI can be an excellent technology. But even the best model will not fix a poorly defined problem, low-quality data, or a badly designed process.
So before asking “which AI should we choose?”, first ask: “what problem do we want it to solve?”
💬 What question would you add to this list? What else is worth agreeing with a vendor before AI becomes part of everyday work?
FAQ
What questions should you ask an AI vendor before signing a contract?
Ask about the specific business problem, data and storage, the model in use, how quality is measured, error handling, integrations, the real annual cost, a pilot option, exit terms, and the solution’s limitations.
Why does it matter what data an AI solution runs on?
AI is only as useful as the data and context it receives. You need to know which data is used, where it is stored, whether it is processed by external models, and whether it can be used for training.
Should you start with a pilot before a full AI rollout?
Yes. A good pilot has a clear goal, a defined timeline, real data, measurable KPIs, and a known cost. It lets you test the outcome on a real problem before buying a promise.
What should happen to company data if you end the relationship with an AI vendor?
Before signing, agree whether you can export data and configuration, in what format, whether you can change the model provider, and how data deletion works. This is both a security issue and a vendor lock-in issue.