Give the same instructions to several AI models and you can receive surprisingly different results. One might produce a clear, carefully considered answer. Another might miss an important requirement. A third might take an approach you had not considered at all.
We often talk about “AI” as though it were a single tool, but there are significant differences between providers and between the models each provider offers. Getting useful results means understanding those differences and choosing a model that suits the work.
The prompt is only part of the picture
There is plenty of advice about writing better prompts, and clear instructions certainly help. Explaining the task, providing relevant context and describing the intended output all give a model a better chance of producing something useful.
However, even a well-written prompt will not produce the same result everywhere. Models differ in how they interpret instructions, approach problems and present their answers.
A model that works well for drafting readable business content might be less suitable for investigating a complicated programming problem. Another might produce working code but struggle with visual design. Some are quick and economical for routine tasks, while others justify more time and expense when the work requires deeper analysis.
If a model repeatedly struggles with a task, trying another model can be just as useful as rewriting the prompt.
Cost needs some context
Prices also vary considerably. For occasional use, the difference might be modest. For an application processing thousands of requests, model choice can have a substantial effect on running costs.
The most expensive model is not automatically the right choice for every job. A faster, cheaper model may be perfectly capable of handling a clearly defined, repetitive task.
Equally, the cheapest response may take more time to check and correct. A useful comparison considers the overall effort required to reach an acceptable result, including the time spent reviewing it.
Models can work as a team
You do not have to use one model for an entire project. It is possible to combine them, assigning different parts of the work according to their strengths, much as you would with a team of people.
For example, one model might help plan an application, another develop its interface, and a third review the code or suggest test cases. A lower-cost model could handle routine processing, with more demanding tasks passed to a more capable model.
This approach still needs oversight. Models can share the same blind spots, and agreement between them does not establish that an answer is correct. Their contributions need to be checked against the actual requirements.
Our world sunclock test
At Neterix, we keep an eye on the many benchmarks available to compare AI models. These provide useful evidence, but we also like to see how models respond to a practical task.
As part of that monitoring, we give each new model the same short prompt. It asks for a world sunclock built using HTML and JavaScript, with multiple clocks and a shadow showing which parts of the world are at night. The prompt also includes a simple style guide.
You can view the world sunclock results and compare the different approaches.
The outputs vary considerably. The task leaves room for design decisions while also requiring the model to understand how the different elements should work together.
We look at more than whether the page looks attractive. Are the clocks arranged in a logical order? Does the night shadow make sense? Does it correctly understand the timezones? Are the cities plotted in the correct places? That last point is particularly revealing: a convincing-looking map can still put a city in the wrong location.
The exercise brings together interpretation, visual design, coding and geographical understanding in a relatively small project.
A useful comparison, with limits
Our sunclock prompt is not a conclusive benchmark, and we would not use it alone to declare one model better than another. It samples a particular combination of abilities, and results can vary between attempts.
It does, however, demonstrate why model choice matters. The same brief can produce very different outcomes, and an impressive appearance does not guarantee that the underlying details are right.
When choosing AI for your organisation, start with a few representative tasks of your own. Compare models using the same instructions, decide what a successful result should contain, and assess accuracy, clarity, speed and cost.
Public benchmarks can help you build a shortlist. Testing models against the work you actually need them to do will tell you much more about which one deserves a place in your workflow.
If you would like to talk through where AI fits in your organisation, and how to adopt it safely, get in touch.