Driving through fog: the real cost of AI, from the CDAIO NZ panel

Everyone is driving through fog with AI. That was how Peter Gavin summed it up, and nobody on the panel disagreed. There is no point pulling over and stopping. There is no point flooring it either.

“The Economics of AI: Cost, Consumption and Control at Scale” brought together Matt Bain, Chief Technology and AI Officer at Spark, Amanda White, Chief Information Officer at Open Country Dairy, and Peter Gavin, General Manager, Digital and Data at New Zealand Trade and Enterprise. Xanthe Sulzberger, Head of Data, Operational Excellence at Fonterra and the conference chair, moderated. It followed Joseph Sullivan’s warning about token bleed, and it picked up exactly where he left off: what AI costs, and how to keep that cost under control without killing the curiosity that makes it worth paying for.

The answers were refreshingly practical. Split personal AI from enterprise AI. Cap consumption deliberately. Feed the models context, because without it they are expensive and useless. And decide what you are building for before you build it.

TL;DR

• Amanda White splits AI into personal AI, which is secure and easy to manage, and enterprise AI, which runs on token consumption and needs controls. At Open Country Dairy, developers build skills and users consume them within limits the organisation sets.
• She has heard horror stories from overseas of bills of $200,000 a month with no value to show for them. New Zealand companies do not have that kind of money, so Open Country stays deliberately conservative.
• Peter Gavin’s NZTE gives everyone personal productivity tools and caps token-based access to a very small number of people, then lifts what works from personal to team to organisational productivity.
• Gavin’s warning on cost: AI models are “Einstein-grade interns” who will be hopeless and very expensive without context, and most of that context still lives in people’s heads.
• Matt Bain says Spark’s funding models are not broken, but change has to be forced, and the knowledge locked in a few architects’ heads is a risk worth capturing now.
• On proofs of concept, Bain loves them and hates “scaling the PoC”. At Fonterra, Xanthe Sulzberger sorts every idea into built to break, built to learn or built to last.

Personal AI and enterprise AI

The panel had met by phone a few weeks earlier to plan the session, and Sulzberger opened by recalling the conclusion they had reached: managing the cost of AI is something leaders have to do.

Bain put it as a question of resourcing. In New Zealand, most technology teams are oversubscribed and allocated to projects. Machine intelligence is a different kind of resource, and leaving it uncapped is something he thinks leaders very much need to focus on.

White draws a clear line between two kinds of AI. Personal AI is the tool on everyone’s desk, the assistant that makes people more proactive. That is quite easy to manage, she said, because it is secure and it makes people more productive. Enterprise AI is the one that runs on consumption, and it is the one that needs controlling.

Her example was a skills hub Open Country Dairy is building. It is locked down: users cannot create their own skills. Developers create the skills, and users consume them on a token basis, with the organisation setting the limits. She has heard horror stories from overseas, none from New Zealand, of companies getting bills of around $200,000 a month and seeing no value from them. New Zealand companies do not have that kind of money, she said, so Open Country is very conservative in its approach.

Driving through fog

Gavin, newly at NZTE and talking to businesses around the world, said everyone is driving through fog with this, and the analogy works well with leadership. There is no point pulling over and stopping. There is also no point flooring it and letting everybody have all they want.

NZTE gives everybody personal productivity tools, and has to manage the enthusiasm around that, which he called wonderful to see. Then it takes a deliberate approach to capping, allowing a very small number of people token-based access to innovate. The aim is to lift what makes a person productive into a team context, and from there into an organisational one, while being careful about where to loosen the reins and where to press the accelerator.

"There's no expert anywhere in the world who can tell you they've smashed this."

Everybody is learning how to do it, he said, and the learning is a good experience. NZTE’s job is to encourage people to try, and to be careful about going too far, too fast, in an uncontrolled way.

The next step change

Sulzberger asked how organisation structures, roles and responsibilities are changing.

White reached for an analogy she uses at Open Country to cut through the hype. Excel and PowerPoint were once new and a massive step change. AI is the next one. In five years, she said, people will not use those tools the way they do today. They will pop a request into a prompt, “can you make me an invoice, can you make me a document”, and it will build it for them. The job now is to teach people the next generation of tools and to use them responsibly, including security: you cannot just chuck data around. For her, that is a cultural shift.

Einstein-grade interns

Gavin has worked in the information industry for about 30 years, and has been hoping throughout his career that knowledge management would have its day. Privacy had its day about 10 years ago, he said, and it set us up well, as a society and as professionals, for what is happening now. Now organisations are starting to ask where the knowledge of the organisation lives and how it is nurtured and maintained, because context is key.

"You've got these Einstein-grade interns available to you now, but they're going to be hopeless and very expensive unless you give them context."

In a lot of cases that context is in people’s heads: how the organisation really works, what customers really want. Many companies rely heavily on structured information and are not accessing the unstructured knowledge in emails, meeting transcripts and documents. Harnessing that deliberately is key for the next step, he said. Knowledge management may finally have its moment, and if you ignore it, it is going to bite you pretty soon.

The cost of not doing it

Sulzberger turned the conversation to opportunity cost: the cost of not doing something because it feels too expensive. Part of her role is building product, and she shared how she works now. A conversation with a customer over coffee, or a nudge in a lift, becomes an idea she thinks about overnight and pops into Claude Code to test the possibilities. She recalled Bain telling her that something which once would have taken three months to deploy at Spark would now take a weekend.

Gavin argued that cost bites hardest where complexity sits in a business. Humans are good at complicated things: given time, we get our heads around them. Complexity is different. It arrives when an enterprise has channels, customers, competition and complex products, and that is where people stub their toes and costs blow out.

His example came from banking. Introduce a new pricing structure on a credit card, and the way certain customer groups use their products could mean they end up being overcharged a fee, through a combination no human would reasonably spot. These tools can pick up that kind of toxic combination and warn you. That hidden cost of complexity, he said, is as much an issue as the opportunity cost of not getting into the work at all.

White said Open Country does not feel behind. It is conservative by nature and has things under control, and it is partnering hard with its vendors. Its optimisation models have run rock solid for four years. Put AI over the top of foundations like that, she said, “now we’re cooking with gas”.

Forcing change

Sulzberger asked Bain whether funding models need to change when technology moves this fast.

He does not think they are broken, but he does think change has to be forced. The work itself has changed: building an API used to take weeks, and now it is done with agents. Where there are large legacy platforms, continuing as before is seen as a big risk. Training alone will not do it, in his experience. Leaders need to create the conditions that make good people think very differently, and constraint is one of them: hold teams to the same targets with less, and they will find another way.

He also named a risk most people in the room recognised. Most organisations have someone architecting systems who is the only person who understands them. He has been in situations where he was told the work could not happen without that person. Capturing that knowledge now, he said, is part of the answer, alongside financial and technical controls to manage the economics.

Gavin questioned the planning horizon itself. He is asking why NZTE would talk about a three-year planning cycle for strategy, after a customer in Christchurch told him: stop asking me for three-year plans, I really cannot see past six to 12 months. You still need a direction and a true north to focus teams, he said, but you need the adaptability to change direction quickly. He pointed to a New Zealand exporter that had its best year yet in North America despite tariffs and continued uncertainty, because of its adaptability.

Questions from the floor

The Slido questions came in thick and fast, and Sulzberger put two of them to the panel.

White was clear. Open Country is deliberate: every piece of AI work has to solve a problem and be outcome based. It is not enough to be more efficient. It has to move the needle, and moving the needle means making more money. It has just put an AI layer over its optimisation models into production for that reason.

The rest makes the organisation more efficient as a whole, so it can do more with less, because people should not be doing repetitive work. And she was firm on one control.

"AI can do the work, but a human must validate it before it goes out anywhere."

Gavin answered with language. The companies having the most success, he said, talk to their people in a particular way.

"We will do more of the right things at a higher quality, with the same."

That matters most for creative roles, in marketing and content, where people can be confronted by and even hostile to AI, because it seems to disrupt the thing they think is magical about their work. NZTE’s message to them is that they are the person who can tell what is magic and what is not. Their judgement is what gives a tool like this its value, and with it they can produce more great work, faster and at a higher quality. That turns the conversation around, he said, and he is seeing more energy from that narrative.

Built to break, built to learn, built to last

Bain argued against following a string of point solutions in an unstructured way, which in his experience leads you into infrastructure that later has to be removed. Better to identify a high-value process and rethink it from scratch. Some of that will take 12 months and some three to six, he said, but chasing it down over time shows how the capability compounds outcomes.

Sulzberger described Fonterra’s monthly prioritisation forum. Ideas come in through the front door, from anyone, including through town halls. Product owners size them and assess the return on investment, they go in front of the executive, and decisions are made from a set budget. Every build is one of three kinds: built to break, to find out quickly whether something works; built to learn, to be hardened and put into production later; or built to last, a product the business knows it will rely on, often a multi-year programme worth building right the first time. People feel listened to and heard, she said, because they get a time frame for what will be built.

Bain then set the record straight on proofs of concept.

"Just for the record, I love PoCs. What I don't enjoy is talking about now we're going to scale the PoC."

You cannot scale a proof of concept, he said. Spark runs a lab approach with three constraints: your imagination, your manager not thinking it is too crazy, and telling everyone what you are doing, so others can see whether you have done something they are interested in. For a very small investment, people get complete freedom in an air-gapped environment to try the latest models and find out quickly whether something is valuable or dangerous. Failure, to him, is someone spending two weeks repeating what a colleague already tried. And if the plan is to ship code straight out of the lab, he said, it is not being done right. A proof of concept should be planned on the assumption it may not prove valuable, and that is different from a pilot, where you already know it works and roll it out gradually.

White agreed: she is a big fan of PoCs too. Iterate fast.

What it means

The panel did not offer a formula, and it was honest about why: nobody has one yet. What it offered was a set of habits. Separate the AI everyone uses from the AI that runs up a token bill. Cap the second deliberately and lift what works. Give the models context, which means taking knowledge management seriously. Keep a human in the loop on anything that leaves the building. And decide, before you start, whether you are building to break, to learn or to last.