What enterprise Australia is actually asking about AI
Key themes from the Data & AI Summit Sydney 2026 - what leaders are wrestling with on governance, trust, POC failure rates, and getting AI into production.
The Data & AI Summit in Sydney brought together engineering leaders, data officers, heads of product and digital transformation leads from some of Australia’s biggest organisations – Nine, NAB, Allianz, Bupa, Pfizer, the Australian Digital Health Agency, TikTok, and Domain, among others.
I went to listen. Here’s what I heard.
The POC graveyard is full
McKinsey’s stat keeps surfacing: 80% of AI proof-of-concepts don’t make it to production.
At a summit full of AI progress, that number landed like a confession. The question isn’t “should we do AI?” – everyone’s doing AI. The question is: why does it keep stalling after the demo?
The POCs most likely to fail share a common trait: they come from the top down. An exec has a vision, a project gets stood up, and somewhere between “here’s the idea” and “here’s the business outcome” the whole thing loses traction. The POCs that scale start with a concrete problem statement – a real workflow pain, a measurable cost, an actual user need – and get something tangible in front of stakeholders fast.
Allianz put it well: “If you get a POC done in a week and it passes or fails, that’s good.” The goal isn’t perfection. It’s learning fast enough to know whether to proceed or pivot.
Key takeaways:
- Start with a concrete, measurable problem – not an exec’s vision of what AI should do
- Fast, bounded POCs beat long, expensive ones every time
- Define exit criteria before you start – what does “pass” and “fail” actually look like?
- A failed POC is valuable data, not a waste of money
Next steps:
Map your current AI initiatives against one question: does each one have a concrete problem statement, or is it driven by a vision of what AI could do? For any active POC without a defined success metric and a hard stop date, set those now. One week is a reasonable target for a fast experiment – if you can’t get something in front of a stakeholder in that time, the scope is too broad. POCs that have been running for months without a production path should probably be called done. The learning is the value. Restive works with engineering leaders to audit AI pipelines, sharpen problem definitions, and build the scaffolding to move experiments into production.
Trust is the architecture problem nobody’s solving
Every single speaker used the word “trust.” Not as a buzzword – as a genuine blocker.
Trust has a few layers here.
Trust in the output
Can I rely on what this model produces? Is it auditable? Allianz made an interesting observation: traditional ML felt like a black box, but modern generative AI – because it can explain its reasoning – is actually easier to trust when documented and tested properly. Policy scores with explanations aren’t just more defensible. They’re more useful.
Trust in the governance
Who owns the AI Risk Appetite Statement? How do you keep your governance framework current when the technology is changing faster than most compliance cycles? Pfizer described 509 active AI use cases. At that scale, you need reusable governance components and guardrails built into the architecture, not bolted on as a checklist.
Trust from the people using it
Jihad Toll from Toll Group made the point that stuck with me most: when AI is executing decisions, people need to be able to understand and challenge those decisions. He referenced Robodebt – 800,000 debt notices issued by an algorithm with no mechanism for contestation. That’s the failure mode everyone in the room is trying to avoid.
Trust isn’t a feature you add at the end. It’s a design constraint you build around from the start.
Key takeaways:
- Explainability isn’t optional – it’s what makes AI defensible at scale
- Governance frameworks need to be architecture decisions, not compliance checklists
- The Robodebt failure mode (decisions at scale with no contestation path) is a live risk for any organisation deploying AI broadly
- Adversarial stress testing and AI red teaming should happen before systems go near production
Next steps:
Start by asking who owns your AI Risk Appetite Statement. If the answer is unclear, that’s the first thing to fix. Document the decision-making logic of your highest-stakes AI systems now – before they’re at scale – and build in a contestation path for people affected by automated decisions. Your governance framework should be reusable across systems, not rebuilt from scratch for each use case. If you’re at Pfizer-scale (500+ use cases), this is already a platform architecture question, not a policy question. Teams navigating this often find it useful to work with external partners who have designed these frameworks across different regulatory environments – it’s a space Restive has worked in with financial services and enterprise clients.
The board wants AI now. The data isn’t ready.
Almost every speaker named the same pressure in different ways. Boards and executives want AI now. They’ve read the articles. They’ve seen the demos. They’re asking “where’s our AI strategy?” before the data foundation is ready, before the governance framework exists, before there’s organisational capacity to absorb the change.
NAB named it directly: “speed-to-value vs depth of assurance.”
The organisations doing this well have found ways to make the tension productive rather than paralyzing. They run fast experiments with clear exit criteria. They use tangible POCs to bring stakeholders along rather than writing requirements documents nobody reads. They treat a failed POC as learning, not failure.
The Henry Ford framing from Allianz is useful: the goal is a faster car, not a faster horse. The organisations stuck in the tension are optimising existing processes. The ones moving through it are asking which processes shouldn’t exist at all once AI is in the picture.
Key takeaways:
- “Speed-to-value vs depth of assurance” is the defining tension in enterprise AI right now
- Organisations resolving it are running faster experiments with tighter scope, not bigger waterfall programs
- A tangible POC is your best stakeholder management tool – better than any slide deck
- The real question isn’t “how do we move faster?” – it’s “which processes shouldn’t exist once AI is involved?”
Next steps:
The most effective way to manage board pressure is with something tangible. A working POC – even a narrow one – is more persuasive than any strategy deck. Pick one high-impact, low-risk use case, deliver it in a defined timeframe, and use that to set expectations for what “AI at scale” actually requires. Before you start, get alignment on what “done” looks like: what’s the measurable outcome, and at what point do you call it a success or a pivot? Having an independent team help you define the experiment scope and success criteria – rather than letting scope creep in during delivery – is often where outside help earns its keep. That’s the kind of work Restive does with engineering and digital leaders who need to move from experiment to embedded practice.
Data sovereignty is a growing blocker
This one came up organically in audience Q&A – not from any panel – and it got more traction than most panel questions.
How do you run AI at enterprise scale without handing your data to US-based cloud providers?
For government agencies this is already a blocker. The Australian Digital Health Agency operates under strict legislative constraints – My Health Record data can’t be used for population research even with individual consent. For financial services, health, and insurance organisations, data residency and sovereignty is a live concern, not a future one.
This isn’t an ideological position. It’s a compliance reality for a growing number of Australian enterprise buyers. The vendors and partners who understand it will have a significant edge over those who treat it as an edge case.
Key takeaways:
- Data sovereignty is a live compliance issue for regulated industries – not a future consideration
- Government agencies are already legislatively blocked from certain AI use cases
- Architecture decisions made now will either create or foreclose options later
- Sovereignty requirements need to be factored in at the platform selection stage, not retrofitted after
Next steps:
If you’re in a regulated industry, map your AI use cases against your data residency obligations before you go deeper into platform selection. The decisions you make at the architecture stage – which cloud provider, which processing environment, where data sits at rest and in transit – will either open or close options for years. This is especially important for any use case that involves personal data, health records, or financial information. It’s worth getting legal and compliance in the room earlier than feels necessary. When you’re ready to build, choose partners who have navigated these constraints before rather than learning on your use case. It’s an area Restive has practical experience in, working with financial services and enterprise clients who can’t simply move data offshore and call it done.
One in ten. That’s the ethical AI gap.
Dr. Catriona Wallace delivered the stat that should be on every executive’s slide deck: only 1 in 10 Australian organisations has an ethical AI policy in place.
Her point wasn’t that AI is inherently dangerous. It’s that machines can be trained to reflect ethical frameworks – and already mirror the values of whoever creates or interacts with them. The moral code can be embedded. But that only works if someone consciously decides what that code should be – and right now, most organisations haven’t done that work.
The stakes aren’t abstract. AI systems are already being used in large-scale targeting contexts where “89% accurate” has been deemed acceptable – which means an 11% error rate has also been deemed acceptable, by someone, somewhere, without most people in the affected organisations knowing it happened.
Ethical leadership here isn’t a philosophy exercise. It’s a decision about what your organisation will and won’t do with these systems – made before you’re in a situation where the pressure to move fast makes that decision for you.
Key takeaways:
- 90% of Australian organisations don’t have an ethical AI policy – this gap will matter more as AI scales
- Ethics isn’t a constraint on AI – it’s a design input
- Every AI system reflects implicit values; the question is whether those are deliberate or accidental
- Define your acceptable error rates explicitly – because someone will, whether you do or not
Next steps:
If your organisation doesn’t have an ethical AI policy, the starting point isn’t a framework – it’s a conversation. Get the right people in the room (technology leaders, legal, HR, whoever is closest to the actual use cases) and answer a few concrete questions: what decisions are we delegating to AI systems? What error rate is acceptable, and who decided that? Is there a human in the loop at the right point? Document the answers. That document is the beginning of your policy. Building this into your delivery process – not as a final review gate but as a design input at the start – is what separates organisations that can stand behind their AI systems from those who will be explaining them later. It’s the kind of thinking Restive builds into projects from day one, not bolted on at the end.
The workforce question nobody’s ready to answer
This one landed harder than most panel questions – and it came from the floor, not the speakers.
With work changing because of AI, what is leadership’s duty of care to employees?
Dr. Catriona Wallace put numbers to it: 85 million jobs affected by AI in the past 12 months, 92 million created. Net positive – on paper. But the 85 million affected and the 92 million created are not the same jobs, in the same industries, held by the same people. The transition is the problem, and most organisations aren’t treating it as one.
Nine’s framing of “partnering AI with people and culture” and Bupa’s insistence that “people and outcomes for users is what matters most” suggest the organisations doing this thoughtfully understand that the human element isn’t a soft consideration – it’s often the actual implementation risk.
The technical problem of getting AI to work is increasingly the easy part. Change management, upskilling, and trust-building with teams whose roles are shifting – that’s where most implementations run into trouble.
Key takeaways:
- The net job creation figure masks real transition pain – organisations need to hold both realities at once
- AI transformation is as much a people and culture challenge as a technology one
- Teams need a clear narrative about how AI changes their work – not just what it automates
- Organisations that treat this as an engineering problem alone will struggle with adoption
Next steps:
Before your next AI rollout, ask what the change management plan is – not as an afterthought, but as a project stream with the same priority as the technical build. What’s the communication strategy for affected teams? Is there a retraining pathway? Who is accountable for the human transition, not just the technical delivery? The organisations doing this well treat internal adoption as a product problem: they talk to the people whose roles are shifting, understand their concerns, and design the change around them. If you’re embedding external consultants into your teams to deliver AI projects, the best ones will bring this thinking with them rather than treating people as a deployment variable. It’s something Restive is deliberate about when working alongside engineering and delivery teams.
What this means for Australian enterprise
The experimentation phase is over. Everyone has done POCs. The question isn’t whether AI works – it demonstrably does, in the right context, with the right problem definition. The question is whether organisations can build the scaffolding – governance, architecture, culture, stakeholder management – to take AI from isolated experiments into embedded, scalable practice.
The organisations leading aren’t the ones with the most AI projects. They’re the ones with the clearest thinking about which problems AI should solve, the most honest accounting of what failed and why, and the most deliberate approach to governance as an architecture decision rather than a compliance exercise.
That’s the right problem to be working on.
Restive works with leaders and heads of enterprise and scale-up organisations across Australia. If you’re navigating the gap between AI experimentation and production-ready systems, let’s talk.
Related articles
