Quote for the day:
"Failure isn't fatal, but failure to change might be" -- John Wooden
How enterprise IT can protect itself from genAI unreliability
The AI-watching-AI approach is scarier, although a lot of enterprises are giving it a go. Some are looking to push any liability down the road by partnering with others to do their genAI calculations for them. Still others are looking to pay third-parties to come in and try and improve their genAI accuracy. The phrase “throwing good money after bad” immediately comes to mind. The lack of effective ways to improve genAI reliability internally is a key factor in why so many proof-of-concept trials got approved quickly, but never moved into production. Some version of throwing more humans into the mix to keep an eye on genAI outputs seems to be winning the argument, for now. “You have to have a human babysitter on it. AI watching AI is guaranteed to fail,” said Missy Cummings, a George Mason University professor and director of Mason’s Autonomy and Robotics Center (MARC). “People are going to do it because they want to believe in the (technology’s) promises. People can be taken in by the self-confidence of a genAI system,” she said, comparing it to the experience of driving autonomous vehicles (AVs). When driving an AV, “the AI is pretty good and it can work. But if you quit paying attention for a quick second,” disaster can strike, Cummings said. “The bigger problem is that people develop an unhealthy complacency.”
Why neglecting AI ethics is such risky business - and how to do AI right
The struggle often comes from the lack of a common vocabulary around AI. This is
why the first step is to set up a cross-organizational strategy that brings
together technical teams as well as legal and HR teams. AI is transformational
and requires a corporate approach. Second, organizations need to understand what
the key tenets of their AI approach are. This goes beyond the law and
encompasses the values they want to uphold. Third, they can develop a risk
taxonomy based on the risks they foresee. Risks are based on legal alignment,
security, and the impact on the workforce. ... As a starting point, enterprises
will need to establish clear policies, principles, and guidelines on the
sustainable use of AI. This creates a baseline for decisions around AI
innovation and enables teams to make the right choices around the type of AI
infrastructure, models, and algorithms they will adopt. Additionally,
enterprises need to establish systems to effectively track, measure, and monitor
environmental impact from AI usage and demand this from their service providers.
We have worked with clients to evaluate current AI policies, engage internal and
external stakeholders, and develop new principles around AI and the environment
before training and educating employees across several functions to embed
thinking in everyday processes.
The risks of entry-level developers over relying on AI
Some CISOs are concerned about the growing reliance on AI code generators —
especially among junior developers — while others take a more relaxed,
wait-and-see approach, saying that this might be an issue in the future rather
than an immediate threat. Karl Mattson, CISO at Endor Labs, argues that the
adoption of AI is still in its early stages in most large enterprises and that
the benefits of experimentation still outweigh the risks. ... Tuskira’s CISO
lists two major issues: first, that AI-generated security code may not be
hardened against evolving attack techniques; and second, that it may fail to
reflect the specific security landscape and needs of the organization.
Additionally, AI-generated code might give a false sense of security, as
developers, particularly inexperienced ones, often assume it is secure by
default. Furthermore, there are risks associated with compliance and
violations of licensing terms or regulatory standards, which can lead to legal
issues down the line. “Many AI tools, especially those generating code based
on open-source codebases, can inadvertently introduce unvetted, improperly
licensed, or even malicious code into your system,” O’Brien says. Open-source
licenses, for example, often have specific requirements regarding attribution,
redistribution, and modifications, and relying on AI-generated code could mean
accidentally violating these licenses.
Language models in generative AI – does size matter?
Firstly, using SLMs rather than full-blown LLMs can bring the cost of that
multi-agent system down considerably. Employing smaller and more lightweight
language models to fulfill specific requirements will be more cost-effective
than using LLMs for every step in an agentic AI system. This approach involves
looking at what would be the right component for each element of a multi-agent
system, rather than automatically thinking that a “best of breed” approach is
the best approach. Secondly, using agentic AI for generative AI use cases should
be adopted where multi-agent processes can provide more value per transaction
than simpler single-agent models. The choice here affects how you think about
pricing your service, what customers expect from AI and how you will deliver
your service overall. Alongside looking at the technical and architecture
elements for AI, you will also have to consider what your line of business team
wants to achieve. While simple AI agents can carry out specific tasks or
automate repetitive tasks, they generally require human input to complete those
requests. Where agentic AI takes things further is through delivering greater
autonomy within business processes through employing that multi-agent approach
to constantly adapt to dynamic environments. With agentic AI, companies can use
AI to independently create, execute and optimize results around that business
process workflow.
Lessons from a Decade of Complexity: Microservices to Simplicity
This shift made us stop and think: if fast growth isn’t the priority anymore,
is microservices still the right choice? ... After going through years of
building and maintaining systems with microservices, we’ve learned a lot,
especially about what really matters in choosing an architecture. Here are
some key takeaways that guide how we think about system design today: Be
pragmatic, not idealistic: Don’t get caught up in trendy architecture patterns
just because they sound impressive. Focus on what makes sense for your team
and your situation. Not every new system needs to start with microservices,
especially if the problems they solve aren’t even there yet. Start simple: The
simplest solution is often the best one. It’s easier to build, easier to
understand, and easier to change. Keeping things simple takes discipline, but
it saves time and pain in the long run. Split only when it really makes sense:
Don’t break things apart just because “that’s what we do”. Split services when
there’s a clear technical reason, like performance, resource needs, or special
hardware. Microservices are just a tool: They’re not good or bad by
themselves. What matters is whether they help your team move faster, stay
flexible, and solve real problems. Every choice comes with tradeoffs: No
architecture is perfect. Every decision has upsides and downsides. What’s
important is to be aware of those tradeoffs and make the best call for your
team.
Massive modernization: Tips for overhauling IT at scale
A core part of digital transformation is decommissioning legacy apps,
upgrading aging systems, and modernizing the tech stack. Yet, as appealing
as it is for employees to be able to use modern technologies,
decommissioning and replacing systems is arduous for IT. ... “You almost do
what I call putting lipstick on a pig, which is modernizing your legacy
ecosystem with wrappers, whether it be web wrappers, front end and other
technologies that allow customers to be able to interact with more modern
interfaces,” he says. ... When an organization is truly legacy, most will
likely have very little documentation of how those systems can be supported,
Mehta says. That was the case for National Life, and it became the first
roadblock. “You don’t know what you don’t know until we begin,” he says.
This is where the archaeological dig metaphor comes in. “You’re building a
new city over the top of the old city, but you’ve got to be able to dig it
only enough so you don’t collapse the foundation.” IT has to figure out
everything a system touches, “because over time, people have done all kinds
of things to it that are not clearly documented,” Mehta says. ... “You have
to have a plan to get rid of” legacy systems. He also discovered that
“decommissioning is not free. Everybody thinks you just shut a switch off
and legacy systems are gone. Legacy decommissioning comes at a cost. You
have to be willing to absorb that cost as part of your new system. That was
a lesson learned; you cannot ignore that,” he says.
Culture is not static: Prasad Menon on building a thriving workplace at Unplugged 3
To cultivate a thriving workplace, organisations must engage in active
listening. Employees should have structured platforms to voice their
concerns, aspirations, and feedback without hesitation. At Amagi, this
commitment to deep listening is reinforced by technology. The company has
implemented an AI-powered chatbot named Samb, which acts as a "listening
manager," facilitating real-time employee feedback collection. This tool
ensures that concerns and suggestions are acknowledged and addressed within
15 days, allowing for a more responsive and agile work environment. "Culture
is not just a feel-good factor—it must be measured and linked to results,"
Menon emphasised. To track and optimise cultural impact, Amagi has developed
a "happiness index" that measures employee well-being across financial,
mental, and physical dimensions. By using data to evaluate cultural
effectiveness, the organisation ensures that workplace culture is not just
an abstract ideal but a tangible force driving business success. ... At the
core of Amagi’s culture is a commitment to becoming "the happiest workplace
in the world." This vision is driven by a leadership model that prioritises
genuine care, consistency, and empowerment. Leaders at Amagi undergo a
six-month cultural immersion programme designed to equip them with the
skills needed to foster a safe, inclusive, and high-performing work
environment.
Speaking the Board’s Language: A CISO’s Guide to Securing Cybersecurity Budget
A major challenge for CISOs in budget discussions is making cybersecurity
risk feel tangible. Cyber risks often remain invisible – that is, until a
breach happens. Traditional tools like heat maps, which visually represent
risk by color-coding potential threats, can be misleading or oversimplified.
While they offer a high-level view of risk areas, heat maps fail to provide
a concrete understanding of the actual financial impact of those risks. This
makes it essential to shift from qualitative risk assessments like heat maps
to cyber risk quantification (CRQ), which assigns a measurable dollar value
to potential threats and mitigation efforts. ... The biggest challenge CISOs
face isn’t just securing budget – it’s making sure decision-makers
understand why they need it. Boards and executives don’t think in terms of
firewalls and threat detection; they care about business continuity, revenue
protection and return on investment (ROI). For cyber investments, though,
ROI is not typically the figure security experts turn to to validate these
investments, largely because of the difficulties in estimating the value of
risk reduction. However, new approaches to cyber risk quantification have
made this a reality. With models validated by real-world loss data, it is
now possible to produce an ROI figure.
Can AI predict who will commit crime?
Simulating the conditions for individual offending is not the same as
calculating the likelihood of storms or energy outages. Offending is often
situational and is heavily influenced by emotional, psychological and
environmental elements (a bit like sport – ever wondered why Predictive AI
hasn’t put bookmakers out of business yet?). Sociological factors also play
a big part in rehabilitation which, in turn, affects future offending.
Predictive profiling relies on past behaviour being a good indicator of
future conduct. Is this a fair assumption? Occupational psychologists say
past behaviour is a reliable predictor of future performance – which is why
they design job selection around it. Unlike financial instruments which warn
against assuming future returns from past rewards, human behaviour does have
a perennial quality. Leopards and spots come to mind. ... Even if the data
could reliably tell us who will be charged with, prosecuted for and
convicted of which specific offence in the future, what should the police do
about it now? Implant a biometric chip and have them under perpetual
surveillance to stop them doing what they probably didn’t know they were
going to do? Fine or imprison them? (how much, for how long?). What standard
of proof will the AI apply to its predictions? Beyond a reasonable doubt?
How will we measure the accuracy of the process?
CISOs battle security platform fatigue
“Adopting more security tools doesn’t guarantee better cybersecurity,” says
Jonathan Gill, CEO at Panaseer. “These tools can only report on what they
can see – but they don’t know what they’re missing.” This fragmented
visibility leaves security leaders making high-stakes decisions based on
partial information. Without a verified, comprehensive system of record for
all assets and security controls, many organizations are operating under
what Gill calls an “illusion of visibility.” “Without a true denominator,”
he explains, “CISOs are unable to confidently assess coverage gaps or prove
compliance with evolving regulatory demands.” And those blind spots aren’t
just theoretical. Every overlooked asset or misconfigured control becomes an
open door for attackers — and they’re getting better at finding them. “Each
of these coverage gaps represents risk,” Gill warns, “and they are
increasingly easy for attackers to find and exploit.” The lack of clear
visibility also muddies accountability. “This creates dark corners that go
overlooked – servers and applications are left without owners, making it
hard to assign responsibility for fixing issues,” Gill says. Even when gaps
are known, security teams often find themselves drowning in data from too
many tools, struggling to separate signal from noise.
No comments:
Post a Comment