7 LLM Risks and API Management Strategies to Keep Data Safe
Overloading an LLM with requests can cause poor service or increased resource
costs, two of the worst outcomes for an organization. Yet with a model denial of
service that is what’s at stake. This happens when attackers cause
resource-heavy operations on LLMs. This could look like a higher-than-normal
task generation or repeated long inputs, to name a few. Authentication and
authorization can be used to prevent unauthorized users from interacting with
the LLM. Rate limiting on the number of tokens per user should also be used to
stop users from burning through an organization’s credits, incurring high costs
and using large amounts of computation resulting in latency injection. ...
Compliance teams’ concern about sensitive information disclosure is perhaps one
of the most severe vulnerabilities limiting LLM adoption. This occurs when
models inadvertently can return sensitive information, resulting in unauthorized
data access, privacy violations and security breaches. One technique that
developers can implement is using specially trained LLM services to identify and
either remove or obfuscate sensitive data.
Michael Dell performed a ‘hard reset’ of his company so it could survive massive industry shifts and thrive again. Here’s how it’s done
A hard reset asks and answers a small set of critical strategy questions. It
starts with revisiting your beliefs. Discuss and debate your updated beliefs
with the team and build a plan to actively test the ones where you disagree or
have the most uncertainty about. Next ask what it will take to build a
defensible competitive advantage going forward: Determine if you still have a
competitive advantage (you probably don’t—otherwise you wouldn’t be in a hard
reset). Glean what elements you can use to strengthen and build an advantage
going forward. Over-index on the assets you can strengthen and discuss what you
will buy or build. Make sure you anchor this in your beliefs around where the
world is going. ... During a hard reset, develop rolling three-month milestones
set towards a six-month definition of success. Limit these milestones to ten or
fewer focused tasks. Remember you are executing these milestones while
continuing the reset process and related discussions, so be realistic with what
you can achieve and avoid including mere operational tactics on the milestone
list.
Software testing’s chaotic conundrum: Navigating the Three-Body Problem of speed, quality, and cost
Companies that prioritize speed over quality end up with the choice of whether
to release to market anyway, and risk reputational damage and client churn, or
push back timelines and go over budget trying to retrofit quality (which isn’t
really possible, by the way). ... Quality is the cornerstone of successful
digital products. Users expect software to function reliably, deliver on its
promises and provide a seamless user experience. Comprehensive testing plays a
large role in making sure users are not disappointed. Developers need to look
beyond basic functional testing and consider aspects like accessibility,
payments, localisation, UX and customer journey testing. However, investing
heavily in testing infrastructure, employing skilled QA engineers and rigorously
testing every feature before release is expensive and slow. ... Quality
engineers are limited by budget constraints, which can affect everything from
resource allocation to investments in tooling. However, underfunding quality
efforts can have disastrous effects on customer satisfaction, revenues and
corporate reputation. To deliver competitive products within a reasonable
timeframe, quality managers need to use available budgets as efficiently as
possible.
Cloud security threats CISOs need to know about
An effective cloud security incident response plan details preparation,
detection and analysis, containment, eradication, recovery and post-incident
activities. Preparation involves establishing an incident response team with
defined roles, documented policies, necessary tools and a communication plan for
stakeholders. Detection and analysis require continuous monitoring, logging,
threat intelligence, incident classification and forensic analysis capabilities.
Containment strategies and eradication processes are essential to prevent the
spread of incidents and eliminate threats, followed by detailed recovery plans
to restore normal operations. Post-incident activities include documenting
actions, conducting root cause analysis, reviewing lessons learned, and updating
policies and procedures. ... Organizations should start by doing a comprehensive
risk assessment to identify critical assets and evaluate potential risks, such
as natural disasters and cyberattacks. Following the assessment, develop and
document DR and BC procedures. Annually review and update the procedures to
reflect changes in the IT environment and emerging threats.
Artificial Intelligence Versus the Data Engineer
So, how does AI change the role of the data engineer? Firstly, the role of the
data engineer has always been tricky to define. We sit atop a large pile of
technology, most of which we didn’t choose or build, and an even larger pile of
data we didn’t create, and we have to make sense of the world. Ostensibly, we
are trying to get to something scientific. A number, a chart, a result that we
can stand behind and defend—but like all great science, getting there also needs
a bit of art. That art comes in the form of the intuition required to sift
through the data, understand the technology, and rediscover all the little
real-world nuances and history that over time have turned some lovely clean data
into a messy representation of the real world. ... What’s exciting for us
beleaguered data engineers is that AI is showing great ability to be a very
helpful tool for these hard-to-master skills that will ultimately make us better
and more productive at our jobs. We have all, no doubt, seen all the great
advancements in AI’s ability to take plain text queries and turn them into
increasingly complex SQL, thus lightening the load of remembering all the
advanced syntax for whichever data platform is in vogue.
CrowdStrike crash showed us how invasive cyber security software is. Is there a better way?
In the wake of this incident it’s worth considering whether the tradeoffs made
by current EDR technology are the right ones. Abandoning EDR would be a gift to
cyber criminals. But cyber security technology can – and should – be done much
better. From a technical standpoint, Microsoft and CrowdStrike should work
together to ensure tools like Falcon operate at arm’s length from the core of
Microsoft Windows. That would greatly reduce the risk posed by future faulty
updates. Some mechanisms already exist that may allow this. Competing technology
to CrowdStrike’s Falcon already works this way. To protect user privacy, EDR
solutions should adopt privacy-preserving methods for data collection and
analysis. Apple has shown how data can be collected at scale from iPhones
without invading user privacy. To apply such methods to EDR, though, we’ll
likely need new research. More fundamentally, this incident raises questions
about why society continues to rely on computer software that is so demonstrably
unreliable.
6 Pillars Of Entrepreneurial Mastery: Elevating Your Business Through Lifelong Learning
Entrepreneurs with a growth mindset understand that abilities and intelligence
can be developed through dedication and hard work. This perspective fosters
resilience, helping to navigate setbacks and failures with a constructive
attitude. By viewing challenges as opportunities for growth, you can become more
adaptable and willing to take calculated risks. Regular self-reflection, seeking
feedback and staying open to new ideas are essential practices for cultivating
this mindset. ... As an entrepreneur, continuously educate yourself on tax
regulations, funding options and financial management best practices. Engaging
with online courses, workshops and financial mentors can provide valuable
insights and help stay abreast of emerging trends. ... In today's digital age,
technology is a major driver of business innovation and efficiency.
Entrepreneurs must stay informed about the latest technological advancements
relevant to their industry. This encompasses the implementation and utilization
of new software, tools, and platforms to streamline operations, enhance
productivity, and improve customer experiences.
Software Architecture in an AI World
Programming isn’t software architecture, a discipline that often doesn’t
require writing a single line of code. Architecture deals with the human and
organizational side of software development: talking to people about the
problems they want solved and designing a solution to those problems. That
doesn’t sound so hard, until you get into the details—which are often
unspoken. Who uses the software and why? How does the proposed software
integrate with the customer’s other applications? How does the software
integrate with the organization’s business plans? How does it address the
markets that the organization serves? Will it run on the customer’s
infrastructure, or will it require new infrastructure? On-prem or in the
cloud? How often will the new software need to be modified or extended? ...
Every new generation of tooling lets us do more than we could before. If AI
really delivers the ability to complete projects faster—and that’s still a big
if—the one thing that doesn’t mean is that the amount of work will decrease.
We’ll be able to take the time saved and do more with it: spend more time
understanding the customers’ requirements, doing more simulations and
experiments, and maybe even building more complex architectures.
Edge AI: Small Is the New Large
The technologies driving these advancements include AI-enabled chips, NPUs,
embedded operating systems, the software stack and pre-trained models.
Collectively, they form a SoC - system on chip. Software, hardware and
applications are key to enabling an intelligent device at the edge. The
embedded software stack in the chip brings it all together and makes it work.
Silicon Valley-based embedUR specializes in creating software stacks for
bespoke edge devices, acting as a "software integrator" that collaborates
closely with chip manufacturers to build custom solutions. "We have the
ability to build managed software, as well as build individual software stacks
for small, medium and large devices. You can think of us as a virtual R&D
team," Subramaniam said. ... OpenAI released a smaller version of the ChatGPT
language model called GPT-4o mini, set to be 60% cheaper than GPT-3.5. But
smaller does not mean less powerful, in terms of AI processing. Despite their
smaller size, SMLs possess substantial reasoning and language understanding
capabilities. For instance, Phi-2 has 2.7 billion parameters, Phi-3 has 7
billion, and Phi-3 mini has 3.8 billion.
Reflecting on Serverless: Current State, Community Thoughts, and Future Prospects
The great power of serverless is that starting with and becoming productive is
much easier. Just think how long it would take a developer who has never seen
either Lambda or Kubernetes to deploy a Hello World backend with public API on
both. As you start building more realistic production applications, the
complexity increases. You must take care of observability, security, cost
optimization, failure handling, etc. With non-serverless, this responsibility
usually falls on the operations team. With serverless, it usually falls on
developers, where there is considerable confusion. ... Issues like serverless
testing, serverless observability, learning to write a proper Lambda handler,
dealing with tenant isolation, working with infrastructure as code tools (too
many AWS options—SAM, CDK, Chalice, which one to choose and why?), and
learning all the best practices overwhelm developers and managers
alike. AWS has published articles on most topics, but there are many
opinions, too many 'hello world' projects that get deprecated within six
months, and not enough advanced use cases.
Quote for the day:
"You are the only one who can use your
ability. It is an awesome responsibility." -- Zig Ziglar
No comments:
Post a Comment