Daily Tech Digest by Kannan Subbiah: data lakes

Showing posts with label data lakes. Show all posts

Daily Tech Digest - May 14, 2026

Quote for the day:

“You may be disappointed if you fail, but you are doomed if you don’t try.” -- Beverly Sills

🎧 Listen to this digest on YouTube Music

Duration: 20 mins • Perfect for listening on the go.

CIOs are put to the test as security regulations across borders recalibrate

The European Union’s Cyber Resilience Act (CRA) marks a transformative shift in global cybersecurity, forcing Chief Information Officers to transition from traditional process-oriented compliance toward a rigorous focus on tangible product safety. Unlike previous frameworks, the CRA extends the CE mark to digital systems, mandating that software, firmware, and internet-connected devices be "secure by design" and "secure by default." This recalibration requires organizations to implement robust vulnerability reporting mechanisms by September 2026 and provide minimum five-year support lifecycles for security updates. CIOs now face the daunting task of overseeing the entire product ecosystem, which includes performing continuous risk assessments and actively managing open-source dependencies. They can no longer remain passive consumers of open-source technology; instead, they must contribute back to these communities to ensure the integrity of their own supply chains. While the regulation introduces significant administrative burdens—such as the creation of Software Bills of Materials and decade-long documentation retention—it also provides a strategic lever. Savvy IT leaders are leveraging these stringent mandates to secure board-level buy-in and the necessary budget for critical security improvements. Ultimately, the CRA demands a fundamental shift in responsibility, where CIOs are held accountable for the end-to-end security of the final products their organizations deliver to the market.

The Mathematics of Backlogs: Capacity Planning for Queue Recovery

The article "The Mathematics of Backlogs: Capacity Planning for Queue Recovery" explains that queue backlogs in distributed systems are predictable arithmetic challenges rather than random mysteries. At the heart of recovery is surplus capacity, defined as the difference between total processing power and arrival rate, meaning systems provisioned only for steady-state traffic will never naturally drain a backlog. A critical insight is the non-linear relationship between utilization and queue growth; as utilization approaches 100%, even minor traffic spikes cause exponential backlog accumulation. To manage this, the author highlights Little's Law for calculating queue delays and provides a clear formula for sizing consumer headroom based on specific Recovery Time Objectives (RTO). The piece also warns of "retry amplification," which can trigger metastable failure states where recovery efforts generate more load than they can actually resolve. In complex, multi-stage pipelines, identifying the true bottleneck is essential to avoid scaling the wrong component. Furthermore, engineers are encouraged to implement load shedding when drain times exceed message TTLs to prevent wasting expensive resources on stale data. Ultimately, by measuring specific metrics like peak backlog size and retry amplification factors after incidents, teams can transition from gut-based guesswork to data-driven operational intuition, ensuring significantly more resilient and predictable system performance during unforeseen failures.

Closing the gap between technical specs and business value through storytelling

Jay McCall’s article explores the critical necessity for infrastructure-focused software companies to pivot from technical specifications to value-driven storytelling. For businesses dealing with backend systems like APIs or security middleware, value is often defined by the absence of failure, making the product essentially invisible to non-technical executives. To bridge this gap, companies must stop relying on abstract metrics like uptime percentages and instead articulate the business outcomes and peace of mind their technology provides. The article advocates for the use of experiential demonstrations, such as AI-driven simulations, which allow prospects to engage with the software and witness its problem-solving capabilities firsthand. Additionally, visual workflows should prioritize the user’s journey over technical architecture, humanizing the product and placing it within a recognizable business context. Grounding these concepts in real-world "before and after" case studies further builds trust by offering tangible templates for success. Ultimately, crafting a repeatable narrative not only accelerates the sales cycle for internal teams but also empowers channel partners to communicate value effectively. By mastering the art of storytelling, technical organizations can translate complex backend sophistication into compelling business cases that resonate with decision-makers and facilitate sustainable scaling in a competitive market.

The Critical Fork: How Leaders Turn Failure Into Better Decisions

In the Forbes article "The Critical Fork: How Leaders Turn Failure Into Better Decisions," author Brent Dykes explores the pivotal moment leaders face when project results fail to meet expectations. He introduces the "Critical Fork" framework, which highlights a fundamental choice between two distinct paths: to deflect or to inspect. Deflection involves shifting blame toward external circumstances or team members, effectively shielding a leader's ego but simultaneously obstructing any potential for organizational growth or objective learning. In contrast, the inspection path encourages leaders to treat disappointing outcomes as valuable data points rather than personal setbacks. By choosing to inspect, organizations can uncover hidden root causes, challenge flawed underlying assumptions, and refine their future strategies with greater precision. Dykes argues that the most effective leaders cultivate a culture of psychological safety where failure is viewed not as a source of shame but as a vital catalyst for deeper analysis. This systematic approach transforms setbacks into "actionable insights," a hallmark of Dykes’ broader professional work in data storytelling and analytics. Ultimately, the article posits that leadership quality is defined less by initial successes and more by the ability to navigate these critical forks. By institutionalizing an inspection mindset, businesses foster resilience and ensure every failure becomes a stepping stone toward more robust and informed strategic choices.

From Bottlenecks to Breakthroughs, Enterprises Are Rethinking Analytics in the Lakehouse Era

The article "From Bottlenecks to Breakthroughs: Enterprises Are Rethinking Analytics in the Lakehouse Era" examines the transformative shift in data management as organizations transition from fragmented architectures to unified platforms. It highlights the immense pressure on centralized data teams to deliver reliable insights at high speed while supporting the complex integrations required for generative AI. Historically, enterprises have faced significant bottlenecks caused by the siloing of data and AI, privacy concerns, and a heavy reliance on highly technical staff. To overcome these hurdles, the article advocates for the lakehouse architecture—pioneered by Databricks—as an open, unified foundation that merges the best features of data lakes and warehouses. By integrating these systems into a "Data Intelligence Platform," companies can democratize access across various skill sets through low-code solutions, such as those provided by Rivery. This evolution enables breakthrough efficiencies, including a reported 7.5x acceleration in data delivery and substantial cost reductions. Ultimately, the piece emphasizes that the winners in the modern era will be those who effectively harness unified governance and seamless orchestration to move beyond operational sprawl. By adopting these integrated strategies, enterprises can finally turn data chaos into actionable intelligence, fostering a proactive environment where AI and analytics thrive in tandem to drive competitive advantage.

Most Remediation Programs Never Confirm the Fix Actually Worked

The article titled "Most Remediation Programs Never Confirm the Fix Actually Worked" argues that despite unprecedented environment visibility, cybersecurity teams struggle to ensure that remediation efforts effectively eliminate underlying risks. Highlighting a stark disparity between exploitation speed and corporate response time, the piece references Mandiant’s M-Trends 2026 report, which identifies a negative mean time to exploit, contrasting sharply with a thirty-two-day median remediation period. The emergence of advanced AI-driven tools like Mythos has further compressed exploitation windows, making traditional "patch and pray" methods increasingly dangerous and obsolete. Many organizations mistakenly equate closing an administrative ticket with resolving a vulnerability; however, vendor patches can be bypassable, and temporary workarounds often fail under evolving network conditions. This critical issue is exacerbated by organizational friction, where security teams identify risks but rely on separate engineering departments to implement fixes, leading to fragmented communication and delayed technical actions. To address these systemic gaps, the article advocates for a fundamental shift from measuring activity to focusing on outcomes. Instead of simply verifying that a specific attack path is blocked, modern programs must incorporate rigorous revalidation to confirm the total removal of the exposure. Ultimately, true security is achieved not through ticket completion, but by creating a self-correcting feedback loop that measures risk closure.

What CISOs need to land a board role

As cybersecurity becomes a critical pillar of organizational stability, Chief Information Security Officers (CISOs) are increasingly pursuing board-level positions to bridge the gap between technical defense and strategic governance. To successfully land these roles, security leaders must shift their focus from operational execution to high-level oversight. The article emphasizes that boards are not seeking another technical operator; rather, they prioritize strategic insight, calm judgment, and the ability to articulate cybersecurity through the lenses of risk appetite, value creation, and long-term resilience. Aspiring CISOs should start by gaining experience in governance-heavy environments, such as non-profit boards or industry committees, to refine their understanding of organizational stewardship. Furthermore, investing in formal governance education, such as NACD or AICD certifications, is highly recommended to build credibility. Networking remains a vital component of the process, as many opportunities arise through established relationships. Effective candidates must also cultivate a "board bio" that highlights their expertise in financial management, regulatory navigation, and crisis response. By reframing cyber issues as matters of trust and corporate strategy rather than just technical threats, CISOs can demonstrate the unique value they bring to a board, ultimately helping companies navigate complex digital landscapes with confidence and strategic foresight.

Everything you need to know about how technology is changing business

Digital transformation is the strategic integration of technology to fundamentally overhaul business operations, efficiency, and effectiveness. Rather than merely replicating existing services in a digital format, a successful transformation involves rethinking core business models and organizational cultures to thrive in an increasingly tech-centric landscape. Key technological drivers include cloud computing, the Internet of Things, and the rapid evolution of artificial intelligence, particularly generative and agentic AI. While the COVID-19 pandemic accelerated adoption, today’s initiatives are fueled by the need to compete with nimble startups and navigate macroeconomic volatility. However, the process is notoriously complex, expensive, and risky, often requiring a shift in mindset from simple IT upgrades to comprehensive business reinvention. Despite criticisms of the term as industry hype, it represents a critical shift where technology is no longer a secondary support function but the primary engine for long-term growth. Experts emphasize that the foundation of this change is a robust, secure data platform that enables trustworthy AI operations. Ultimately, digital transformation is a continuous journey of innovation that enables established firms to adapt, scale, and deliver enhanced customer experiences. By prioritizing outcomes over buzzwords, organizations can bridge the gap between innovation and execution, ensuring they remain relevant in a global economy where every successful company is effectively a technology business.

Intelligent digital identity infrastructure for GenAI

The article explores the transformative convergence of the Modular Open Source Identity Platform (MOSIP) and Generative Artificial Intelligence (GenAI) to build a sophisticated, intelligent digital identity infrastructure. As a foundational digital public good, MOSIP offers a vendor-neutral framework that preserves national digital sovereignty while ensuring secure and scalable citizen identity systems. By integrating GenAI, these platforms move beyond static registration to become intuitive, human-centric service hubs. Key benefits include the deployment of multilingual conversational assistants that assist underserved populations with enrollment, the automation of legacy record digitization through intelligent document processing, and enhanced fraud detection capable of identifying sophisticated AI-generated deepfakes. Furthermore, GenAI empowers administrators with natural language tools to derive actionable insights from complex demographic data. However, the author emphasizes that this integration must adhere to strict principles of privacy by design, explainability, and human oversight to prevent data exploitation and surveillance risks. By utilizing technologies like container orchestration, vector databases, and localized small language models, nations can create a modular and sovereign ecosystem. Ultimately, this synergy aims to transition identity from a mere database record to a dynamic "Identity as a Service," fostering global digital inclusion by bridging literacy and language barriers for citizens everywhere.

73 Seconds to Breach, 24 Hours to Patch: The Case for Autonomous Validation

The article titled "73 Seconds to Breach, 24 Hours to Patch: The Case for Autonomous Validation" explores the widening performance gap between modern attackers and traditional security defenses. It highlights a startling reality where AI-driven threats can breach a network in just 73 seconds, while organizations typically require 24 hours or longer to deploy critical patches. This vulnerability is deepened by the fact that the median time from a CVE publication to a working exploit has plummeted to only ten hours as of 2026. According to the piece, the core challenge is not a lack of security software but the "spaghetti handoff"—the fragmented, slow communication between different teams and disconnected security tools. To address this, the article champions the transition to autonomous security validation, a strategy that merges Breach and Attack Simulation with automated penetration testing. By creating a continuous, AI-powered loop for alert triage, simulation, and remediation deployment, companies can eliminate manual bottlenecks and respond at machine speed. Ultimately, this shift is framed as a mandatory evolution for surviving the "Post-Mythos" era of cybersecurity, where defenses must become as proactive, dynamic, and rapid as the sophisticated, automated exploits they seek to prevent.

Daily Tech Digest - April 24, 2025

Quote for the day:

“Remember, teamwork begins by building trust. And the only way to do that is to overcome our need for invulnerability.” -- Patrick Lencioni

Algorithm can make AI responses increasingly reliable with less computational overhead

The algorithm uses the structure according to which the language information is organized in the AI's large language model (LLM) to find related information. The models divide the language information in their training data into word parts. The semantic and syntactic relationships between the word parts are then arranged as connecting arrows—known in the field as vectors—in a multidimensional space. The dimensions of space, which can number in the thousands, arise from the relationship parameters that the LLM independently identifies during training using the general data. ... Relational arrows pointing in the same direction in this vector space indicate a strong correlation. The larger the angle between two vectors, the less two units of information relate to one another. The SIFT algorithm developed by ETH researchers now uses the direction of the relationship vector of the input query (prompt) to identify those information relationships that are closely related to the question but at the same time complement each other in terms of content. ... By contrast, the most common method used to date for selecting the information suitable for the answer, known as the nearest neighbor method, tends to accumulate redundant information that is widely available. The difference between the two methods becomes clear when looking at an example of a query prompt that is composed of several pieces of information.

Bring Your Own Malware: ransomware innovates again

The approach taken by DragonForce and Anubis shows that cybercriminals are becoming increasingly sophisticated in the way they market their services to potential affiliates. This marketing approach, in which DragonForce positions itself as a fully-fledged service platform and Anubis offers different revenue models, reflects how ransomware operators behave like “real” companies. Recent research has also shown that some cybercriminals even hire pentesters to test their ransomware for vulnerabilities before deploying it. So it’s not just dark web sites or a division of tasks, but a real ecosystem of clear options for “consumers.” We may also see a modernization of dark web forums, which currently resemble the online platforms of the 2000s. ... Although these developments in the ransomware landscape are worrying, Secureworks researchers also offer practical advice for organizations to protect themselves. Above all, defenders must take “proactive preventive” action. Fortunately and unfortunately, this mainly involves basic measures. Fortunately, because the policies to be implemented are manageable; unfortunately, because there is still a lack of universal awareness of such security practices. In addition, organizations must develop and regularly test an incident response plan to quickly remediate ransomware activities.

Phishing attacks thrive on human behaviour, not lack of skill

Phishing draws heavily from principles of psychology and classic social engineering. Attacks often play on authority bias, prompting individuals to comply with requests from supposed authority figures, such as IT personnel, management, or established brands. Additionally, attackers exploit urgency and scarcity by sending warnings of account suspensions or missed payments, and manipulate familiarity by referencing known organisations or colleagues. Psychologs has explained that many phishing techniques bear resemblance to those used by traditional confidence tricksters. These attacks depend on inducing quick, emotionally-driven decisions that can bypass normal critical thinking defences. The sophistication of phishing is furthered by increasing use of data-driven tactics. As highlighted by TechSplicer, attackers are now gathering publicly available information from sources like LinkedIn and company websites to make their phishing attempts appear more credible and tailored to the recipient. Even experienced professionals often fall for phishing attacks, not due to a lack of intelligence, but because high workload, multitasking, or emotional pressure make it difficult to properly scrutinise every communication.

What Steve Jobs can teach us about rebranding

Humans like to think of themselves as rational animals, but it comes as no news to marketers that we are motivated to a greater extent by emotions. Logic brings us to conclusions; emotion brings us to action. Whether we are creating a poem or a new brand name, we won’t get very far if we treat the task as an engineering exercise. True, names are formed by putting together parts, just as poems are put together with rhythmic patterns and with rhyming lines, but that totally misses what is essential to a name’s success or a poem’s success. Consider Microsoft and Apple as names. One is far more mechanical, and the other much more effective at creating the beginning of an experience. While both companies are tremendously successful, there is no question that Apple has the stronger, more emotional experience. ... Different stakeholders care about different things. Employees need inspiration; investors need confidence; customers need clarity on what’s in it for them. Break down these audiences and craft tailored messages for each group. Identifying the audience groups can be challenging. While the first layer is obvious—customers, employees, investors, and analysts—all these audiences are easy to find and message. However, what is often overlooked is the individuals in those audiences who can more positively influence the rebrand. It may be a particular journalist, or a few select employees.

Coaching AI agents: Why your next security hire might be an algorithm

Like any new team member, AI agents need onboarding before operating at maximum efficacy. Without proper onboarding, they risk misclassifying threats, generating excessive false positives, or failing to recognize subtle attack patterns. That’s why more mature agentic AI systems will ask for access to internal documentation, historical incident logs, or chat histories so the system can study them and adapt to the organization. Historical security incidents, environmental details, and incident response playbooks serve as training material, helping it recognize threats within an organization’s unique security landscape. Alternatively, these details can help the agentic system recognize benign activity. For example, once the system knows what are allowed VPN services or which users are authorized to conduct security testing, it will know to mark some alerts related to those services or activities as benign. ... Adapting AI isn’t a one-time event, it’s an ongoing process. Like any team member, agentic AI deployments improve through experience, feedback, and continuous refinement. The first step is maintaining human-in-the-loop oversight. Like any responsible manager, security analysts must regularly review AI-generated reports, verify key findings, and refine conclusions when necessary.

Cyber insurance is no longer optional, it’s a strategic necessity

Once the DPDPA fully comes into effect, it will significantly alter how companies approach data protection. Many enterprises are already making efforts to manage their exposure, but despite their best intentions, they can still fall victim to breaches. We anticipate that the implementation of DPDPA will likely lead to an increase in the uptake of cyber insurance. This is because the Act clearly outlines that companies may face penalties in the event of a data breach originating from their environment. Since cyber insurance policies often include coverage for fines and penalties, this will become an increasingly important risk-transfer tool. ... The critical question has always been: how can we accurately quantify risk exposure? Specifically, if a certain event were to occur, what would be the financial impact? Today, there are advanced tools and probabilistic models available that allow organisations to answer this question with greater precision. Scenario analyses can now be conducted to simulate potential events and estimate the resulting financial impact. This, in turn, helps enterprises determine the appropriate level of insurance coverage, making the process far more data-driven and objective. Post-incident technology also plays a crucial role in forensic analysis. When an incident occurs, the immediate focus is on containment.

Adversary-in-the-Middle Attacks Persist – Strategies to Lessen the Impact

One of the most recent examples of an AiTM attack is the attack on Microsoft 365 with the PhaaS toolkit Rockstar 2FA, an updated version of the DadSec/Phoenix kit. In 2024, a Microsoft employee accessed an attachment that led them to a phony website where they authenticated the attacker’s identity through the link. In this instance, the employee was tricked into performing an identity verification session, which granted the attacker entry to their account. ... As more businesses move online, from banks to critical services, fraudsters are more tempted by new targets. The challenges often depend on location and sector, but one thing is clear: Fraud operates without limitations. In the United States, AiTM fraud is progressively targeting financial services, e-commerce and iGaming. For financial services, this means that cybercriminals are intercepting transactions or altering payment details, inducing hefty losses. Concerning e-commerce and marketplaces, attackers are exploiting vulnerabilities to intercept and modify transactions through data manipulation, redirecting payments to their accounts. ... As technology advances and fraud continues to evolve with it, we face the persistent challenge of increased fraudster sophistication, threatening businesses of all sizes.

From legacy to lakehouse: Centralizing insurance data with Delta Lake

Centralizing data and creating a Delta Lakehouse architecture significantly enhances AI model training and performance, yielding more accurate insights and predictive capabilities. The time-travel functionality of the delta format enables AI systems to access historical data versions for training and testing purposes. A critical consideration emerges regarding enterprise AI platform implementation. Modern AI models, particularly large language models, frequently require real-time data processing capabilities. The machine learning models would target and solve for one use case, but Gen AI has the capability to learn and address multiple use cases at scale. In this context, Delta Lake effectively manages these diverse data requirements, providing a unified data platform for enterprise GenAI initiatives. ... This unification of data engineering, data science and business intelligence workflows contrasts sharply with traditional approaches that required cumbersome data movement between disparate systems (e.g., data lake for exploration, data warehouse for BI, separate ML platforms). Lakehouse creates a synergistic ecosystem, dramatically accelerating the path from raw data collection to deployed AI models generating tangible business value, such as reduced fraud losses, faster claims settlements, more accurate pricing and enhanced customer relationships.

How AI and Data-Driven Decision Making Are Reshaping IT Ops

Rather than relying on intuition, IT decision-makers now lean on insights drawn from operational data, customer feedback, infrastructure performance, and market trends. The objective is simple: make informed decisions that align with broader business goals while minimizing risk and maximizing operational efficiency. With the help of analytics platforms and business intelligence tools, these insights are often transformed into interactive dashboards and visual reports, giving IT teams real-time visibility into performance metrics, system anomalies, and predictive outcomes. A key evolution in this approach is the use of predictive intelligence. Traditional project and service management often fall short when it comes to anticipating issues or forecasting success. ... AI also helps IT teams uncover patterns that are not immediately visible to the human eye. Predictive models built on historical performance data allow organizations to forecast demand, manage workloads more efficiently, and preemptively resolve issues before they disrupt service. This shift not only reduces downtime but also frees up resources to drive innovation across the enterprise. Moreover, companies that embrace data as a core business asset tend to nurture a culture of curiosity and informed experimentation.

The DFIR Investigative Mindset: Brett Shavers On Thinking Like A Detective

You must be technical. You have to be technically proficient. You have to be able to do the actual technical work. And I’m not to rely on- not to bash a vendor training for a tool training, you have to have tool training, but you have to have exact training on “This is what the registry is, this is how you pull the-” you have to have that information first. The basics. You gotta have the basics, you have the fundamentals. And a lot of people wanna skip that. ... The DF guys, it’s like a criminal case. It’s “This is the computer that was in the back of the trunk of a car, and that’s what we got.” And the IR side is “This is our system and we set up everything and we can capture what we want. We can ignore what we want.” So if you’re looking at it like “Just in case something is gonna be criminal we might want to prepare a little bit,” right? So that makes DF guys really happy. If they’re coming in after the fact of an IR that becomes a case, a criminal case or a civil litigation where the DF comes in, they go, “Wow, this is nice. You guys have everything preserved, set up as if from the start you were prepared for this.” And it’s “We weren’t really prepared. We were prepared for it, we’re hoping it didn’t happen, we got it.” But I’ve walked in where drives are being wiped on a legal case.

Daily Tech Digest - January 06, 2025

Should States Ban Mandatory Human Microchip Implants?

“U.S. states are increasingly enacting legislation to pre-emptively ban employers from forcing workers to be ‘microchipped,’ which entails having a subdermal chip surgically inserted between one’s thumb and index finger," wrote the authors of the report. "Internationally, more than 50,000 people have elected to receive microchip implants to serve as their swipe keys, credit cards, and means to instantaneously share social media information. This technology is especially popular in Sweden, where chip implants are more widely accepted to use for gym access, e-tickets on transit systems, and to store emergency contact information.” ... “California-based startup Science Corporation thinks that an implant using living neurons to connect to the brain could better balance safety and precision," Singularity Hub wrote. "In recent non-peer-reviewed research posted on bioarXiv, the group showed a prototype device could connect with the brains of mice and even let them detect simple light signals.” That same piece quotes Alan Mardinly, who is director of biology at Science Corporation, as saying that the advantages of a biohybrid implant are that it "can dramatically change the scaling laws of how many neuros you can interface with versus how much damage you do to the brain."

AI revolution drives demand for specialized chips, reshaping global markets

There’s now a shift toward smaller AI models that only use internal corporate data, allowing for more secure and customizable genAI applications and AI agents. At the same time, Edge AI is taking hold, because it allows AI processing to happen on devices (including PCs, smartphones, vehicles and IoT devices), reducing reliance on cloud infrastructure and spurring demand for efficient, low-power chips. “The challenge is if you’re going to bring AI to the masses, you’re going to have to change the way you architect your solution; I think this is where Nvidia will be challenged because you can’t use a big, complex GPU to address endpoints,” said Mario Morales, a group vice president at research firm IDC. “So, there’s going to be an opportunity for new companies to come in — companies like Qualcomm, ST Micro, Renesas, Ambarella and all these companies that have a lot of the technology, but now it’ll be about how to use it. ... Enterprises and other organizations are also shifting their focus from single AI models to multimodal AI, or LLMs capable of processing and integrating multiple types of data or “modalities,” such as text, images, audio, video, and sensory input. The input from diverse resources creates a more comprehensive understanding of that data and enhances performance across tasks.

How to Address an Overlooked Aspect of Identity Security: Non-human Identities

Compromised identities and credentials are the No. 1 tactic for cyber threat actors and ransomware campaigns to break into organizational networks and spread and move laterally. Identity is the most vulnerable element in an organization’s attack surface because there is a significant misperception around what identity infrastructure (IDP, Okta, and other IT solutions) and identity security providers (PAM, MFA, etc.) can protect. Each solution only protects the silo that it is set up to secure, not an organization’s complete identity landscape, including human and non-human identities (NHIs), privileged and non-privileged users, on-prem and cloud environments, IT and OT infrastructure, and many other areas that go unmanaged and unprotected. ... Most organizations use a combination of on-prem management tools, a mix of one or more cloud identity providers (IdPs), and a handful of identity solutions (PAM, IGA) to secure identities. But each tool operates in a silo, leaving gaps and blind spots that cause increased attacks and blind spots. 8 out of 10 organizations cannot prevent the misuse of service accounts in real-time due to visibility and security being sporadic or missing. NHIs fly under the radar as security and identity teams sometimes don’t even know they exist.

Version Control in Agile: Best Practices for Teams

With multiple developers working on different features, fixes, or updates simultaneously, it’s easy for code to overlap or conflict without clear guidelines. Having a structured branching approach prevents confusion and minimizes the risk of one developer’s work interfering with another’s. ... One of the cornerstones of good version control is making small, frequent commits. In Agile development, progress happens in iterations, and version control should follow that same mindset. Large, infrequent commits can cause headaches when it’s time to merge, increasing the chances of conflicts and making it harder to pinpoint the source of issues. Small, regular commits, on the other hand, make it easier to track changes, test new functionality, and resolve conflicts early before they grow into bigger problems. ... An organized repository is crucial to maintaining productivity. Over time, it’s easy for the repository to become cluttered with outdated branches, unnecessary files, or poorly named commits. This clutter slows down development, making it harder for team members to navigate and find what they need. Teams should regularly review their repositories and remove unused branches or files that are no longer relevant.

Abusing MLOps platforms to compromise ML models and enterprise data lakes

Machine learning operations (MLOps) is the practice of deploying and maintaining ML models in a secure, efficient and reliable way. The goal of MLOps is to provide a consistent and automated process to be able to rapidly get an ML model into production for use by ML technologies. ... There are several well-known attacks that can be performed against the MLOps lifecycle to affect the confidentiality, integrity and availability of ML models and associated data. However, performing these attacks against an MLOps platform using stolen credentials has not been covered in public security research. ... Data poisoning: This attack involves an attacker having access to the raw data being used in the “Design” phase of the MLOps lifecycle to include attacker-provided data or being able to directly modify a training dataset. The goal of a data poisoning attack is to be able to influence the data that is being trained in an ML model and eventually deployed to production. ... Model extraction attacks involve the ability of an attacker to steal a trained ML model that is deployed in production. An attacker could use a stolen model to extract sensitive training data such as the training weights used, or to use the predictive capabilities used in the model for their own financial gain.

Get Going With GitOps

GitOps implementations have a significant impact on infrastructure automation by providing a standardized, repeatable process for managing infrastructure as code, Rose says. The approach allows faster, more reliable deployments and simplifies the maintenance of infrastructure consistency across diverse environments, from development to production. "By treating infrastructure configurations as versioned artifacts in Git, GitOps brings the same level of control and automation to infrastructure that developers have enjoyed with application code." ... GitOps' primary benefit is its ability to enable peer review for configuration changes, Peele says. "It fosters collaboration and improves the quality of application deployment." He adds that it also empowers developers -- even those without prior operations experience -- to control application deployment, making the process more efficient and streamlined. Another benefit is GitOps' ability to allow teams to push minimum viable changes more easily, thanks to faster and more frequent deployments, says Siri Varma Vegiraju, a Microsoft software engineer. "Using this strategy allows teams to deploy multiple times a day and quickly revert changes if issues arise," he explains via email.

Balancing proprietary and open-source tools in cyber threat research

First, it is important to assess the requirements of an organization by identifying the capabilities needed, such as threat intelligence platforms or malware analysis tools. Next, evaluating open-source tools which can be cost-effective and customizable, but may require community support and frequent updates. In contrast, proprietary tools could offer advanced features, dedicated support, and better integration with other products. Finally, think about scalability and flexibility, as future growth may necessitate scalable solutions. ... The technology is not magic, but it is a powerful tool to speed up processes and bolster security procedures while also reducing the gap between advanced and junior analysts. However, as of today, the technology still requires verification and validation. Globally, the need for security experts with a dual skill set in security and AI will be in high demand. Because the adoption of generative AI systems increases, we need people who understand these technologies because threat actors are also learning. ... If a CISO needs to evaluate effectiveness of these tools, they first need to understand their needs and pain points and then seek guidance from experts. Adopting generative AI security solutions just because it is the latest trend is not the right approach.

Get your IT infrastructure AI-ready

Artificial intelligence adoption is a challenge many CIOs grapple with as they look to the future. Before jumping in, their teams must possess practical knowledge, skills, and resources to implement AI effectively. ... AI implementation is costly and the training of AI models requires a substantial investment. "To realize the potential, you have to pay attention to what it's going to take to get it done, how much it's going to cost, and make sure you're getting a benefit," Ramaswami said. "And then you have to go get it done." GenAI has rapidly transformed from an experimental technology to an essential business tool, with adoption rates more than doubling in 2024, according to a recent study by AI at Wharton ... According to Donahue, IT teams are exploring three key elements: choosing language models, leveraging AI from cloud services, and building a hybrid multicloud operating model to get the best of on-premise and public cloud services. "We're finding that very, very, very few people will build their own language model," he said. "That's because building a language model in-house is like building a car in the garage out of spare parts." Companies look to cloud-based language models, but must scrutinize security and governance capabilities while controlling cost over time.

What is an EPMO? Your organization’s strategy navigator

The key is to ensure the entire strategy lifecycle is set up for success rather than endlessly iterating to perfect strategy execution. Without properly defining, governing, and prioritizing initiatives upfront, even the best delivery teams will struggle to achieve business goals in a way that drives the right return for the organization’s investment. For most organizations, there’s more than one gap preventing desired results. ... The EPMO’s job is to strip away unnecessary complexity and create frameworks that empower teams to deliver faster, more effectively, and with greater focus. PMO leaders should ask how this process helps to hit business goals faster. So by eliminating redundant meetings and scaling governance to match project size and risk, delivery timelines can shorten. This kind of targeted adjustment keeps momentum high without sacrificing quality or control. ... For an EPMO to be effective, ideally it needs to report directly to the C-suite. This matters because proximity equals influence. When the EPMO has visibility at the top, it can drive alignment across departments, break down silos, drive accountability, and ensure initiatives stay connected to overall business objectives serving as the strategy navigator for the C-suite.

Data Center Hardware in 2025: What’s Changing and Why It Matters

DPUs can handle tasks like network traffic management, which would otherwise fall to CPUs. In this way, DPUs reduce the load placed on CPUs, ultimately making greater computing capacity available to applications. DPUs have been around for several years, but they’ve become particularly important as a way of boosting the performance of resource-hungry workloads, like AI training, by completing AI accelerators. This is why I think DPUs are about to have their moment. ... Recent events have underscored the risk of security threats linked to physical hardware devices. And while I doubt anyone is currently plotting to blow up data centers by placing secret bombs inside servers, I do suspect there are threat actors out there vying to do things like plant malicious firmware on servers as a way of creating backdoors that they can use to hack into data centers. For this reason, I think we’ll see an increased focus in 2025 on validating the origins of data center hardware and ensuring that no unauthorized parties had access to equipment during the manufacturing and shipping processes. Traditional security controls will remain important, too, but I’m betting on hardware security becoming a more intense area of concern in the year ahead.

Quote for the day:

"Nothing in the world is more common than unsuccessful people with talent." -- Anonymous

Daily Tech Digest - September 15, 2024

Data Lakes Evolve: Divisive Architecture Fuels New Era of AI Analytics

“Data lakes led to the spectacular failure of big data. You couldn’t find anything when they first came out,” Sanjeev Mohan, principal at the SanjMo tech consultancy, told Data Center Knowledge. There was no governance or security, he said. What was needed were guardrails, Mohan explained. That meant safeguarding data from unauthorized access and respecting governance standards such as GDPR. It meant applying metadata techniques to identify data. “The main need is security. That calls for fine-grained access control – not just throwing files into a data lake,” he said, adding that better data lake approaches can now address this issue. Now, different personas in an organization are reflected in different permissions settings. ... This type of control was not standard with early data lakes, which were primarily “append-only” systems that were difficult to update. New table formats changed this. Table formats like Delta Lake, Iceberg, and Hudi have emerged in recent years, introducing significant improvements in data update support. For his part, Sanjeev Mohan said standardization and wide availability of tools like Iceberg give end-users more leverage when selecting systems.

Data at the Heart of Digital Transformation: IATA's Story

It's always good to know what the business goals are, from a strategic perspective, which informs the data that is needed to enable digital transformation. Data is at the heart of digital transformation. Business strategy comes first and then data strategy, followed by technology strategy. At IATA, we formed the Data Steering Group and identified critical datasets across the organization. We then set up a data catalog and established a governance structure. This was followed by the launch of the Data Governance Committee and the role of a chief data officer. We're going to be implementing an automated data catalog and some automation tools around data quality. Data governance has allowed us to break down data silos. It has also enabled us to establish IATA's industry data strategy. We treat data as an asset, and that data is not owned by any particular division but looked at holistically at the organizational level. And that has allowed us opportunities to do some exciting things in the AI and analytics space and even in the way we deal with our third-party data suppliers and member airlines.

New Android Warning As Hackers Install Backdoor On 1.3 Million TV Boxes

"This is a clear example of how IoT devices can be exploited by malicious actors,” Ray Kelly, fellow at the Synopsys Software Integrity Group, said, “the ability of the malware to download arbitrary apps opens the door to a range of potential threats.” Everything from a TV box botnet for use in distributed denial of service attacks through to stealing account credentials and personal information. Responsibility for protecting users lies with the manufacturers, Kelly said, they must “ensure their products are thoroughly tested for security vulnerabilities and receive regular software updates.” "These off-brand devices discovered to be infected were not Play Protect certified Android devices,” a Google spokesperson said, “If a device isn't Play Protect certified, Google doesn’t have a record of security and compatibility test results.” Whereas these Play Protect certified devices have undergone testing to ensure both quality and user safety, other boxes may not have done. “To help you confirm whether or not a device is built with Android TV OS and Play Protect certified, our Android TV website provides the most up-to-date list of partners,” the spokesperson said.

Engineers Day: Top 5 AI-powered roles every engineering graduate should consider

Generative AI engineer: They play a pivotal role in analysing vast datasets to extract actionable insights and drive data-informed decision-making processes. This role demands a comprehensive understanding of statistical analysis, machine learning techniques, and programming languages such as Python and R. ... AI research scientist: They are at the forefront of advancing AI technologies through groundbreaking research and innovation. With a robust mathematical background, professionals in this role delve into programming languages such as Python and C++, harnessing the power of deep learning, natural language processing, and computer vision to develop cutting-edge solutions. ... Machine Learning engineer: Machine learning engineers are tasked with developing cutting-edge machine learning models and algorithms to address complex problems across various industries. To excel in this role, professionals must develop a strong proficiency in programming languages such as Python, along with a deep understanding of machine learning frameworks like TensorFlow and PyTorch. Expertise in data preprocessing techniques and algorithm development is also quite crucial here.

Kubernetes attacks are growing: Why real-time threat detection is the answer for enterprises

Attackers are ruthless in pursuing the weakest threat surface of an attack vector, and with Kubernetes containers runtime is becoming a favorite target. That’s because containers are live and processing workloads during the runtime phase, making it possible to exploit misconfigurations, privilege escalations or unpatched vulnerabilities. This phase is particularly attractive for crypto-mining operations where attackers hijack computing resources to mine cryptocurrency. “One of our customers saw 42 attempts to initiate crypto-mining in their Kubernetes environment. Our system identified and blocked all of them instantly,” Gil told VentureBeat. Additionally, large-scale attacks, such as identity theft and data breaches, often begin once attackers gain unauthorized access during runtime where sensitive information is used and thus more exposed. Based on the threats and attack attempts CAST AI saw in the wild and across their customer base, they launched their Kubernetes Security Posture Management (KSPM) solution this week. What is noteworthy about their approach is how it enables DevOps operations to detect and automatically remediate security threats in real-time.

Begun, the open source AI wars have

Open source leader julia ferraioli agrees: "The Open Source AI Definition in its current draft dilutes the very definition of what it means to be open source. I am absolutely astounded that more proponents of open source do not see this very real, looming risk." AWS principal open source technical strategist Tom Callaway said before the latest draft appeared: "It is my strong belief (and the belief of many, many others in open source) that the current Open Source AI Definition does not accurately ensure that AI systems preserve the unrestricted rights of users to run, copy, distribute, study, change, and improve them." ... Afterwards, in a more sorrowful than angry statement, Callaway wrote: "I am deeply disappointed in the OSI's decision to choose a flawed definition. I had hoped they would be capable of being aspirational. Instead, we get the same excuses and the same compromises wrapped in a facade of an open process." Chris Short, an AWS senior developer advocate, Open Source Strategy & Marketing, agreed. He responded to Callaway that he: "100 percent believe in my soul that adopting this definition is not in the best interests of not only OSI but open source at large will get completely diluted."

What North Korea’s infiltration into American IT says about hiring

Agents working for the North Korean government use stolen identities of US citizens, create convincing resumes with generative AI (genAI) tools, and make AI-generated photos for their online profiles. Using VPNs and proxy servers to mask their actual locations — and maintaining laptop farms run by US-based intermediaries to create the illusion of domestic IP addresses — the perpetrators use either Western-based employees for online video interviews or, less successfully, real-time deepfake videoconferencing tools. And they even offer up mailing addresses for receiving paychecks. ... Among her assigned tasks, Chapman maintained a PC farm of computers used to simulate a US location for all the “workers.” She also helped launder money paid as salaries. The group even tried to get contractor positions at US Immigration and Customs Enforcement and the Federal Protective Services. (They failed because of those agencies’ fingerprinting requirements.) They did manage to land a job at the General Services Administration, but the “employee” was fired after the first meeting. A Clearwater, FL IT security company called KnowBe4 hired a man named “Kyle” in July. But it turns out that the picture he posted on his LinkedIn account was a stock photo altered with AI.

Contesting AI Safety

The dangers posed by these machines arise from the idea that they “transcend some of the limitations of their designers.” Even if rampant automation and unpredictable machine behavior may destroy us, the same technology promises unimaginable benefits in the far future. Ahmed et al. describe this epistemic culture of AI safety that drives much of today’s research and policymaking, focused primarily on the technical problem of aligning AI. This culture traces back to the cybernetics and transhumanist movements. In this community, AI safety is understood in terms of existential risks—unlikely but highly impactful events, such as human extinction. The inherent conflict between a promised utopia and cataclysmic ruin characterizes this predominant vision for AI safety. Both the AI Bill of Rights and SB 1047 assert claims about what constitutes a safe AI model but fundamentally disagree on the definition of safety. A model deemed safe under SB 1047 might not satisfy the Safe and Effective principle of the White House AI Blueprint; a model that follows the AI Blueprint could cause critical harm. What does it truly mean for AI to be safe?

Why Companies Should Embrace Ethical Hackers

Security researchers (or hackers, take your pick) are generally good people motivated by curiosity, not malicious intent. Making guesses, taking chances, learning new things, and trying and failing and trying again is fun. The love of the game and ethical principles are two separate things, but many researchers have both in spades. Unfortunately, the government has historically sided with corporations. Scared by the Matthew Broderick movie WarGames plot, Ronald Reagan initiated legislation that resulted in the Computer Fraud and Abuse Act of 1986 (CFAA). Good-faith researchers have been haunted ever since. Then there is The Digital Millennium Copyright Act (DMCA) of 1998, which made it explicitly illegal to “circumvent a technological measure that effectively controls access to a work protected under [copyright law],” something necessary to study many products. A narrow harbor for those engaging in encryption research was carved out in the DMCA, but otherwise, the law put researchers further in danger of legal action against them. All this naturally had a chilling effect as researchers grew tired of being abused for doing the right thing. Many researchers stopped bothering with private disclosures to companies with vulnerable products and took their findings straight to the public.

Why AI Isn't Just Hype - But A Pragmatic Approach Is Required

It is far better to take a pragmatic view where you open yourself up to the possibilities but proceed with both caution and some help. That must start with working through the buzzwords and trying to understand what people mean, at least at a top level, by an LLM or a vector search or maybe even a Naive Bayes algorithm. But then, it is also important to bring in a trusted partner to help you move to the next stage to build an amazing new digital product, or to undergo a digital transformation with an existing digital product. Whether you’re in start-up mode, you are already a scale-up with a new idea, or you’re a corporate innovator looking to diversify with a new product – whatever the case, you don’t want to waste time learning on the job, and instead want to work with a small, focused team who can deliver exceptional results at the speed of modern digital business. ... Whatever happens or doesn’t happen to GenAI, as an enterprise CIO you are still going to want to be looking for tech that can learn and adapt from circumstance and so help you do the same. At the end of the day, hype cycle or not, AI is really the one tool in the toolbox that can continuously work with you to analyse data in the wild and in non-trivial amounts.

Quote for the day:

"Your attitude is either the lock on or key to your door of success." -- Denis Waitley

Daily Tech Digest - July 22, 2024

AI regulation in peril: Navigating uncertain times

Existing laws are often vague in many fields, including those related to the environment and technology, leaving interpretation and regulation to the agencies. This vagueness in legislation is often intentional, for both political and practical reasons. Now, however, any regulatory decision by a federal agency based on those laws can be more easily challenged in court, and federal judges have more power to decide what a law means. This shift could have significant consequences for AI regulation. Proponents argue that it ensures a more consistent interpretation of laws, free from potential agency overreach. However, the danger of this ruling is that in a fast-moving field like AI, agencies often have more expertise than the courts. ... The judicial branch has no such existing expertise. Nevertheless, the majority opinion said that “…agencies have no special competence in resolving statutory ambiguities. Courts do.” ... Going forward, then, when passing a new law affecting the development or use of AI, if Congress wished for federal agencies to lead on regulation, they would need to state this explicitly within the legislation. Otherwise, that authority would reside with the federal courts.

Fostering Digital Trust in India's Digital Transformation journey

In this era where digital interactions dominate, trust is the anchor for building resilient organizations and stronger relationships with stakeholders and customers. As per ISACA’s State of Digital Trust 2023 research, 90 percent of respondents in India say digital trust is important and 89 percent believe its importance will increase in the next five years. Nowhere is this truer than in India, the world’s largest digitally connected democracy and a burgeoning hub of digital innovation and transformation. ... A key hurdle in building and maintaining digital trust in most countries is the absence of a standardized conceptual framework for measurement and access to reliable internet infrastructure and digital literacy. In India’s case, with a rapidly expanding digital footprint comes an equally high threat of issues such as lack of funding, unavailability of technological resources, shortage of skilled workforce, lack of alignment between digital trust and enterprise goals, inadequate governance mechanisms, the spread of misinformation through social media, etc. leading to financial fraud and data theft.

Tech debt: the hidden cost of innovation

While tech debt may seem like an unavoidable cost for any business heavily investing in innovation, delving deeper into its causes can reveal issues that may derail operations entirely. Many organisations struggle to find a solution, as the time required for risk analysis can seem unfeasible. Yet, by recognising early signs, businesses can leverage the right tools and find the right partners to facilitate a low-risk and controlled modernisation of legacy systems. Any IT modernisation program requires a strategic, evidence-based approach, starting with a rigorous fact-finding process to identify opportunities and inefficiencies within legacy systems. ... Making a case for modernisation requires articulating the expected benefits, costs and challenges beforehand. This begins with a comprehensive analysis that identifies existing system functionality and data against business and technical requirements, highlighting any gaps or challenges. ... In extreme situations, it may be necessary to replace an entire system. This is always the last resort due to the large investment needed and the disruption it can cause.

Fake Websites, Phishing Surface in Wake CrowdStrike Outage

These fake sites often promise quick fixes or falsely offer cryptocurrency rewards to lure visitors into accessing malicious content. George Kurtz, CEO of CrowdStrike, emphasized the importance of using official communication channels, urging customers to be wary of imposters. "Our team is fully mobilized to secure and stabilize our customers' systems," Kurtz said, noting the significant increase in phishing emails and phone calls impersonating CrowdStrike support staff. Imposters have also posed as independent researchers selling fake recovery solutions, further complicating efforts to resolve the outage. Rachel Tobac, founder of SocialProof Security, warned about social engineering threats in a series of tweets on X, formerly Twitter. "Criminals are exploiting the outage as cover to trick victims into handing over passwords and other sensitive codes," Tobac warned. She advised users to verify the identity of anyone requesting sensitive information. The surge in cybercriminal activity in the wake of the outage follows a common tactic used by cybercriminals to exploit chaotic situations.

Under-Resourced Maintainers Pose Risk to Africa's Open Source Push

To shore up security and avoid the dangers of under-resourced projects, companies have a few options, all starting with determining which OSS their developers and operations rely on. To that end, software bills of materials (SBOMs) and software composition analysis (SCA) software can help enumerate what's in the environment, and potentially help trim down the number of packages that companies need to check, verify, and manage, says Chris Hughes, chief security adviser for software supply chain security firm Endor Labs. "There's simply so much software, so many projects, so many libraries, that the idea of ... monitoring them all actively is just — it's very hard," he says. Finally, educating developers and package managers on how to produce and manage code securely is another area that can produce significant gains. The OpenSSF, for example, has created a free course LFD 121 as part of that effort. "We'll be building a course on security architectures, which will also be released later this year," OpenSSF's Arasaratnam says. "As well as a course on security for not just engineers, but engineering managers, as we believe that's a critical part of the equation."

Cross-industry standards for data provenance in AI

Knowing the source and history of datasets can help organizations better assess their reliability and suitability for training or fine-tuning AI models. This is crucial because the quality of training data directly affects the performance and accuracy of AI models. Understanding the characteristics and limitations of the training data also allows for a better assessment of model performance and potential failure modes. ... As AI regulations such as the EU AI Act evolve, data provenance becomes increasingly important for demonstrating compliance. It allows organizations to show that they use data appropriately and align with relevant laws and regulations. ... Organizations should start by reviewing the standards documentation, including the Executive Overview, use case scenarios, and technical specifications (available in GitHub). Launching a proof of concept (PoC) with a data provider is recommended to build internal confidence. Organizations lacking resources or deploying a PoC “light” may opt to use our metadata generator tool to create and access standardized metadata files

Why an Agile Culture Is Critical for Enterprise Innovation

In the end, embracing agility isn’t just about staying afloat in the turbulent waters of AI innovation; it’s about turning those waves into opportunities for growth and transformation. Because in this ever-evolving landscape, the businesses that thrive will be the ones that are flexible, responsive, and always ready to adapt to whatever comes next. Which brings me to my next point – you need to start loving failure. This requires a whole reframe because in the world of AI, getting things wrong can actually be the fastest way to get things right. Most companies are so scared of getting it wrong that they never try anything new and are frozen like a deer in headlights. In AI, that’s a death sentence. ... Be prepared for resistance. Change is scary, and you’ll always have a few “blockers” who are negative in their approach. These are the people you need to win over the most. In the meantime, you just need to weather the storm. Lastly, remember that becoming agile is a journey, not a destination. It’s about creating a mindset of continuous improvement. Always in beta? That’s absolutely fine and in the fast-paced world of AI, that’s exactly where you want to be.

The Rise of Cybersecurity Data Lakes: Shielding the Future of Data

Beyond real-time threat detection and analysis, cybersecurity data lakes offer organizations a powerful platform for vulnerability prediction and risk assessment. By examining past incidents, organizations can uncover trends and commonalities in security breaches, weak points in their defenses, and recurring threats. Cybersecurity data lakes store vast amounts of data spanning extended periods, which is a rich source of information for identifying recurring vulnerabilities or attack vectors. With techniques such as time-series analysis and pattern recognition, organizations can uncover historical vulnerability patterns through rigorous testing and use this knowledge to anticipate and mitigate future risks. In fact, this is one of the reasons why the global pentesting market is expected to rise to a value of $5 billion by 2031, with more innovative approaches like blackbox pentesting to exploit hidden attack vectors and using AI for vulnerability assessment (VAS) to improve efficiency. When combined with other vulnerability assessment methods like threat modeling and red team exercises, predictive modeling can also help organizations identify potential attack paths and attack surface areas and proactively implement defensive measures.

Internships can be a gold mine for cybersecurity hiring

Though an internship can pay off for an employer in the form of a fresh crop of talent to hire, it requires the company to invest time, planning, oversight, and resources. Designating one or more people to manage the process internally can make things easier for the organization. “Sit down with the supervisory personnel so they understand what that position is being advertised for, what the expected outcomes are and how to manage that intern, the program needs, and how they have to report [on that intern],” ... If possible, Smith recommends mentoring an intern, not simply ticking off a bureaucratic checklist of their tasks: “I do fervently believe you essentially need a sponsor, someone who’s going to take the intern under his or her wing and nurture that relationship, nurture that person.” Chiasson warns employers to manage their own expectations as carefully as they manage the interns themselves. Rather than expecting a unicorn to show up — an intern with one or more degrees, several technical certifications and other prior workplace experience — she urges companies to “take them on and then train them based on what you require.”

Desirable Data: How To Fall Back In Love With Data Quality

With so much data being pumped out at breakneck rates, it can seem like an insurmountable challenge to ensure data accuracy, completeness, and consistency. And despite technological, governance and team efforts, poor data can still endure. As such, maintaining data quality can feel like a perennial challenge. But quality data is fundamental to a company’s digital success. In order to create a business case for embracing data quality, you have to, firstly, demonstrate the far-reaching consequences of poor data quality on organisational performance. If you can present the problem from a business standpoint — backed by evidence and real-world scenarios of data quality issues leading to incurred costs, reputational risk, and uncapitalised opportunities — you can implement proactive measures and trigger a desire by top-level management to adapt processes. To bring your case to life, you then have to find ways of quantifying the business impact of data quality issues. This could take the form of illustrating the effect of bad data on a marketing campaign, showing the difference with and without data quality in relation to usable records, sales leads, and how this impacts your revenue.

Quote for the day:

"Defeat is not bitter unless you swallow it." -- Joe Clark