Daily Tech Digest - January 24, 2024

8 data strategy mistakes to avoid

Denying business users access to information because of data silos has been a problem for years. When different departments, business units, or groups keep data stored in systems not available to others, it diminishes the value of the data. Data silos result in inconsistencies and operational inefficiencies, says John Williams, executive director of enterprise data and advanced analytics at RaceTrac, an operator of convenience stores. ... Data governance should be at the heart of any data strategy. If not, the results can include poor data quality, lack of consistency, and noncompliance with regulations, among other issues. “Maintaining the quality and consistency of data poses challenges in the absence of a standardized data management approach,” Williams says. “Before incorporating Alation at RaceTrac, we struggled with these issues, resulting in a lack of confidence in the data and redundant efforts that impeded data-driven decision-making.” Organizations need to create a robust data governance framework, Williams says. This involves assigning data stewards, establishing transparent data ownership, and implementing guidelines for data accuracy, accessibility, and security.


Regulators probe Microsoft’s OpenAI ties — is it only smoke, or is there fire?

Given that some of the world’s most powerful agencies regulating antitrust issues are looking into Microsoft’s relationship with OpenAI, the company has much to fear. Are they being fair, though? Is there not just smoke, but also fire? You might argue that AI — and genAI in particular — is so new, and the market so wide open, that these kinds of investigations are exceedingly preliminary, would only hurt competition, and represent governmental overreach. After all, Google, Facebook, Amazon, and billion-dollar startups are all competing in the same market. That shows there’s serious competition. But that’s not quite the point. The OpenAI soap opera shows that OpenAI is separate from Microsoft in name only. If Microsoft can use its $13 billion investment to reinstall Altman (and grab a seat on the board), even if it’s a nonvoting one, it means Microsoft is essentially in charge of the company. Microsoft and OpenAI have a significant lead over all their competitors. If governments wait too long to probe what’s going on, that lead could become insurmountable. 


AI will make scam emails look genuine, UK cybersecurity agency warns

The NCSC, part of the GCHQ spy agency, said in its latest assessment of AI’s impact on the cyber threats facing the UK that AI would “almost certainly” increase the volume of cyber-attacks and heighten their impact over the next two years. It said generative AI and large language models – the technology that underpins chatbots – will complicate efforts to identify different types of attack such as spoof messages and social engineering, the term for manipulating people to hand over confidential material. “To 2025, generative AI and large language models will make it difficult for everyone, regardless of their level of cybersecurity understanding, to assess whether an email or password reset request is genuine, or to identify phishing, spoofing or social engineering attempts.” Ransomware attacks, which had hit institutions such as the British Library and Royal Mail over the past year, were also expected to increase, the NCSC said. It warned that the sophistication of AI “lowers the barrier” for amateur cybercriminals and hackers to access systems and gather information on targets, enabling them to paralyse a victim’s computer systems, extract sensitive data and demand a cryptocurrency ransom.


Burnout epidemic proves there's too much Rust on the gears of open source

An engineer is keen to work on the project, opens up the issue tracker, and finds something they care about and want to fix. It's tricky, but all the easy issues have been taken. Finding a mentor is problematic since, as Nelson puts it, "all the experienced people are overworked and burned out," so the engineer ends up doing a lot of the work independently. "Guess what you've already learned at this point," wrote Nelson. "Work in this project doesn't happen unless you personally drive it forward." The engineer becomes a more active contributor. So active that the existing maintainer turns over a lot of responsibilities. They wind up reviewing PRs and feeling responsible for catching mistakes. They can't keep up with the PRs. They start getting tired ... and so on. Burnout can manifest itself in many ways, and dodging it comes down to self-care. While the Rust Foundation did not wish to comment on the subject, the problem of burnout is as common – if not more so – in the open source world as it is in the commercial one.


Steadfast Leadership And Identifying Your True North

When you apply the idea of true north across all facets of your organization, you can effectively keep your team aligned and moving in tandem. But without a clear and definitive direction, there’s no way to gauge whether everyone is rowing in the same direction. Clarifying your distinct true north is just the beginning. Once it’s established, team members at all levels, especially leadership, must understand it, refer to it often, and measure performance against it. This looks like continuously reviewing departmental metrics and the attitudes of teams and individuals to ensure that they are in alignment with the organization’s cardinal direction. Leaders must be able to see the connection between the processes and goals of individual teams and how they contribute to or inhibit long-term goals. If individuals or teams work against the desired direction (sometimes unknowingly!), it can slow or, in some cases, even reverse progress. The antidote is long-term alignment, but this can only come after a deep understanding of how the day-to-day affects long-term success, which requires accurate metrics, widespread accountability, and thorough analysis.


The Rise of the Serverless Data Architectures

The big lesson is that there is no free lunch. You have to understand the tradeoffs. If I go with Aurora, I have to not think about some things, I have to think about other things. Transactions are not an issue. Cold start is an issue, minimum payment may be an issue. If I go with something like DynamoDB, then things are perfect, but I have a key-value store. There's all kinds of things to take into consideration and make sure that you understand what each system is actually capable of delivering. The one thing to note is that while you will have to make tradeoffs, if you decide you want a very elastic system, look at the situation. It does not require changing the whole way you ever use the database. Meaning if you like key-value stores, there will be several for you to choose from. If you like relational, there will be a bunch. If you like specific type of relational, and MySQL fans will be. If you like Postgres, there are going to be. You don't have to change a lot about your worldview. This is not the same case if you try serverless functions, which is, learn a whole new way to write code and manage code and so on, because I'm still trying to wrap my head around how to build functionality from a lot of small independent functions.


Navigating Generative AI Data Privacy and Compliance

Developers play a crucial role in protecting companies from the legal and ethical challenges linked to generative AI products. Faced with the risk of unintentionally exposing information (a longstanding problem) or now having the generative AI tool leak it on its own (as occurred when ChatGPT users reported seeing other people’s conversation histories), companies can implement strategies like the following to minimize liability and help ensure the responsible handling of customer data. ... Using anonymized and aggregated data serves as an initial barrier against the inadvertent exposure of individual customer information. Anonymizing data strips personally identifiable elements so that the generative AI system can learn and operate without associating specific details with individual users. ... Through meticulous access management, developers can restrict data access exclusively to individuals with specific tasks and responsibilities. By creating a tightly controlled environment, developers can proactively reduce the likelihood of data breaches, helping ensure that only authorized personnel can interact with and manipulate customer data within the generative AI system.


The Intersection of DevOps, Platform Engineering, and SREs

By automating manual processes, embracing continuous integration and continuous delivery (CI/CD), and instilling a mindset of shared responsibility, DevOps empowers teams to respond swiftly to market demands, ensuring that software is not just developed but delivered efficiently and reliably. Platform Engineering emerges as a key player in shaping the infrastructure that underpins modern applications. It is the architectural foundation that supports the deployment, scaling, and management of applications across diverse environments. The importance of Platform Engineering lies in providing a standardized, scalable, and efficient platform for development and operations teams. By offering a set of curated tools, services, and environments, Platform Engineers enable seamless collaboration and integration of DevOps practices. ... The importance of SREs lies in their dedication to ensuring the reliability, scalability, and high performance of systems and applications. SREs introduce a data-driven approach, defining service level objectives (SLOs) and error budgets to align technical operations with business objectives.


Quantum-secure online shopping comes a step closer

The researchers’ QDS protocol involves three parties: a merchant, a client and a third party (TP). It begins with the merchant preparing two sequences of coherent quantum states, while the client and the TP prepare one sequence of coherent states each. The merchant and client then send a state via a secure quantum channel to an intermediary, who performs an interference measurement and shares the outcome with them. The same process occurs between the merchant and the TP. These parallel processes enable the merchant to generate two keys that they use to create a signature for the contract via one-time universal hashing. Once this occurs, the merchant sends the contract and the signature to the client. If the client agrees with the contract, they use their quantum state to generate a key in a similar way as the merchant and send this key to the TP. Similarly, the TP generates a key from their quantum state after receiving the contract and signature. Both the client and the TP can verify the signature by calculating the hash function and comparing their result to the signature. 


The Top 10 Things Every Cybersecurity Professional Needs to Know About Privacy

The intersection between privacy and cybersecurity is ever increasing and the boundaries between the two ever blurring. By way of example – data breaches lived firmly in the realm of cybersecurity for many years. However, since the adoption of GDPR and mandatory disclosure requirements of several data protection and privacy laws around the world, the balance of responsibility and ownership of data breaches has become blurred. ... the language of privacy is very different from that of cybersecurity – cybersecurity professionals talk about penetration tests, vulnerability assessments, ransomware attacks, firewalls, operating systems, malware, anti-virus, etc. Meanwhile, privacy professionals talk about data protection impact assessments, case law judgements, privacy by design and default, legitimate interest assessments, proportionality, etc. In fact, the language of privacy is not even consistent in its own right, with much confusion between the fundamental differences between data protection and privacy and its definitions across jurisdictions.



Quote for the day:

"Leaders should influence others in such a way that it builds people up, encourages and edifies them so they can duplicate this attitude in others." -- Bob Goshen

Daily Tech Digest - January 23, 2024

How human robot collaboration will affect the manufacturing industry

Traditional manufacturing systems frequently struggle to adjust to shifting demands and product variances. Human-robot collaboration gives flexibility, which is critical in today’s market. Robots are easily programmed and reprogrammed, allowing firms to quickly alter production lines to suit new goods or design changes. This adaptability is critical in an era where customer preferences shift quickly, and companies are trying to work in line with the shifting preferences of the customers. ... While the initial investment in robotics technology may be significant, the long-term cost savings from human-robot collaboration are attractive. Automated procedures in the manufacturing industries lower labor costs, boost productivity, and reduce errors to a great extent, resulting in a more cost-effective manufacturing operation. ... There is a notion that automation will replace human occupations, on the contrary, the collaboration is intended to supplement human abilities. Human workers may focus on critical thinking, problem-solving, and creativity by automating mundane and physically demanding jobs.


Mastering System Design: A Comprehensive Guide to System Scaling for Millions

Horizontal scaling emerges as a strategic solution to accommodate increasing demands and ensure the system’s ability to handle a burgeoning user base. Horizontal scaling involves adding more servers to the system and distributing the workload across multiple machines. Unlike vertical scaling, which involves enhancing the capabilities of a single server, horizontal scaling focuses on expanding the server infrastructure horizontally. One of the key advantages of horizontal scaling is its potential to improve system performance and responsiveness. By distributing the workload across multiple servers, the overall processing capacity increases, alleviating performance bottlenecks and enhancing the user experience. Moreover, horizontal scaling offers improved fault tolerance and reliability. The redundancy introduced by multiple servers reduces the risk of a single point of failure. In the event of hardware issues or maintenance requirements, traffic can be seamlessly redirected to other available servers, minimizing downtime and ensuring continuous service availability. Scalability becomes more flexible with horizontal scaling. 


Backup admins must consider GenAI legal issues -- eventually

LLMs requiring a massive amount of data and, by proxy, dipping into nebulous legal territory is inherent to GenAI services contracts, said Andy Thurai, an analyst at Constellation Research. Many GenAI vendors are now offering indemnity or other legal protections for customers. ... "It's a [legal] can of worms that enterprises can't afford to open," Thurai said. Unfortunately for enterprise legal teams, the need to create guidance is fast approaching. Lawsuits by organizations such as the New York Times are looking to take back IP control and copyright from the OpenAI's proprietary and commercial LLM model. Those suits are entirely focused on the contents of data itself rather than the mechanics of backup and storage that backup admins would concern themselves with, said Mauricio Uribe, chair of the software/IT and electrical practice groups at law firm Knobbe Martens. The business advantages of GenAI within backup technology are still unproven and unknown, he added. Risks such as patent infringement remain a possibility. Backup vendors are implementing GenAI capabilities such as support chatbots into their tools now, such as Rubrik's Ruby and Cohesity's Turing AI. But neither incorporates enterprise customer data or specific customer information, according to both vendors.


CFOs urged to reassess privacy budgets amid rising data privacy concerns

The ISACA Privacy in Practice 2024 survey report reveals that only 34% of organizations find it easy to understand their privacy obligations. This lack of clarity can lead to non-compliance and increased risk of data breaches. Additionally, only 43% of organizations are very or completely confident in their privacy team’s ability to ensure data privacy and achieve compliance with new privacy laws and regulations. ... To address the challenges outlined in the survey, organizations are taking proactive steps to strengthen their privacy programs. Training plays a crucial role in mitigating workforce gaps and privacy failures. Half of the respondents (50%) note that they are training non-privacy staff to move into privacy roles, while 39% are increasing the usage of contract employees or outside consultants. Organizations are also investing in privacy awareness training for employees. According to the survey, 86% of organizations provide privacy awareness training, with 66% offering training to all employees annually. Moreover, 52% of respondents provide privacy awareness training to new hires. 


Cisco sees headway in quantum networking, but advances are slow

Cisco has said that it envisions quantum data centers that could use classic LAN models to tie together quantum computers, or a quantum-based network that transmits quantum bits (qubits) from quantum servers at high-speeds to handle commercial-grade applications. “Another trend will be the growing importance of quantum networking which in 4 or 5 years – perhaps more – will enable quantum computers to communicate and collaborate for more scalable quantum solutions,” Centoni stated. “Quantum networking will leverage quantum phenomena such as entanglement and superposition to transmit information.” The current path for quantum researchers and developers is to continue to grow radix, expand mesh networking (the ability for network fabrics to support many more connections per port and higher bandwidth), and create quantum switching and repeaters, Pandey said. “We want to be able to carry quantum signals over longer distances, because quantum signals deteriorate rapidly,” he said. “We definitely want to enable them to handle those signals within a data center footprint, and that’s technology we will start experimenting on.”


Navigating the Digital Transformation: The Role of IT

While many acknowledged engaging with the six core elements of the Rewired framework, few participants considered themselves frontrunners in significant progress. This underscores the complexity and ongoing nature of digital transformation, necessitating continuous adaptation across leadership, culture, and technology. Organizations are directing efforts towards both front-end (customer experience) and back-end (operational optimization), recognizing the interconnected nature of digital transformation. Success stories include consolidating Robotic Process Automation (RPE), Artificial Intelligence (AI), and low-code development within a single organizational department. This integration facilitates synergies and holistic advancements in digital capabilities. The evolving nature of ERP transformations was also discussed, with a shift towards continuous improvements and a focus on operating models and ways of working, moving beyond purely technological considerations. The insights from this roundtable underscore the multifaceted nature of digital transformation.


Harvard Scientists Discover Surprising Hidden Catalyst in Human Brain Evolution

“Brain tissue is metabolically expensive,” said the Human Evolutionary Biology assistant professor. “It requires a lot of calories to keep it running, and in most animals, having enough energy just to survive is a constant problem.” For larger-brained Australopiths to survive, therefore, something must have changed in their diet. Theories put forward have included changes in what these human ancestors consumed or, most popularly, that the discovery of cooking allowed them to garner more usable calories from whatever they ate. ... The shift was probably a happy accident. “This was not necessarily an intentional endeavor,” Hecht posited. “It may have been an accidental side effect of caching food. And maybe, over time, traditions or superstitions could have led to practices that promoted fermentation or made fermentation more stable or more reliable.” This hypothesis is supported by the fact that the human large intestine is proportionally smaller than that of other primates, suggesting that we adapted to food that was already broken down by the chemical process of fermentation. 


Digital Personal Data Protection Act marks a new era of business-friendly governance

Surprising the business community, the DPDP Act 2023 removed the data localization requirements, marking a significant departure from the previous iterations of the Act. The earlier DPDP Bills required certain categories of personal data to be stored and processed within the country. The provision faced staunch global opposition, particularly from the US, which criticized India's requirements as discriminatory and trade distortive. In contrast, the DPDP Act, 2023 adopts a more inclusive approach, granting firms autonomy in the choice and location of cloud services for storing and processing personal data of their users. By prioritizing cost-effectiveness and competitiveness for the firms, the removal of data localisation requirements signals a more accommodating government stance. In addition to scrapping data localization requirements, the DPDP Act 2023 also allows unrestricted cross-border transfer of Indian users’ personal data abroad, barring certain destination countries. Firms would not be required to conduct post-transfer impact assessments or to ensure that the destination country has similar data protection standards– mandated in other jurisdictions like the EU and Vietnam. 


Cybersecurity: The growing partnership between HR and risk management

HR professionals themselves can also be attractive targets to bad actors. The access they have to sensitive employee and company data can be a goldmine for hackers, putting a target on the back of those within the HR organization. As such, HR leaders should put proactive, pre-breach policies in place for their own functional colleagues. Policies might include contacting internal and external parties who ask for changes to sensitive information, such as invoice numbers, email passwords, direct deposit details, and software updates. They should also include policies for remote workers and incidence response. ... When you purchase cyber insurance, you get access to pre-breach planning and policy templates, which for many organizations, is just as important as the breach coverage. While the optimal amount of insurance depends on many factors — including size, revenues, number of employees and access to confidential information — HR organizations of all sizes and structures benefit from pre-breach planning and policymaking.


IT services spending signals major role change for CIOs ahead

“This evolution in what CIOs do, the value proposition they bring to the company, is evident in the long-term playout. But it is not yet as evident to the CIOs themselves,” Lovelock said. He sees CIOs still thinking they are riding the same talent waves of the past, facing a temporary problem that they will solve: that their staff will come back, that hiring will resume, that attrition rates will decline, and that they will be able to attract the skills they need at prices they can afford. “It doesn’t look like they will ever be able to do that. There are too many things IT staff with these key resources and skills are looking for that are outside of the CIO’s control to deliver,” he said. With increasing reliance on IT services and consulting to deliver outcomes ranging from commoditized customer support to differentiating generative AI implementations, the CIO role may soon become less about being that one-stop shop for business support, overseeing project and products developed in-house, and more about weaving together myriad services undertaken by an increasingly heterogeneous mix of talent sources, predominantly beyond the CIO’s direct purview.



Quote for the day:

''Thinking is easy acting is difficult, and to put one's thoughts into action is the most difficult thing in the world.'' -- Johann Wolfgang von Goethe

Daily Tech Digest - January 22, 2024

Cybersecurity Trends and Predictions 2024 From Industry Insiders

In this new world, private clouds and private infrastructure are a safer place to be. It's critical for security posture, especially for a brand that's been around a long time and its core functionalities haven't changed — it's critical to any modern environment despite the new threats. The basics haven't changed; they've just increased. Organizations need to be critical about their ITOps strategy to ensure configuration management and drift control, which is key to maintaining the security posture for an organization. Organizations will depend more on agents to manage configurations and prevent drift with the right set of technologies while tracking any and every change made to the golden images for configuration in their estate and keep their infrastructure inline as part of the security posture while also being secure in compliance standards. ... That fact won't stop startups from claiming that they have used GenAI to create a security silver bullet. While AI, particularly deep learning, will always have a place in solving security challenges, organizations will be better served by avoiding the AI panic and ensuring any security solutions help them optimize the security basics


Russia-based group hacked emails of Microsoft’s senior leadership

This is not the first time Midnight Blizzard or Nobelium has targeted the company. Last year, Microsoft had accused it of using social engineering to carry out a cyberattack on Microsoft Teams. Though the attack was initiated in late November 2023, it was detected only on January 12, 2024. “The incidence shows, like in earlier such cases, that even the most sophisticated cyber security systems are far from being adequate. ... Microsoft stressed that the attack was not because of a vulnerability in its products or services. “To date, there is no evidence that the threat actor had any access to customer environments, production systems, source code, or AI systems. We will notify customers if any action is required,” the company blog post read. However, analysts believe that possibly not enough was done to secure the email accounts of senior leadership. “The breach also hints at the possibility that best practices, such as zero-trust security, are not necessarily being applied to email accounts of senior leadership, who have been the primary targets in this case,” said Kumar. He added that a “weak link the security chain” might have led to the compromise of the employee emails.


The Ethical Frontier: Navigating the Moral Landscape of Big Data and AI

Corporations are now looking beyond the bottom line to uphold ethical practices as they leverage big data and AI. The first step in this direction is ensuring transparency. Companies need to be clear about how they are collecting data, what they’re using it for, and how AI algorithms make decisions. This transparency is crucial in building trust with consumers and stakeholders. Another pivotal aspect is the prevention of biases in AI. Machine learning algorithms can inadvertently perpetuate and amplify existing biases if they are fed with skewed datasets. Corporations must actively engage in ‘debiasing’ techniques and diversity initiatives to ensure fairness and inclusivity in AI-driven decisions. Privacy, too, cannot be an afterthought. With regulations like the General Data Protection Regulation (GDPR) setting a precedent, businesses are more accountable for protecting individual’s data. Implementing robust privacy measures and giving users control over their data is both an ethical obligation and a business imperative. Various ethical frameworks have been proposed to guide businesses in this new terrain. 


DevSecOps risks: How can Indian tech mitigate software hijacking

It’s not surprising that these hijacking methods have gained prominence in India in recent years, as up to 96% of applications contain at least one open-source component. As Indian developers collaborate on software production, there is one word they should become familiar with when it comes to securing the software development pipeline: Curation. At a high level, the word Curation is defined as the act of thoughtfully selecting and organising items, a process typically associated with articles, images, music, and so on. In this case, however, the items being curated are open-source software components, acting as an automated lock to safeguard the gateway of the software pipeline. It entails filtering, tracking, and managing software packages based on preset policies to ensure the use of reliable components across the development lifecycle. Curating software components streamline development by guaranteeing the safety, reliability, and current status of packages. The idea is to protect against both known and unknown risks through a comprehensive approach that strengthens the organisation’s software supply chain by establishing a trusted source of packages. Approved packages could then be cataloged for re-use, or to point.


CISOs are not just the keepers of our data – they must be its custodians

Effective navigation of this intricate regulatory landscape extends beyond mere compliance: it necessitates strategic, ongoing commitment. While data owners may define policies, custodians are responsible for implementing and ensuring adherence to these policies. The landscape of data custodianship in the digital age is one defined by constant evolution, where CISOs emerge as the linchpins of responsible information management. As organizations navigate the complexities of the regulatory and compliance landscape, understanding and embracing the essentials of data custodianship becomes paramount to fostering a culture of trust, accountability, and ethical data practices. The proactive role of CISOs, positioned as natural custodians, is central to fortifying organizations against evolving cyber threats and ensuring compliance with privacy regulations. By systematically integrating stringent measures aligned with prevailing industry standards, these CISOs exemplify the commitment required to uphold privacy and security imperatives. In the face of an ever-evolving regulatory panorama, such organizations demonstrate the resilience necessary to navigate complexities and ensure ethical data practices.


Unlocking Accountability: How Real-Time App Monitoring Empowers Engineering Teams

In the realm of software development—particularly with the advent of real-time application monitoring—employee retention, especially of developers, is paramount. Their deep understanding of the nuances of our applications and their ability to respond swiftly to the insights provided by real-time monitoring are invaluable. Maintaining a team of satisfied, engaged developers is crucial in this context. It’s not just about reducing turnover; it’s about fostering a culture where the engineers feel invested in the continuous improvement and success of our products. When developers are genuinely satisfied with their work and their environment, it reflects in the quality of their output. They become proactive in identifying and addressing issues, often before they escalate, thanks to the real-time data at their fingertips. The shift toward more dynamic monitoring practices has underscored the need for a supportive, collaborative environment. A culture where developers are encouraged to share insights and take initiative leads to a more responsive and adaptable team. This environment not only supports the technical aspects of our work but also enhances the overall morale and commitment of our developers.


Data Modernization: Turning an Ugly Duckling into a Swan

Regulatory and compliance requirements in healthcare, finance, transportation, and communications all require a cross-section of data for reporting that comes from many different systems. To tap into all that data and combine it in a single data repository for reporting purposes requires system integration. Such system integration requires that data be modernized across all systems into standard forms that can be passed from system to system and consolidated. To achieve this degree of integration and interoperability, data modernization tasks must be built into compliance and regulatory projects. It is up to the CIO and other IT leaders to explain to management and users why this data modernization work is needed. ... Customer relationship management (CRM) systems must integrate data from disparate systems owned by many different departments within the enterprise. System integration and data modernization are needed because the end goal of CRM is to deliver to any authorized employee anywhere a uniform, 360-degree view of each customer. 


Cloud-Computing in the Post-Serverless Era: Current Trends and Beyond

A primitive and a construct in programming have distinct meanings and roles. A primitive is a basic data type inherently part of a programming language. It embodies a basic value, such as an integer, float, boolean, or character, and does not comprise other types. Mirroring this concept, the cloud - just like a giant programming runtime - is evolving from infrastructure primitives like network load balancers, virtual machines, file storage, and databases to more refined and configurable cloud constructs. Like programming constructs, these cloud constructs orchestrate distributed application interactions and manage complex data flows. However, these constructs are not isolated cloud services; there isn’t a standalone "filtering as a service" or "event emitter as service." There are no "Constructs as a Service," but they are increasingly essential features of core cloud primitives such as gateways, data stores, message brokers, and function runtimes. This evolution reduces application code complexity and, in many cases, eliminates the need for custom functions. 


Three Best Practices for Optimizing the Benefits of Your Modern Data Stack

Today, businesses are embracing a democratized approach to data. The universal semantic layer enables everyone to become a data product creator, meaning that enterprises are distributing the ability to create data products to the business. As a result, the role of IT is transforming from that of controlling all the data to that of creating and managing platforms that allow business units to create their own data products and ask their own questions about that data. IT is no longer a bottleneck but has become a data enabler for all business units. The trend toward democratization has a profound impact on the way we work with data.  ... Another trend is the inclusivity of data and analytics roles. The modern data stack doesn't discriminate between data engineers, analytics engineers, or BI developers. It accommodates both code and no-code enthusiasts, making data accessible to everyone, regardless of technical background. This also means that anyone can access the data in their BI tool of choice, whether that be Power BI, Tableau, or Excel. The semantic layer is the key to truly enabling that business-friendly representation that works for every user, no matter their skill level or BI platform preference.


Without clear guidance, SEC’s new rule on incident reporting may be detrimental

The challenge with these new guidelines arises from the SEC’s directive that mandates registrants disclose any cybersecurity incident deemed materially significant, detailing, “… the nature, scope, and timing of the incident, as well as the material impact or reasonably likely material impact of the incident on the registrant, including its financial condition and results of operations.” This requirement leaves considerable interpretive leeway, and concrete definitions are likely to emerge only through legal precedent. Naturally, companies are hesitant to become test cases for these definitions. This ambiguity may prompt businesses to over-communicate with the SEC, ensuring exhaustive compliance with the immediate disclosure requirements. However, this approach risks diluting the significance of “material” information. Investors relying on a company’s 8-K filings for insights into the impact of a cyber incident might consequently overlook critical details amid the information overload. To counter this, the SEC needs to engage in proactive dialogues to clarify disclosure requirements, particularly regarding the frequency and extent of details needed. 



Quote for the day:

"Success consists of getting up just one more time than you fall." -- Oliver Goldsmith

Daily Tech Digest - January 21, 2024

What is RAG? More accurate and reliable LLMs

Retrieval-Augmented Generation (RAG) is an AI framework that significantly impacts the field of Natural Language Processing (NLP). It is designed to improve the accuracy and richness of content produced by language models. Here’s a synthesis of the key points regarding RAG from various sources: RAG is a system that retrieves facts from an external knowledge base to provide grounding for large language models (LLMs). This grounding ensures that the information generated by the LLMs is based on accurate and current data, which is particularly important given that LLMs can sometimes produce inconsistent outputs; The framework operates as a hybrid model, integrating both retrieval and generative models. This integration allows RAG to produce text that is not only contextually accurate but also rich in information. The capability of RAG to draw from extensive databases of information enables it to contribute contextually relevant and detailed content to the generative process; RAG addresses a limitation of foundational language models, which are generally trained offline on broad domain corpora and are not updated with new information post-training.


Redefining Quantum Bits: The Graphene Valley Breakthrough

Because quantum information is much more prone to being corrupted – and therefore become unsuitable for computational tasks – by the surrounding environment than its classical counterpart, researchers who study different qubit candidates must characterize their coherence properties: these tell them how well and for how long quantum information can survive in their qubit system. In most traditional quantum dots, electron spin decoherence can be caused by the spin-orbit interaction, which introduces an unwanted coupling between the electron spin and the vibrations of the host lattice, and the hyperfine interaction between the electron spin and the surrounding nuclear spins. In graphene as well as in other carbon-based materials, spin-orbit coupling and hyperfine interaction are both weak: this makes graphene quantum dots especially appealing for spin qubits. The results reported by Garreis, Tong, and co-authors add one more promising facet to the picture. ... The hexagonal symmetry observed in this so-called real space is also present in momentum space, where the vertices of the lattice don’t correspond to the spatial locations of carbon atoms but to values of momentum associated with the free electrons on the lattice.


5 Ways AI Can Make Your Human-To-Human Relationships More Effective

Understanding your audience is a major challenge for many business leaders. After all, if you knew what did or didn’t appeal to your audience, it would be much easier to speak to them in a meaningful, engaging way that sparks lasting connections. And AI can help here, too. This was illustrated to me during a recent conversation with James Webb, co-founder and CTO of Comb Insights, whose app uses proprietary AI to provide sentiment scores on comments on social media posts. "Using AI to quickly evaluate the overall sentiment of the comments on a post can give business leaders an immediate understanding of whether their content resonated with their audience,” he told me in an interview. “Seeing the ratio of positive to neutral or negative comments, and seeing the most common words that show up in the comments, can provide quick insights into why a post succeeded or failed. With this instant understanding of their audience, businesses can pivot in the type of social media content they produce so they can strengthen these important digital relationships.”


The missing link of the AI safety conversation

From a practical standpoint, the high cost of AI development means that companies are more likely to rely on a single model to build their product — but product outages or governance failures can then cause a ripple effect of impact. What happens if the model you’ve built your company on no longer exists or has been degraded? Thankfully, OpenAI continues to exist today, but consider how many companies would be out of luck if OpenAI lost its employees and could no longer maintain its stack. Another risk is relying heavily on systems that are randomly probabilistic. We are not used to this and the world we live in so far has been engineered and designed to function with a definitive answer. Even if OpenAI continues to thrive, their models are fluid in terms of output, and they constantly tweak them, which means the code you have written to support these and the results your customers are relying on can change without your knowledge or control. Centralization also creates safety issues. These companies are working in the best interest of themselves. If there is a safety or risk concern with a model, you have much less control over fixing that issue or less access to alternatives.


Intro to Digital Fingerprints

Digital fingerprinting is a technique used to identify users across different websites based on their unique device and browser characteristics. These characteristics - fingerprint parameters, can include various software, hardware (CPU, RAM, GPU, media devices - cameras, mics, speakers), location, time zone, IP, screen size/resolution, browser/OS languages, network, internet provider-related and other attributes. The combination of these parameters creates a unique identifier - fingerprint, that can be used to track a user's online activity. Fingerprints play a crucial role in online security, enabling services to identify and authenticate unique users. They also make it possible for users to trick such systems to stay anonymous online. However, if you can manipulate your fingerprints, you can run tens or hundreds or more different accounts to pretend that they are unique, authentic users. While this may sound cool, it has serious implications as it can make it possible to create an army of bots that can spread spam and fakes all over the internet, potentially resulting in fraudulent actions.


Looking at a data-driven financial future for India

In the intricate landscape of financial services, managing vast data, complex silos, and strict compliance demands a strategic solution. A hybrid data mesh is an innovative approach to financial operations that brings flexibility and coherence. This method combines a distributed architecture with an SSOT, ensuring accurate, secure, and compliant data handling. Data distribution across systems and functions facilitates quick insights while adhering to quality and privacy standards. The hybrid data mesh concept integrates the advantages of a distributed architecture tailored to domain-specific data with the SSOT, providing enhanced flexibility and scalability. This fusion ensures data coherence and accuracy while allowing domain independence, reinforcing security, and streamlining traceable and auditable compliance. Predictive models can be tailored to specific products or customer segments by harnessing AI and ML tools, enhancing decision-making in a dynamic market. This streamlined approach identifies growth opportunities and nurtures a culture of adaptability and innovation.


L&D trends that will define 2024

AI-assisted coding/software development employs AI to help write and review code. The potential of the technology to assist new developers in improving their code and saving time is valuable. The edtech sector, in particular, will employ AI to create customised learning experiences besides using tools that offer instant feedback on code. We could be looking at automating assessments for unbiased, error-free evaluations. Manually identifying personalised learning journeys for numerous individuals is time-consuming and extremely difficult. AI-assisted coding can help solve this operational challenge. Soon, we’ll give users quick, accurate responses and allow them to accelerate their learning journeys. ... Organisations will focus on data-driven, business-aligned learning initiatives for specific job-role competencies. This is to qualify L&D impact by easily tracking employee metrics such as job performance, efficiency, engagement, and employee satisfaction in new ways. When properly implemented, the accumulated data can raise confidence levels among higher-ups and lead to sustained investment in training practices. Organisations also analyse the information to identify areas of positive impact and focus on L&D in those regions for frequently better outcomes. 


New Guidance Urges US Water Sector to Boost Cyber Resilience

"Cyber threat actors are aware of - and deliberately target - single points of failure," the guidance states. "A compromise or failure of a water and wastewater sector organization could cause cascading impacts throughout the sector and other critical infrastructure sectors." The incident response guide aims to provide organizations with best practices for all four stages of the incident response life cycle - from preparation through detection, recovery and post-incident activities. The guidance says "the cyber incident reporting landscape is constantly evolving" and encourages water sector officials to review their reporting obligations and "consider engaging in additional voluntary reporting and/or information sharing" measures. Eric Goldstein, CISA's executive director for cybersecurity, said in a statement announcing the joint guidance that the U.S. water and wastewater sector "is under constant threat from malicious cyber actors." "In the new year, CISA will continue to focus on taking every action possible to support 'target-rich, cyber-poor' entities like WWS utilities by providing actionable resources and encouraging all organizations to report cyber incidents," he said.


Banking at the Precipice: Navigating the Fifth Industrial Revolution

As retail banking stands amid the Fourth Industrial Revolution’s digital transformation, leaders now must prepare for an imminent Fifth Industrial Revolution poised to profoundly reshape markets and experiences. Defined by extreme personalization, mass customization and precision augmentation, the emerging revolution’s exact disruptions remain somewhat undefined. Yet advancements in generative artificial intelligence, ambient interfaces and hyper-connectivity hint at consumer-in-command days ahead. ... Most of these Fifth Industrial Revolution financial applications seem unimaginable today. Imagine augmented live views layering physical surfaces like a retail store, billboard or car dealership with tailored offers based on persona identification and real-time transactional and behavioral data. Moving further, imagine a ‘digital twin agent’ seamlessly negotiating a personalized deal or pre-approved financing instantly. In this world, augmented and mixed reality interfaces, bridging physical and virtual worlds, will be able to move money experiences from transactions to value-based propositions based on where your eyes focus and engagements you have had in the past.


How generative AI is changing entrepreneurship

Entrepreneurs are expected to do a wide range of time-consuming tasks, from writing emails and answering phone calls to orchestrating product demonstrations and coding a website. “AI does all of those things well,” Mollick said. “It lets you focus more on what your top skill is, and it kind of handles everything else.” Generative AI can also serve as a guide. “A third of Americans have a business idea that they haven’t acted on because they don’t know what to do next,” Mollick said. “The AI can tell you what to do next, help you write the emails, [and] help you build the product.” Mollick noted that users should be aware of the benefits and limitations of the technology. “It’s kind of like an intern who wants to make you happy and therefore lies a lot and is kind of naive [and] never admits that they made a mistake,” he said. “Once you think about [AI] that way, you end up in much better shape.” Generative AI is a new general-purpose technology — one that comes around once in a generation and touches just about everything humans do, Mollick said, like electricity, computers, and the internet have. For entrepreneurs, generative AI can assist with researching ideas, coming up with logos and names, creating a website, and more, Mollick said.



Quote for the day:

"Leadership is not about titles, positions, or flow charts. It is about one life influencing another." -- John C. Maxwell

Daily Tech Digest - January 20, 2024

CISOs Struggle for C-Suite Status Even as Expectations Skyrocket

In many instances, CISOs who want clear risk guidance from their board don't get it. Barely more than one-third (36%) described their board as offering them clear enough insight into their organization's risk tolerance levels for them to act upon. "The evolution of the CISO role over the past few years has accelerated dramatically," says Nick Kakolowski, research director at IANS. With organizations digitizing more of their operations, CISOs are taking on more responsibilities and have become de facto owners of digital risk, he says. "[But] organizations haven't figured out how to support and empower them as the scope of the role grows." Concerns have been growing within the CISO community in recent years about the escalating expectations around the role, even as their ability to meet those expectations has remained largely unchanged. Incidents like one last October where the SEC charged SolarWinds CISO Tim Brown with fraud and internal control failures over the 2020 breach at the company, and where a judge sentenced former Uber CISO Joe Sullivan to three years of probation over a 2016 breach, have fueled those concerns. 


Three of four CISOs ready for job change

“Satisfaction has been rising consistently for the past few years, but last year, it dipped,” says IANS Research Director Nick Kakolowski. “Last year, the pressure on CISOs ratcheted up big time with the new SEC rules and CISOs being held personally liable for breaches. ... “The environment surrounding CISOs is extremely turbulent right now, and their individual exposure to lawsuits is at an all-time high. CISOs face a real danger of being indicted or sued for things outside of their control,” adds Patrick “Pat” Arvidson, chief strategy officer for Interpres, a maker of a threat-informed defense surface management platform. ... Another finding in the report is that CISOs aren’t getting the facetime with boards that they need. Eighty-five percent of CISOs in the survey indicated their board should offer clear guidance on their organization’s risk tolerance for the CISO to act on, but only 36% found that to be the case. “We are seeing some boards figuring this out and being effective there, but across the board, there’s either a lack of visibility at the board level—CISOs aren’t consistently reporting to the board—or CISOs and boards haven’t figured out how to speak each other’s language,” Kakolowski says.


How Accelerated Adoption of a Data Governance Framework Helped a Large Financial Services Organization Build a Snowflake Data Vault

The Domain Working Group meetings were instrumental in helping both business stakeholders and technology developers walk through examples and requirements for merging sometimes incomplete, inaccurate, and inconsistent data from 3 sources into a single complete, accurate, and consistent golden record. As business stakeholders started to understand the savings in time spent querying 3 data sources, reconciling and explaining differences between sources, and deciding which data is most trusted, and also started to see the benefits of having a single authoritative view of their domain data, enthusiasm for the Data Vault initiative increased. Embedding data governance practices and tools by creating a data governance workstream within a business or technology project is one of many approaches an organization can take to expand or accelerate engagement, adoption, and implementation of a data governance program. The success of this Data Vault project was partially attributed to the established data governance framework and team, but the biggest benefit was the adoption of data governance by dozens of previously unaware employees through exposure to the data governance program and witnessing real-life benefits of active end-to-end data governance made part of their everyday job responsibilities.


Putting a Number on Bad Data

Several quantifiable metrics can serve as a starting point for evaluating the cost of bad data, including the rate of occurrence or number of incidents per year, time to detection, and time to resolution. ... Number and frequency of incidents: While some companies may experience data incidents on a daily basis, others may go days – if not weeks – without one. The criticality of the incidents can vary from something “minor,” such as stale data linked to a dashboard that nobody has used in ages, to a data duplication problem causing the server to overcharge and ultimately go down. ... Mean time to resolution (MTTR): What happens once an incident is reported? MTTR is the average time spent between becoming aware of a data incident and resolving it. The resolution time is greatly influenced by the criticality of the incident and the complexity of the data platform, which is why we are considering the average for the purpose of this framework. ... Mean time to production (MTTP) is the average time it takes to ship new data products or, in other words, the average time to market for data products. This could be the time spent by an analyst “cleaning” the data for a data science model. 


Microservices Architecture: Navigating the Buzz

Despite the apparent advantages, there are various challenges that I think are important to highlight. Worth noting is that they are all avoidable when considered and planned around upfront. A common reason why teams end up sticking with a traditional monolithic approach includes the fact that microservices bring increased complexity. This complexity comes in the form of teams needing to understand how to design, build, and manage distributed systems. More specifically, not knowing how to implement a reliable communication protocol for microservices to be able to communicate is a recurring pain point that leads to decreased system performance, and in turn, has teams switching back to their monolithic system. Another challenge that arises from having an increased number of interactions comes in the form of system testing and debugging. Aside from these difficulties, another major concern when considering microservices includes that of security. Implementing robust authentication, authorization, and encryption across each and every service is crucial.


Attribute-based encryption could spell the end of data compromise

The history of ABE goes back to a ground-breaking 2005 paper titled “Fuzzy Identity-Based Encryption.” Fifteen years later, recognizing the paper’s significance, the International Association for Cryptologic Research (IACR) gave it a 2020 Test of Time Award. One of its co-authors, Dr. Brent Waters, later said the paper has had a three-fold impact. First, there has been the concept of ABE as its own application with distinctive new use cases, several of which are discussed below. Second, the cryptographic research community not only has spent years studying ABE, but also used ABE as a building block, leveraging it to obtain new results in work on other problems. Third, according to Dr. Waters, the work in ABE “inspired us to rethink encryption in even bigger and grander ways.” One such overflow has been functional encryption, which allows a user to learn only a function of a data set. For ABE, the end goal is fine-grained access to the data itself. On its own, that’s a revolution. An ABE scheme can provide the right user with a key to very specific data. Not to an entire file cabinet, so to speak, but to a single line item within a category of filed documents.


The Cashless Future: Convenience Versus Privacy and Freedom

While convenience reigns, privacy and governmental control remain crucial considerations. Financial inclusion must also be championed, ensuring everyone has access to secure and equitable payment methods. This transition demands careful navigation, balancing innovation with the principles of trust and individual freedom. The spectre of Central Bank Digital Currencies (CBDCs) further fuels the debate. Some fear a dystopian future controlled by governments with direct access to digital money. However, this ignores the historical evolution of ethics, regulations and frameworks. Laws like the Ten Commandments and the Magna Carta, enacted in our tribal and agrarian past, have evolved alongside society, forming the cornerstone of trust and control within our modern, interconnected world. We willingly relinquish information through regulatory KYC and AML practices in exchange for the security and transparency of banks. Similarly, could CBDCs provide a trusted digital foundation without succumbing to the anxieties of overreach? Perhaps the future holds not a revolution, but an evolution. A landscape where a foundational digital currency, overseen by central banks, coexists with diverse ecosystems like Disney coins or Amazon credits.


Harnessing the Power of Diverse Data with Fern Halper

Halper began with a quick overview of what constitutes diverse data. “Diverse data is pretty much just what it sounds like -- data in formats other than structured data,” she said. “This includes unstructured and semistructured data (for example, XML and JSON) and data from different sources (such as social media and IoT devices).” She explained that this diverse data is becoming more important as companies seek any way to compete better in their markets. “For example, a company can use the unstructured or semistructured data from their call center interactions to better predict when a customer might churn.” This diverse data can also be used to uncover hidden insights, make better predictions, and otherwise better respond to what’s happening on the ground, she added. “This reflects what we saw in the survey, which was that the primary driver for using diverse data, cited by 53% of respondents, was to better understand customers. This was followed by use cases related to operational efficiency, which were cited by 43%.” The conversation then turned to the subject of how organizations were managing all this data.


Deprecated npm packages that appear active present open-source risk

The problem is probably much worse because Aqua only checked direct dependencies, not transient ones as well — the dependencies of dependencies. The dependency chain for npm packages can go many levels deep and not accounting for this is a common reason why vulnerable code might make it into projects undetected. “​​This situation becomes critical when maintainers, instead of addressing security flaws with patches or CVE assignments, opt to deprecate affected packages,” the Aqua researchers said in their report. “What makes this particularly concerning is that, at times, these maintainers do not officially mark the package as deprecated on npm, leaving a security gap for users who may remain unaware of potential threats.” ... The npm repository package maintainers do have the option of marking packages as deprecated, which will appear as a warning to users visiting the page. They can also include a note for users with additional information such as alternatives. This can be considered as official deprecation. However, other signs can indicate that a project is dead even if it doesn’t have a big warning on it. 


How CEOs can mitigate compounding risks

Leaders should instruct their risk management functions to broaden the aperture on the risk scenarios they monitor to include compounding risks. For example, once risk managers have identified the top risks to the business, they often create an enterprise-level risk management map. Instead, the team should consider how and which individual risks could combine to create a new compounding risk, with particular focus on risks that may be minor individually but have high frequency (IT outages, for instance). Looking at the business through the lens of the customer rather than through product offerings can help risk managers see small but recurring friction points that could cause customers to leave. ... Many compounding risks stem from trends with long-term time horizons such as climate change, market or business model innovations, or changing consumer behaviors. These risks tend to build slowly until they hit the tipping point of becoming existential for the organization. A horizon planning approach can help management teams address risks that can emerge at various stages by looking at three horizons: first, maintaining and defending the core business; second, nurturing emerging businesses; and third, creating genuinely new businesses.



Quote for the day:

"Whenever you see a successful business, someone once made a courageous decision." -- Peter F. Drucker