Daily Tech Digest - May 02, 2024

3 Reasons Data Engineers Are the Unsung Heroes of GenAI

Ultimately, the success of GenAI depends on data quality. Without accurate, reliable data consistently made available to LLMs, even the most advanced models won’t produce useful outputs. Over the last five years, leading data engineers have adopted observability tooling — including automated monitoring and alerting, similar to DevOps observability software — to help improve data quality. Observability helps data teams monitor and proactively respond to incidents like failed Airflow jobs, broken APIs and misformatted third-party data that put data health at risk. And with end-to-end data lineage, teams gain visibility into upstream and downstream dependencies. Data engineers can provide transparency when observability tooling is applied across the modern AI stack, including vector databases. Lineage allows engineers to trace the source of the data as it’s converted to embeddings, then use that data to generate rich text that the LLM puts in front of the user. 

Why you’ll soon have a digital clone of your own

Far more interesting than an avatar that looks and acts like you is one that thinks and communicates like you — a virtual you with an AI brain for interacting with others on your behalf. Meta is working on tech called “Creator A.I.” that will enable real Instagram influencers to create fake digital AI versions of themselves to interact with fans through direct messages and comments. That initiative is a glimpse of the near future of business communication. We’ve been talking about “digital transformation” for a decade. But it’s only recently when that transformation involved digitizing ourselves. Businesses are now looking to embrace the concept of digital avatars for all the same reasons as other digital transformation initiatives: Higher productivity and lower costs. This process involves the cloning of existing people. Any day now, an industry will emerge where your face and body are scanned, your voice is recorded and your communications are fed into the system, so it knows how you use words. From that point, a virtual version of you can leave high-resolution video messages from a simple command you give to your AI glasses.

Euroviews: 'Quiet hiring' is silently revolutionising the workplace

Quiet hiring is gaining traction as a strategic alternative to traditional recruitment methods. Instead of constantly expanding the workforce through external hires, companies are increasingly focusing on upskilling and retraining the talent right under their noses, to meet evolving business needs. ... Quiet hiring also encourages a culture of continuous learning and development within the workforce, driving employees to proactively seek out and pursue professional growth opportunities. This is done by empowering existing talent, encouraging them to explore different roles and projects within the company, and investing in their development. By taking these steps, management teams can mitigate talent shortages, all while also driving employee loyalty, satisfaction, and retention. Tangible skills have never been more important. In fact, our latest research reveals that companies are integrating intentional learning with key skills like leadership, collaboration, and analytical thinking, aiming to both nurture professional growth and cultivate a supportive team culture.

Understanding emerging AI and data privacy regulations

The AI Act is the first comprehensive AI regulation adopted by EU lawmakers. It imposes obligations on both providers and deployers of AI systems, and specifically targets practices and systems that pose some risk to the health and safety or the fundamental rights of individuals, with the purpose of ensuring “a high level of protection of health, safety, fundamental rights (…), including democracy, the rule of law and environmental protection, against the harmful effects of AI systems.” ... The second category encompasses high-risk systems for which providers are subject to a whole set of requirements including risk management, data governance, record-keeping, transparency, human oversight, accuracy, robustness, and cybersecurity.  ... The third category encompasses certain AI systems that are subject to transparency requirements to make sure the public is in a position to recognize artificially generated or manipulated outputs. The fourth category encompasses other systems which are largely unregulated. Importantly, general-purpose AI models and systems are also in scope.

Indian SaaS Market: Good times ahead

There is a substantial demand for SaaS solutions from Small and Medium Businesses (SMBs) seeking to embrace digitisation and enhance their market reach and customer experience. SaaS companies must adopt a mobile-first strategy and tailor their go-to-market strategies to address the largely untapped SMB segment to increase adoption and ease of usage. In India, where personal connections play a significant role, human interaction will be crucial in selling the product. While horizontal SaaS solutions (e.g., CRM, HR, etc.) have traditionally dominated the market, there is a growing preference for vertical-specific solutions tailored to specific industries such as healthcare, education, finance, and logistics. Vertical SaaS companies stand to gain a loyal customer base by offering integrated solutions aligned with users’ systems and workflows. This trend is likely to gain further traction in 2024 as companies look for more specialised solutions. The emergence of Generative AI presents new opportunities and challenges for SaaS solutions providers. With a large pool of AI talent in India, SaaS companies can capitalise on this trend to develop innovative categories of services and revenue streams.

Maryland Legislature Passes State Privacy Bill With Robust Requirements And Broad Threshold For Application

MODPA prohibits the sale of sensitive data, which is a subset of personal data. A “sale of personal data” means the “exchange of personal data by a controller, a processor, or an affiliate of a controller or processor to a third party for monetary or other valuable consideration.” “Sensitive data” includes data revealing: racial or ethnic origin; religious beliefs; consumer health data; sex life; sexual orientation; status as transgender or nonbinary; national origin; and citizenship or immigration status. It also includes genetic and biometric data, personal data of a consumer who the controller knows or has reason to know is a child, and precise geolocation data. ... MODPA imposes guardrails with respect to the processing and sale of minors’ personal data. Controllers are prohibited from selling personal data of a consumer or using that data for purposes of targeted advertising if the controller knew or should have known that the consumer is under the age of 18. This prohibition is strict compared to other laws that require actual knowledge of consumers’ age or provide an opportunity for consumers to opt-in for the processing and sale of minors’ data.

Expectations vs. reality: A real-world check on generative AI

There are gen AI success stories in verticals like document engineering, where Docugami offers custom small language models that build a knowledge graph from a customer’s own complex documents, and can be used for both document generation and to extract data. And commercial insurance is a vertical Docugami CEO Jean Paoli says has been an early adopter, including statements of value, certificates of insurance, as well as policy documents with renewal dates, penalties, and liabilities. That’s critical information describing the risk of both individual customers and the entire portfolio, which has been difficult to manually extract and consolidate to use for generating new quotes, or representing the portfolio to reinsurers. “These are real scenarios that save you millions of dollars, not few hundred bucks,” Paoli says. Like everyone else, large Docugami customers created gen AI committees and started pilots in 2023, but many have already moved from discovery to implementation, starting production deployments at least six months ago and seeing real returns, chief business officer Alan Yates says. 

9 Steps Towards an Agile Architecture

The first step in breaking these limiting beliefs is to gain consensus that the system will be released in a series of increments. The essence of the Minimum Viable Product (MVP) approach is that each product increment will attempt to deliver at least one outcome to at least a subset of the users of the system. The organization will measure its success in achieving these outcomes and use that feedback to decide what it should work on next. Using an MVP/MVA approach changes the conversations that a development team and its stakeholders have. ... Learning to develop architecture incrementally takes practice, and no team is going to master it immediately. Most teams lean toward developing more architecture than they need, only to find that some of what they did was wrong, or at least unnecessary. They will probably struggle to balance trade-offs. They will also probably learn that some of the choices they made are wrong. All of this is good. They have to learn the right balance on their own, and they can only do this by making some wrong choices and seeing the impact.

Tracking Data Breaches: Targeting of Vulnerabilities Surges

To penetrate corporate networks, hackers most often gained unauthorized access to a victim's web applications, including cloud-based email or collaboration tools, Verizon said. The next most common ways that attackers broke in involved phishing emails, followed by exploiting vulnerabilities in web applications, stealing credentials for desktop sharing software, and using stolen VPN credentials. For the first time, Verizon's annual report counts software vulnerabilities - such as the widely exploited MOVEit bug - as a supply chain security metric. "We want to give organizations the ability to see what they can effect in their environment by choosing vendors who follow 'secure by design' practices," Widup said. The Clop campaign targeting MOVEit was notable in part because it seemed to run for only two or three days. While vendor Progress Software quickly issued a patch, the latest count from security firm Emsisoft is that Clop's blitzkrieg tactics directly or indirectly affected 2,770 organizations and exposed data pertaining to 95 million individuals.

How to Manage a Rapidly Growing IT Team

The key to keeping pace with rapid growth lies in structuring an IT team that embraces new technology and training to stay current, says Diane Rafferty, managing director, national technology group, at employment placement firm Atrium, via email. "If you don't offer training on the most up-to-date technologies, your team will have to do more with less." Leadership must clearly define expectations, yet leave room for movement, Velco says. "Autonomy inspires ownership of roles and collaborative problem-solving beyond defined bounds," he explains. Structures alone don't scale, Velco notes, but processes do. "Implementing agile frameworks like Scrum or Kanban encourages frequent reassessment and adjustment." When it comes to preparing for team growth, Daniele Gemignani, CTO at no-code business process automation solution provider Pipefy, is a strong believer in mentorship and leadership development programs. "For teams with a mix of seniority levels, pairing junior members with more experienced mentors can accelerate their development and readiness to take on more complex tasks or leadership roles," she says

Quote for the day:

"Don’t be distracted by criticism. Remember — the only taste of success some people get is to take a bite out of you." -- Zig Ziglar

No comments:

Post a Comment