Daily Tech Digest - March 23, 2021

How Synthetic Data Levels The Playing Field

Synthetic data can be defined as data not collected from real-world events. Today, specific algorithms are available to generate realistic synthetic data used as a training dataset. Deep Generative Networks/Models can learn the distribution of training data to generate new data points with some variations. While it is not always possible to learn the models’ exact distribution, algorithms can come close. ... The big players already have a stronghold on data and have created monopolies or ‘data-opolies’. Synthetic data generation models can address this power imbalance. Secondly, the rising number of cyberattacks, especially after the pandemic, has raised privacy and security concerns. The situation is especially worrying when huge amounts of data are stored in one place. By creating synthetic data, organisations can mitigate this risk. Thirdly, whenever datasets are created, they reflect real-world biases, resulting in the over-representation or under-representation of certain sections of society. The machine learning algorithms based on such datasets amplify such biases resulting in further discrimination. Synthetic data generation can fill in the holes and help in creating unbiased datasets.


Researchers Discover Two Dozen Malicious Chrome Extensions

While malicious extensions are an issue with all browsers, it's especially significant with Chrome because of how widely used the browser is, Maor says. It's hard to say what proportion of the overall Chrome extensions currently available are malicious. It's important to note that just a relatively small number of malicious extensions are needed to infect millions of Internet users, he says. One case in point was Awake Security's discovery last June of over 100 malicious Google Chrome extensions that were being used as part of a massive global campaign to steal credentials, take screenshots, and carry out other malicious activity. Awake Security estimated that there were at least 32 million downloads of the malicious extensions. In February 2020, Google removed some 500 problematic Chrome extensions from its official Chrome Web Store after being tipped off to the problem by security researchers. Some 1.7 million users were believed affected in that incident. In a soon-to-be-released report, Cato says it analyzed five days of network data collected from customer networks to see if it could identify evidence of extensions communicating with command-and-control servers. 


What IT Leaders Need to Know About Open Source Software

Despite conventional wisdom, open-source solutions are, by their nature, neither more nor less secure than proprietary third-party solutions. Instead, a combination of factors, such as license selection, developer best practices and project management rigor, establish a unique risk profile for each OSS solution.The core risks related to open source include: Technical risks, including general quality of service defects and security vulnerabilities; Legal risks, including factors related to OSS license compliance as well as potential intellectual property infringements; Security risks, which begin with the nature of OSS acquisition costs. The total cost of acquisition for open source is virtually zero, as open-source adopters are never compelled to pay for the privilege of using it. Unfortunately, one critical side effect of this low burden of acquisition is that many open-source assets are either undermanaged or altogether unmanaged once established in an IT portfolio. This undermanagement can easily expose both quality and security risks because these assets are not patched and updated as frequently as they should be. Finally, vendor lock-in can still be a risk factor, given the trend among vendors to add proprietary extensions on top of an open-source foundation (open core).


Applying Stoicism in Testing

To consider and look for the unknown information about a system, you need to have justice. In Stoicism, this stands for “showing kindness and generosity in our relationships with others”. And because you don’t know everything, you need other people to help you out. Gathering information is also about creativity, so you have to gather inspiration from past experience, and with your colleagues must be able to connect dots that weren’t connected before. Once I even stated, “The knowledge (and information) you gather as a tester about the software can be an interesting input for new software products and innovations”. But as a Stoic, stay humble ;-). After gathering all the information you need, you should use your wisdom (“based on reasoning and judgment”) to come to conclusions so that you can answer the question “Is this software ready to be used?” Although our customers are the best testers, we as testers are in the position that we are (or at least should be) able to answer the question at every step in the software development: if the software is going to production, what can happen? The information you put on the table for your stakeholder should be based on facts. 


Data Analyst vs. Data Scientist

The typical data analyst role is consulting-centric, as can be seen from the Indeed job spec example. What they are preoccupied with for the most part is wrangling data from Excel spreadsheets and SQL databases, extracting insightful conclusions via retrospective analyses, A/B tests, and generally providing evidence-based business advice. The last point illustrates why reporting routines with visualisation tools such as Tableau are as pivotal as pivoted tables. Data modelling on the other hand is often limited to basic supervised learning or its stats equivalent: regression analysis. ... To be fair, data scientists are for that reason expected to be more than analytical wizards. They are supposed to be builders who employ advanced programming to create pipelines that predict and recommend in production environments with near perfect accuracy. Compared with analysts, who’re like investigative reporters, they are a lot more product development than consulting oriented. Although it’s also required of a data scientist to provide data-led commercial advice. Some say the title was coined to manifest that the role was a confluence of three fields: maths/statistics, computer science and domain expertise.


Tech projects for IT leaders: How to build a home lab

If you're like most technology leaders, the closest you get to the actual technology you select and manage is creating PowerPoint decks that tell others about the performance, maintenance and updating of that technology. There's nothing fundamentally wrong with this of course; you can be a fantastic leader of a construction firm without having swung a hammer, or a cunning military strategist who has never rucked over a hill or fired a weapon. However, hands-on time with the fundamental building blocks of your domain can make you a better leader, just as the architect who spends time in the field and understands the materials and building process makes him or her more effective at creating better structures. ... Think of a home lab as the technology equivalent of the scientist's laboratory. It's a place where you can experiment with new technologies, attempt to interconnect various services in novel ways and quickly clean things up when you're done. While you might be picturing a huge rack of howling servers, fortunately for us you can now create the equivalent of a small data center on a single piece of physical equipment.


IOTA still wants to build a better blockchain, and get it right this time

What went wrong then, and how is IOTA going to fix it -- besides introducing a new wallet? Schiener focused on some key technical decisions that proved wrong, and are being retracted. IOTA wanted to be quantum-proof, and that's why it used a "special cryptography," as Schiener put it. IOTA's cryptography only allowed, for example, to utilize an address once. Reusing an address could lead to a loss of funds. Another questionable decision was choosing to use ternary, rather than binary encoding for data. That was because, according to Schiener, the hypothesis of the future was that ternary is a much better and more efficient way to encode data. The problem is, as he went on to add, that this also needs ternary hardware to work. There are more, having to do with the way the ledger is created. It's still a DAG, but it has different algorithms. Schiener said that over the last one and a half years, IOTA has been reinvented and rewritten from the ground up. This new phase of the project is Chrysalis, which is this new network upgrade. With Chrysalis, IOTA is also moving toward what it calls Coordicide.


Browser powered scanning in Burp Suite

One of the main guiding principles behind Burp’s new crawler is that we should always attempt to navigate around the web applications behaving as much as possible like a human user.. This means only clicking on links that can actually be seen in the DOM and strictly following the navigational paths around the application (not randomly jumping from page to page).Before we had browser-powered scanning, the crawler (and the old spider) essentially pretended to be a web browser. It was responsible for constructing the requests that are sent to the server, parsing the HTML in the responses, and looking for new links that could be followed in the raw response we observed being sent to the client. This model worked well in the Web 1.0 world, where generally the DOM would be fully constructed on the server before being sent over the wire to the browser. You could be fairly certain that the DOM observed in an HTTP response was almost exactly as it would be displayed in the browser. As such, it was relatively easy to observe all the possible new links that could be followed from there.Things start to break down with this approach in the modern world. 


Fintech disruption of the banking industry: innovation vs tradition?

The first was the rise of the internet. Constantly improving speeds and widespread access meant hundreds of millions of consumers were suddenly able to access digital services. The second was the rise of the smartphone. This hardware transformed consumer behaviour beyond recognition. Apps and other software products providing significant upfront value made smartphones indispensable — just think of Shopify, Google Maps and Uber. The third driver which paved the way for fintech providers’ success was the financial crisis in 2008. Not only did this bring the traditional banking system to the brink of collapse, but consumers were far less trusting of the big banks thereafter. The new breed of financial services providers was not tied down by legacy infrastructures and, with smaller teams and flexible IT infrastructures, they were more agile. And this allowed them to easily circumnavigate the new regulatory and compliance requirements that were introduced in the wake of the financial downturn. Fintech providers sought to solve problems the banks could not. Or at least to do what the banks do, but better. 


Top 3 Cybersecurity Lessons Learned From the Pandemic

As the world began relying on these new digital capabilities, new risks and challenges were introduced. Organizations that were well-equipped to extend visibility and control to this new way of working found themselves in a far better situation than those that were scrambling to completely reengineer their security capabilities. The ones that had built an empowered and proactive security team, backed by robust processes and supported by effective technology, were able to adapt and overcome. Organizations that were locked into a rigid operational model, overly reliant on vendor platforms or lacking a defined set of processes to support their new reality, struggled to keep pace. ... Since the pandemic began, we have seen an increased emphasis and shift toward zero trust and security access service edge (SASE) principles. With strong identity and access management capabilities, insights into services and APIs, and visibility into remote endpoint devices, security teams can put themselves in position for rapid and effective responses — even within this unique virtual setting. Access to sensitive and confidential data is the new perimeter for an organization's cybersecurity posture.



Quote for the day:

"A tough hide with a tender heart is a goal that all leaders must have." -- Wayde Goodall

Daily Tech Digest - March 22, 2021

Bitcoin’s Greatest Feature Is Also Its Existential Threat

The botnet’s designers are using this idea to create an unblockable means of coordination, but the implications are much greater. Imagine someone using this idea to evade government censorship. Most Bitcoin mining happens in China. What if someone added a bunch of Chinese-censored Falun Gong texts to the blockchain? What if someone added a type of political speech that Singapore routinely censors? Or cartoons that Disney holds the copyright to? In Bitcoin’s and most other public blockchains there are no central, trusted authorities. Anyone in the world can perform transactions or become a miner. Everyone is equal to the extent that they have the hardware and electricity to perform cryptographic computations. This openness is also a vulnerability, one that opens the door to asymmetric threats and small-time malicious actors. Anyone can put information in the one and only Bitcoin blockchain. Again, that’s how the system works. Over the last three decades, the world has witnessed the power of open networks: blockchains, social media, the very web itself. What makes them so powerful is that their value is related not just to the number of users, but the number of potential links between users.


India’s Quest Towards Quantum Supremacy

The digital partnership between the Indian Institute of Science Education and Research (IISER) at Pune and Finland’s Aalto University has created a high probability of getting its first quantum computer. ... Talking about the partnership, Neeta Bhushan, the joint secretary (Central Europe), external affairs ministry, stated that the idea of jointly developing a quantum computer with the use of AI and 5G technology is an important area of collaboration for both countries. Considering that Nokia and other Finnish companies are leading the world in mobile technology growth, this digital collaboration will witness the two countries collaborating on quantum technologies and computing. Hence, the partnership will have the leverage to deploy the latest technologies available with both countries. ... The partnership can lead us towards a new ecosystem altogether, and many things can be expected out of the same. The post-COVID changes in global power-sharing and the recent technological developments to handle the crisis have brought India to the centre stage. Consequently, quantum encryption is one of the basic applications derived from this collaboration.


Remote working still isn't perfect. These are the things that need fixing

A new report from O2 Business explores these insights in greater depth. The UK mobile operator surveyed 2,099 workers who had previously been office-based to understand how their needs and expectations of work had changed. It found that the majority of employees welcomed the notion of splitting their time between the office and home-working going forward, but also called for a closer alignment of operations, IT and HR in order to support individual work choices and maximize workplace productivity. Generally, employees are satisfied with their organization's response to the pandemic, O2 found: 69% of workers felt that their employers had supported them during the pandemic, with just 11% disagreeing with this statement. But less than two-thirds (65%) of employees felt confident that their organization was prepared for the future world of work. O2 said this indicated some businesses would struggle to adapt to the more flexible working arrangements that many are planning to adopt post-pandemic. The mad scramble to remote working has been one of the most trying aspects for businesses over the past year.


Fight microservices complexity with low-code development

A low-code platform takes care of nearly everything that conventionally is coded for an application. Most of the low-level programming and integration work is taken care of via tool configurations, which saves developers a lot of time and headaches. However, think carefully about where you apply low-code in a microservices architecture. As long as the app is simple, clean and doesn't require many integration points, low-code development might be the right alternative to more manual and complex microservices projects. Low-code builds are an easy choice for applications that don't need to integrate with other databases or only rely on a series of small tables. Short-lived conference apps or marketing promotions that run with user ID information are good examples of this. However, a low-code approach does not replace large-scale microservices development. Once you need to share information between applications in real time, the tools and programming techniques involved become much more sophisticated. While the low-code approach helps developers steer clear of over-engineering apps that don't need it, low-code likely won't provide the database integration, messaging or customization capabilities needed for an enterprise-level microservices architecture.


Edge Computing Growth Drives New Cybersecurity Concerns

Effectively protecting the edge means understanding how cybersecurity protection schemas work in an enterprise that uses not only edge computing, but also the cloud and traditional resources. Most enterprises are clearly focused on data security and application security, and are using tools such as web application firewalls (WAF), runtime application self-protection (RASP), data exfiltration protection and, of course, endpoint protection. Since the edge has the ability to “touch” data and applications, as well as use identity to connect and determine entitlements, a great deal of potentially sensitive information passes through the edge. Much, if not all of that traffic moves through a content delivery network (CDN), where hosts provide the connectivity and, hopefully, wrap encryption around that traffic to protect it from interception. However, intrusion and data exfiltration still happens. “Digital transformation is driving more and more applications to the edge, and with that movement, businesses are losing visibility into what is actually happening on the network, especially where edge operation occurs,” Hathaway said. “Gaining visibility allows cybersecurity professionals to get a better understanding of what is actually happening at the edge,” he said.


Move Your Automation Efforts From Pilot To Reality

Talent is another crucial part of the equation that not enough customers take into account. I’ve worked with many customers that don’t have dedicated automation centers of excellence, or specific in-house expertise to tackle automation the right way. An enterprise with multiple technologies in place must ensure that those technologies are communicating with each other. By bringing together technical experts, your processes can be better visualized and monitored end-to-end across the organization, leading to a higher chance of success. The complexity and effort involved in this kind of endeavour can be off-putting, but it’s worth the reward. Nor is it truly as complicated as it sounds — execution management systems, for example, already bring together technologies like process mining, automation and AI into a seamless, intelligent execution layer. Bring in or train the right people to champion it, and you’ve got a headstart on the next step of the journey. So while many companies haven’t been able to bring the full promise of automation to bear at scale just yet, that promise is getting closer to becoming a reality every day.


HowTo: Optimize Certificate Management to Identify and Control Risk

End-to-end certificate management gives businesses complete visibility and lifecycle control over any certificate in their environment, helping them reduce risk and control operational costs. Even in the most complex enterprise environments, certificate automation offers speed, flexibility and scale. Full visibility over all digital certificates and keys means that even the largest enterprises can have a centralized view of digital identities and security processes. Security leaders can then access expiration dates and maintain cryptographic strength while avoiding the time-consuming, demanding, and risky task of manually discovering, supervising, and renewing certificates. As organizations continue to grow and evolve, so does the range of certificates deployed and the set of people deploying them, which increases the potential for certificates to be installed in your environment that are out of sight of IT security teams and left unmanaged. To avoid being blindsided by these “rogue” certificates, enterprises are turning toward automated universal discovery.


On the Road to Good Cloud Security: Are We There Yet?

The research also uncovered a disconnect that raises the question: Is that confidence misplaced? When asked to rate the level of visibility the security team had into their organization's use of specific cloud service types, including software-as-a-service (SaaS), platform-as-a-service (PaaS), and infrastructure-as-a-service (IaaS), that same level of confidence faltered. For example, when asked to rate the security team's level of visibility into their organization's SaaS usage on a five-point scale, with 1 being the highest level, only 18% gave it a 1 and 27% gave it a 2. Visibility into PaaS and IaaS was rated as only slightly better. At the same time, respondents' knowledge of the shared responsibility model was found to be lacking. When asked to indicate whether the customer or cloud provider was responsible for securing a list of seven different elements that make up an IaaS account, around half of respondents gave the wrong answer. Specifically, 63% erroneously indicated that the cloud provider was responsible for securing virtual network connections, 55% erroneously indicated that the cloud provider was responsible for securing applications, and 50% got it wrong when they said the cloud provider was responsible for securing users who were accessing cloud data and applications.


5 AI-for-Industry Myths Debunked

Up until, and during, the AI hype in the nineties, artificial intelligence was a scientific discipline that almost exclusively dealt with data and algorithms. Over the past decades however, the field has matured, and AI has become an integral part of automated decisioning systems that are at the heart of what we do as individuals and organizations. Consequently, a large portion of AI research, development, and implementation encompasses people and processes. I remember having a business conversation with a large energy provider in which we were talking about automated systems and data-driven methods that, driven by customer data and smart meters, could enhance their customers’ experience. One hour into the meeting, they suddenly asked: “This all looks very promising, but shouldn’t we also do something with AI?” ... If you have the combined luck and skills, you can probably cook a decent meal with ingredients that come from a randomly filled refrigerator. The real question, however, is: “What do you want to achieve?” In the example of the refrigerator, it might occasionally be an effective solution if you need to quickly fill stomachs and don’t have time to go shopping. 


Cloudflare wants to be your corporate network backbone

With Magic WAN, Cloudflare aims to simplify that. Cloudflare's global Anycast network is already built for high performance and availability to serve its core CDN business. The company has data centers in more than 200 cities across over 100 countries with local peering at internet exchange points. Regardless of where branch offices or employees are located, chances are high they'll always connect to a server close to them and then the traffic will be routed through Cloudflare's private network efficiently benefiting from its performance optimizations, smart routing and security. With Magic WAN organizations only need to set up Anycast GRE tunnels from their offices or datacenters to Cloudflare and they can then define their private networks and routing rules in a central dashboard. Cloudflare's existing Argo Tunnel, Network Interconnect and soon IPsec can also be used to connect datacenters and VPCs to its network, while roaming employees will connect using Cloudflare WARP, a secure tunneling solution that's built around the highly performant Wireguard VPN protocol. This also solves the scalability and performance issues that organizations have faced with traditional VPN gateways and concentrators when they were suddenly faced with a large remote workforce due to the pandemic.



Quote for the day:

"A true dreamer is one who knows how to navigate in the dark" -- John Paul Warren

Daily Tech Digest - March 21, 2021

CompTIA creates blockchain industry group to promote new use cases

"With growing interest in the deployment of blockchain technology in business applications, the time is right for us to expand our offerings related to this emerging technology," said Nancy Hammervik, executive vice president of industry relations and CEO of the CompTIA Tech Careers Academy. Members of the group will gain access to resources and forums where the community will be able to promote different use cases, share ideas, hold in-depth discussions and make connections to network with peers. CompTIA already has the Blockchain Advisory Council and the technology industry group will build on the organization's expertise in the space. The council is currently made up of industry leaders and innovators while also finding ways for "technology companies and their customers can leverage blockchain technology in their businesses," according to a statement from the organization. "The Blockchain Technology Interest Group is designed for the curious and the experienced to share ideas and discussions related to blockchain technology," said Kathleen Martin, senior manager for CompTIA's member communities and technology interest groups.


The State of Serverless Computing 2021

Organizations going serverless with their backend services are billed by their serverless vendor based on their compute consumption and do not have to reserve and pay for a fixed amount of bandwidth or number of servers, as the service is auto-scaling with the incoming demand. In the initial days of web development, developers had to own or rent physical hardware to run and test their application code. Running production applications was another nightmare because now they had to keep those servers running during the lifecycle of the application. Cloud computing and virtualization brought much-needed relief to web developers. Now they could rent virtual servers from a cloud vendor according to their needs. The problem was they still had to over-purchase to keep up with traffic spikes or their application would break down. Except, much of the server space they were paying for was going to waste. Cloud vendors introduced auto-scaling compute models to mitigate the problem. However, auto-scaling in response to an unsolicited spike in traffic (such as a DDoS Attack) could turn quite expensive.


GDPR: Transferring Data Outside The EU

If you want to transfer or store data in other countries, you must ensure that there is adequate protection of the data. You’ll need to complete a risk assessment about the nature of the personal data being transferred and the adequacies of the other organisation’s controls and protections offered by their local legal system. For most circumstances contractual protections will be sufficient. The UK Information Commissioner’s Office has some guidance as to how to approach this. The EU has helpfully provided standard contractual clauses in the form of model contracts (for both Controllers and Processors) but these have not been fully updated for GDPR at the time of writing. For large multinationals there is the opportunity to use ‘binding corporate rules’ for transfers within multinational organisations and a certification mechanism. Lawyers Allen and Overy have produced a useful guide to binding corporate rules. There are also some exemptions and exceptions such as specific consent, a contract, substantial public interest, law enforcement, the exchange of airline passenger data etc. The UK Information Commissioner’s Office has some useful guidelines.


Blockchain Tech: Path Towards Scalable Businesses

If the past of the Indian IT sector is a great story of resilience and growth, the future cannot be less bright though the sectors of growth would have to be identified, capacity-building will have to be sustained and regulatory framework would have to be supportive. That is where blockchain tech becomes a great unfolding story for India. A country that became the back-office of global IT giants has a new tale to unwind. Millions of quality but affordable software developers would need a nudge to churn out new solutions using shared ledger architecture. Before we jump into this new wealth era, a quick introduction to the blockchain is important. In a simple sense, blockchains are ledgers but with key properties of decentralization, programmability and antifragility. That transforms ledger databases into protocols of new scalable business models. That is why blockchain is called institutional technology that can redesign a set of fragmented stakeholders to unified business partners. This new model can be merged with AI (artificial intelligence) and IoT (internet of things) to create mind-blowing businesses. India stands at the edge of this era ideally prepared to embrace it.


Distributed Ledger Technology

When it comes specifically to MSME financing, DLT brings a holy trinity of value: trust, transparency and traceability. These benefits can make it easier for MSMEs to build a digital credit history and for banks to assess MSME creditworthiness. Some DLT projects, such as the European-based we.trade, are focused specifically on serving MSME firms, amplifying the benefits of DLT for smaller firms that will be able to access solutions tailored more specifically to their needs. The increased transparency provided by DLT can also make it easier for Tier 2 suppliers and lower which are often small businesses, to access finance. Common supply chain finance solutions are usually only available to established Tier 1 suppliers, which are able to convince their big corporate buyers of their trustworthiness. By enhancing visibility into deeper-tier suppliers, DLT can make their access to finance easier. Various companies like Linklogis and Skuchain are leveraging DLT to this effect. Another potential benefit of DLT for MSMEs is its ability to allow traditional processes or sources of finance to be bypassed.


Uncertainty around India's crypto policy is making blockchain firms anxious

Optimism started to rebuild, and surging Bitcoin prices began to lure millennials. When it comes to transferring Bitcoin and other digital assets, India is of late providing more volume than China on popular peer-to-peer platforms. The risk that India would hit back with a new law to make criminals out of crypto professionals and investors was always present. So practitioners tried to educate policymakers, appealing for sensible regulation starting with definitions for what is a utility token, which digital asset is to be viewed as a security, and which is to be treated as a currency. The trouble is with bureaucrats. They say they want blockchain, but not cryptocurrencies. It’s as silly as wanting airports with duty-free shops but no flights. From the Reuters story, it doesn’t appear that the final regulation will be much different from what a draft bill had recommended in 2019. A government panel report, which had provided the backdrop for the draft legislation, said that authorities would be fine with distributed ledger technologies for delivery of any services, or “for creating value,” without involving cryptocurrencies “for making or receiving payment.”


Technical debt is costing banks innovation and agility. Can cloud help?

The reason banks often do nothing to address it is they convince themselves that they have to rewrite a 40-year-old platform, and then they’re looking at a hundred-million-dollar price tag. So they kick the can down the road. But the longer the debt persists, the greater the consequences. Banks get less agile, less able to innovate code. They become more vulnerable to cybersecurity breaches. Addressing those breaches can drain the development budget, so that all the firm can afford to do is fix emergencies rather than deliver new capabilities, entrenching it in a vicious cycle. Right! That’s actually one of the biggest drains of technical debt: not money but people. Technical debt is a talent issue, too. The more antiquated code that a company struggles to maintain, the more it inhibits the modern tooling and services that developers want to use to build applications. A good developer can work anywhere. They won’t choose to work in a place where the environment is dated. They can go work in companies that are ultra modern with an engaging culture. Banks tend to have the mindset of, “It’s going to cost a boatload of money that I’ll never get approved if I have to replace or rewrite thousands of applications.” That’s where we come in and say: You don’t have to rewrite all of this.


What is Azure Blockchain Service?

Azure Blockchain Service is designed to support multiple ledger protocols. Currently, it provides support for the Ethereum Quorum ledger using the Istanbul Byzantine Fault Tolerance (IBFT) consensus mechanism. These capabilities require almost no administration and all are provided at no additional cost. You can focus on app development and business logic rather than allocating time and resources to managing virtual machines and infrastructure. In addition, you can continue to develop your application with the open-source tools and platform of your choice to deliver your solutions without having to learn new skills. Deploying Azure Blockchain Service is done through the Azure portal, Azure CLI, or through Visual Studio code using the Azure Blockchain extension. Deployment is simplified, including provisioning both transaction and validator nodes, Azure Virtual Networks for security isolation as well as service-managed storage. In addition, when deploying a new blockchain member, users also create, or join, a consortium. Consortiums enable multiple parties in different Azure subscriptions to be able to securely communicate with one another on a shared blockchain.


Make Agile a stepping stone toward future fit adaptability

For many tech leaders, being adaptive is the same as business agility. However, the execution engine for adaptability is your software development and delivery capabilities. Your business can't be adaptive if you don't have great software delivery capabilities. That's why "Agile" has become foundational to being adaptive and for achieving business agility. Forrester research shows that over 72% of enterprise development leaders executed Agile capabilities or were planning to be more Agile in 2019–2020. The Agile Manifesto, published almost 20 years ago, is still the cornerstone for any truly Agile organization. The first point is this: Being truly Agile goes beyond the Webster dictionary description of the adjective. Webster defines agile as "having a quick resourceful and adaptable character" or "marked by ready ability to move with quick easy grace." Agile as defined by the Agile Manifesto carries a broader meaning: the core values and the 12 principles. The Manifesto's definition of Agile carries the meaning of the Webster dictionary definition as a mandatory condition, but not as a sufficient one. So, what does it mean for enterprises in 2021 to be truly Agile and therefore adaptive? Start by going beyond just adopting agile (lowercase), and develop your cultural DNA and organizational strategy around the values and principles established in the manifesto.


New Malware Hidden in Apple IDE Targets macOS Developers

The malware is executed when a developer using the Trojanized version of the TabBarInteraction Xcode project launches what is known as the build target in Xcode. The XcodeSpy malware contacts the attacker's command-and-control (C2) server and drops the EggShell backdoor on the development machine, SentinelOne said in a report this week. "An Xcode project is a repository for all the files, resources, and information required to build one or more software products," Stokes says. "A project contains all the elements used to build a product and maintain the relationships between those elements." Injecting malware into an Xcode project gives attackers a way to target developers and potentially backdoor the developer's apps and the customers of those apps, he says. With XcodeSpy itself, though, the attackers appear to be only directly targeting the developers themselves, according to SentinelOne. The security vendor said a sample of XcodeSpy was found on a US-based victim's Mac in late 2020. The company's report did not disclose the identity of the victim but described the organization as a frequent target of North Korean advanced persistent threat actors.



Quote for the day:

"Leadership has a harder job to do than just choose sides. It must bring sides together." -- Jesse Jackson

Daily Tech Digest - March 20, 2021

‘Black Mirror’ or better? The role of AI in the future of learning and development

An AI-assisted learning-development tool can search a variety of sources internally or externally to find content that is relevant to a particular learning or performance outcome. Digital marketers and online publishers have been using AI to generate content for simple stories for years now. Odds are you have read an online article or blog post created by a bot and didn’t even realize it. In the learning space, there are tools such as Emplay and IBM’s Watson that can support this. For example, let’s say a designer wants to create a quick microlearning on how a vacuum pump works. The designer could engage an AI bot to crawl internal or external networks for potential resources — including videos and images. The AI agent then analyzes them, aligning pieces to specific learning outcomes, prioritizing resources for relevance and tagging them by modality. Ultimately, this would free up the designer to focus more on learner-centric design and delivery. ... As you can see, there are many potential benefits to the adoption of AI in the learning space. However, before we invest in AI, it is important to first explore the risks and practical issues of adopting AI across the enterprise.


Facebook is making a bracelet that lets you control computers with your brain

The wristband, which looks like a clunky iPod on a strap, uses sensors to detect movements you intend to make. It uses electromyography (EMG) to interpret electrical activity from motor nerves as they send information from the brain to the hand. The company says the device, as yet unnamed, would let you navigate augmented-reality menus by just thinking about moving your finger to scroll. A quick refresher on augmented reality: It overlays information on your view of the real world, whether it’s data, maps, or other images. The most successful experiment in augmented reality was Pokémon Go, which took the world by storm in 2016 as players crisscrossed neighborhoods in search of elusive Pokémon characters. That initial promise has faded over the intervening years, however, as companies have struggled to translate the technology into something appealing, light, and usable. Google Glass and Snap Spectacles bombed, for example: people simply did not want to use them. Facebook thinks its wristband is more user friendly. Too soon to tell. The product is still in research and development at the company’s internal Facebook Reality Labs, and I didn’t get to have a go.


Uncertainty And Innovation At Speed

As uncertainty continues to rise and the unexpected becomes more common, organizations may not always have the luxury to conduct extensive analysis before acting. Indeed, high uncertainty and rapid change tend to reduce the relevance of the data that companies may have traditionally used for planning. They may need to place bets on multiple possible futures. Above all, they will need capacity for rapid innovation—every day, not just in a crisis. Companies executed the rapid innovations described above by repurposing existing knowledge, resources and technology. A recent article suggests that organizations in all industries may be able to use repurposing to achieve ultrafast innovation to develop new solutions to our current and future challenges. Some innovation thinkers take inspiration from venture capital. Venture capital firms tend to tie funding to the achievement of milestones that reduce investment risk, such as proving technical feasibility or product-market fit. This approach instills a sense of urgency in startup companies: Their very survival may depend on achieving a funding milestone. A crisis such as the COVID-19 pandemic can produce a sense of urgency in even large organizations. But banking on effective innovation in response to a crisis is not a robust strategy.


What is data governance? A best practices framework for managing data assets

When establishing a strategy, each of the above facets of data collection, management, archiving, and use should be considered. The Business Application Research Center (BARC) warns it is not a “big bang initiative.” As a highly complex, ongoing program, data governance runs the risk of participants losing trust and interest over time. To counter that, BARC recommends starting with a manageable or application-specific prototype project and then expanding across the company based on lessons learned. ... Most companies already have some form of governance for individual applications, business units, or functions, even if the processes and responsibilities are informal. As a practice, it is about establishing systematic, formal control over these processes and responsibilities. Doing so can help companies remain responsive, especially as they grow to a size in which it is no longer efficient for individuals to perform cross-functional tasks. ... Governance programs span the enterprise, generally starting with a steering committee comprising senior management, often C-level individuals or vice presidents accountable for lines of business. 


How Google's balloons surprised their creator

In the AI community, there's one example of AI creativity that seems to get cited more than any other. The moment that really got people excited about what AI can do, says Mark Riedl at the Georgia Institute of Technology, is when DeepMind showed how a machine learning system had mastered the ancient game Go – and then beat one of the world's best human players at it. "It ended up demonstrating that there were new strategies or tactics for countering a player that no one had really ever used before – or at least a lot of people did not know about," explains Riedl. And yet even this, an innocent game of Go, provokes different feelings among people. On the one hand, DeepMind has proudly described the ways in which its system, AlphaGo, was able to "innovate" and reveal new approaches to a game that humans have been playing for millennia. On the other hand, some questioned whether such an inventive AI could one day pose a serious risk to humans. "It's farcical to think that we will be able to predict or manage the worst-case behaviour of AIs when we can't actually imagine their probable behaviour," wrote Jonathan Tapson at Western Sydney University after AlphaGo's historic victory.


AI Can Help Companies Tap New Sources of Data for Analytics

Just as Google applications can tell you, on the basis of your home address, calendar entries, and map information, that it’s time to leave for the airport if you want to catch your flight, companies can increasingly take advantage of contextual information in their enterprise systems. Automation in analytics — often called “smart data discovery” or “augmented analytics” — is reducing the reliance on human expertise and judgment by automatically pointing out relationships and patterns in data. In some cases the systems even recommend what the user should do to address the situation identified in the automated analysis. Together these capabilities can transform how we analyze and consume data. Historically, data and analytics have been separate resources that needed to be combined to achieve value. If you wanted to analyze financial or HR or supply chain data, for example, you had to find the data — in a data warehouse, mart, or lake — and point your analytics tool to it. This required extensive knowledge of what data was appropriate for your analysis and where it could be found, and many analysts lacked knowledge of the broader context. However, analytics and even AI applications can increasingly provide context. 


5 Reasons to Make Machine Learning Work for Your Business

The original promise of machine learning was efficiency. Even as its uses have expanded beyond mere automation, this remains a core function and one of the most commercially viable use cases. Using machine learning to automate routine tasks, save time and manage resources more effectively has a very attractive paid of side effects for enterprises that do it effectively: reducing expenses and boosting net income. The list of tasks that machine learning can automate is long. As with data processing, how you use machine learning for process automation will depend on which functions exert the greatest drag on your time and resources. ... Machine learning has also proven its worth in detecting trends in large data sets. These trends are often too subtle for humans to tease out, or perhaps the data sets are simply too large for “dumb” programs to process effectively. Whatever the reason for machine learning’s success in this space, the potential benefits are clear as day. For example, many small and midsize enterprises use machine learning technology to predict and reduce customer churn, looking for signs that customers are considering competitors and trigger retention processes with higher probabilities of success.


Accelerating data and analytics transformations in the public sector

Too often, the lure of exciting new technologies influences use-case selection—an approach that risks putting scarce resources against low-priority problems or projects losing momentum and funding when the initial buzz wears off, the people who backed the choice move on, and newer technologies emerge. Organizations can find themselves in a hype cycle, always chasing something new but never achieving impact. To avoid this trap, use cases should be anchored to the organization’s (now clear) strategic aspiration, prioritized, then sequenced in a road map that allows for deployment while building capabilities. There are four steps to this approach. First, identify the relevant activities and processes for delivering the organization’s mission—be that testing, contracting, and vendor management for procurement, or submission management, data analysis, and facilities inspection for a regulator—then identify the relevant data domains that support them.... Use cases should be framed as questions to be addressed, not tools to be built. Hence, a government agency aspiring to improve the uptime of a key piece of machinery by 20 percent while reducing costs by 5 percent might first ask, “How can we mitigate the risk of parts failure?” and not set out to build an AI model for predictive maintenance.


Quantum computing breaking into real-world biz, but not yet into cryptography

A Deloitte Consulting report echoed Baratz's views, stating that quantum computers would not be breaking cryptography or run at computational speeds sufficient to do so anytime soon. However, it said quantum systems could pose a real threat in the long term and it was critical that preparations were carried out now to plan for such a future. On its impact on Bitcoin and blockchain, for instance, the consulting firm estimated that 25% of Bitcoins in circulation were vulnerable to a quantum attack, pointing in particular to the cryptocurrency that currently were stored in P2PK (Pay to Public Key) and reused P2PKH (Pay to Public Key Hash) addresses. These potentially were at risk of attacks as their public keys could be directly obtained from the address or were made public when the Bitcoins were used. Deloitte suggested a way to plug such gaps was post-quantum cryptography, though, these algorithms could pose other challenges to the usability of blockchains. Adding that this new form of cryptography currently was assessed by experts, it said: "We anticipate that future research into post-quantum cryptography will eventually bring the necessary change to build robust and future-proof blockchain applications."


What Is Open RAN (Radio Access Network)?

Open radio access network (RAN) is a term for industry-wide standards for RAN interfaces that support interoperation between vendors’ equipment. The main goal for using open RAN is to have an interoperability standard for RAN elements such as non-proprietary white box hardware and software from different vendors. Network operators that opt for RAN elements with standard interfaces can avoid being stuck with one vendor’s proprietary hardware and software. Open RAN is not inherently open source. The Open RAN standards instead aim to undo the siloed nature of the RAN market, where a handful of RAN vendors only offer equipment and software that is totally proprietary. The open RAN standards being developed use virtual RAN (vRAN) principles and technologies because vRAN brings features such as network malleability, improved security, and reduced capex and opex costs. ... An open RAN ecosystem gives network operators more choice in RAN elements. With a multi-vendor catalog of technologies, network operators have the flexibility to tailor the functionality of their RANs to the operators’ needs. Total vendor lock-in is no longer an issue when organizations are able to go outside of one RAN vendor’s equipment and software stack.



Quote for the day:

"Challenges in life always seek leaders and leaders seek challenges." -- Wayde Goodall

Daily Tech Digest - March 19, 2021

Are A Conscious Artificial Intelligence & Smart Robots Possible?

It would be like teaching a kid by showing a picture of a horse and then a rhino, and then telling him a unicorn is something between these two, so he could mostly identify it without having seen an actual picture before. So the machine would be programmed such that it does not erase the earlier data also known as “catastrophic forgetting” but like the brain have the capability of ”continual learning” by selective activation of cells & overlap networks, and rather use the information to analyse the next dataset or “transfer learning”. Moreover, efforts are underway to teach the machine by just one or two examples, and not the millions of correct examples needed earlier which made the data computation very humungous and actually limited the capability of the machine. Human beings can multi-task effortlessly – can switch efficiently between frying an egg, working in an office, playing badminton and writing music, without compromising each of these activities individually. The UChicago researchers have developed “context-dependent gating” and “synaptic stabilization”, entailing activation of random-only 20 percent of a neural network for each new task, a single node may be involved in dozens of operations; thereby learning as many as 500 tasks with only a small decrease in accuracy.


New phishing campaign targets taxpayer credentials

The scam could result in steep financial losses for taxpayers. Last year alone, the IRS identified more than $2.3 billion in tax fraud schemes. The new infection process is designed to evade antivirus tools and tricks targets into installing the malware via a tax-themed Word Document containing a malicious macro that downloads an OpenVPN client on the targeted machine. The malware dropper establishes a connection to the legitimate cloud service “imgur” and downloads the NetWire or Remcos payloads by way of a technique called steganography, where the malicious code is hidden within an innocuous looking jpeg image file. ... The malware includes a variety of functions including the remote execution of shell commands on the infected machine, browser credential and history theft, the downloading and execution of additional malware payloads, screen captures and keylogging, as well as file and system management capabilities. Both NetWire and Remcos are commercial RATs that are available for online for as little as $10 per month, and both include following the Malware-as-a-Service (MaaS) model, offering their customers subscription-based services with choice of licensing plans, 24/7 customer support and periodic software updates.


Digital transformation: 4 strategy questions to ask

Internal buy-in is the most important aspect of any digital transformation and adoption strategy, and the easiest way to help promote that is to identify internal champions. Clearly defining the team responsible for the implementation of a new tool or process will help give an incentive for that team to ensure adoption is prioritized throughout the organization. It will also help clarify where employees can direct questions. ... Training teams on new processes and tools is easier said than done. It’s important to find a better way to train, not just to ensure that digital transformation is successful and to make sure training really sinks in, but also to make sure your employees feel supported. Building effective training programs is a great way to show your employees that you’re invested in their success and their careers more broadly – helping to increase retention. ... Finally, be sure to set measurable, attainable goals around your digital transformation strategy. These may look vastly different from tool to tool or organization to organization, but adoption will increase if every user understands how transformation efforts will be evaluated.


5 Ways Machine Learning Is Revolutionizing the Healthcare Industry

Machine learning established new methods in drug discovery, such as precision medicine and next-generation sequencing, which can ensure a drug has the intended effect on patients. With the implementation of machine learning techniques, medical experts can develop algorithms to treat disease progression and design specific treatments for each patient, like those with Type 2 diabetes. ... Machine learning aids medical experts in determining the risk for each patient, depending on their symptoms, past medical records, and family history. ML streamlines the process of finding treatments for evolving illnesses, as well as helping researchers to track possible pandemics and to understand better why some diseases are more prevalent in specific cultures and demographics. ... Modern hospitals are high-tech environments run by advanced machines and the staff who are trained. The hospitals are increasingly shifting towards automation, to a future where diagnoses can be made accurately. Machine learning can accelerate disease diagnostics and make the risk of misdiagnosis less likely.


Apps that help parents protect kids from cybercrime may be unsafe too

Parental control apps need many permissions to access particular systems and functions on devices. 80% of parental control apps request access to location, contacts and storage. While these permissions help the apps carry out detailed monitoring, some of them may not be necessary for the app to function as described. For instance, several apps designed to monitor children’s online activity ask for permissions such as “read calendar”, “read contacts” and “record audio” — none of which are justified in the app description or the privacy policy. Many are considered “dangerous permissions”, which means they are used to access information that could affect the user’s privacy and make their device more vulnerable to attack. For example, Boomerang requests more than 91 permissions, 16 of which are considered “dangerous”. The permission “access fine location” for instance, allows the app to access the precise geographic location of the user. The “read phone state” allows the app to know your phone number, network information and status of outgoing calls. It’s not just the apps that get that information. Many of these apps embed data hungry third-party software development kits (SDKs). SDKs are a set of software tools and programs used by developers to save them from tedious coding.


Company uses cognitive neuroscience to help train police officers

The brain has two systems, what you're consciously aware of, and then the non-conscious part, where almost everything happens. That's where your drives and biases and urges and impulses all come up from this, what we call the backchannel of the brain, the non-conscious part of the brain. Most of our actions and behaviors are initiated there a lot of times without our awareness. When we are thinking through something and thinking through answers or trying to problem-solve, we can direct our conscious brain to kind of override some of those impulses and urges and really take control of what we're doing and what we're thinking and how we're behaving. But when we're under stress, our brain is built for the non-conscious brain to take over, to help us with survival, or to help us get out of a scrape, something like that. With police officers, when they've been trained how to respond in certain ways to help people out, to de-escalate events, things like that, when stress starts to rise and get higher, then their non-conscious brain really starts to take over the processing. And that's where even really good officers can do things they later regret is because the non-conscious brain and those urges and impulses to say something or do something happens, and they might regret that later.


How 4 cities are modernizing their IT infrastructure through the cloud

The city's cybersecurity team leads threat management and operates a 24-hour security operations center. The team works with more than 100 city agencies and offices to ensure systems are built and operated in a secure manner to make sure public assistance and healthcare are not compromised. NYC Cyber Command also manages an NYC Secure app that alerts users to unsecure Wi-Fi networks, unsafe Android apps and system tampering. The team uses a cloud infrastructure to find and mitigate threats. The Cyber Command uses a variety of Google cloud services including Cloud Storage, Computer Storage, Kubernetes Engine and Workspace. The team uses BigQuery to analyze batch and streaming data. When the pandemic started, DC Water already had 90% of the organization's systems on the cloud, according to a blog post on Microsoft. The final step was moving in-person operations and services. The organization worked with ESRI to move applications, operational processes and customer requests to Azure. Goals for this work included improving data security and replacing paper processes with digital ones. Durmus Cesur, the manager of work and asset management for DC Water, told Microsoft in the blog post that Azure was the best solution to provide continuous availability and scalability.


Ransom Payments Have Nearly Tripled

A new report from Palo Alto Networks -- which uses data from ransomware investigations, data-leak sites, and the Dark Web — found 337 victims in 56 industries, with manufacturing, healthcare, and construction companies suffering 39% of ransomware attacks in 2020. In addition, ransom demands skyrocketed during the year, doubling both the highest ransom demand — to $30 million—and the highest-known paid ransom, $10 million. The average victim paid more than $312,000, almost a third of the average demand. ... The Palo Alto report combines two sources of the threat intelligence: 252 incidents investigated by the company's data-breach response service over the past two years, and a survey of public leak sites and the Dark Web. Almost two thirds of the incident response cases investigated by the company came in one of four industries in 2020: healthcare, manufacturing, information technology, or construction. ... "As organizations shifted to remote workforces due to the COVID-19 pandemic, ransomware operators adapted their tactics accordingly, including the use of malicious emails containing pandemic-based subjects and even malicious mobile apps claiming to offer information about the virus," the company stated.


Importance of Teaching Data Science in CS Programs

Besides being a lucrative career, data science is among the careers of tomorrow. New innovations in the industrial sectors are highly reliant on data. Technology is becoming dynamic and more data is generated as more people access the internet. With huge amounts of data, industries rely on data scientists to make smart business decisions. In the current digital world, data literacy is very important. People should learn how they can generate meaningful insights from raw data. Data is an untapped potential that can be used to develop various sectors. Fortunately, with the inception of machine learning technologies, organizations can predict and classify information accurately and intelligently. Data science, machine learning, and other similar technologies are subsets of artificial intelligence, which are the driving force behind future products such as self-driving cars and autonomous robots. Such developments are not fiction anymore. The emergence of reinforcement learning and natural language processing has also contributed to these advancements. ... The importance and urgency of data science in the 21st Century cannot be ignored. From providing great insights, statistics, aiding decision-making to hire suitable candidates, data science is overly valuable.


Five Steps To Thinking Like A Software Company

Leading companies feature software stacks that are modular, facilitating rapid innovation. Their developers frequently build in-house software products or platforms by leveraging free, but valuable, open-source software, as well as licensed components for routine functionality. This allows them to create applications faster. One executive stressed the importance of designing components with change in mind, because reconfiguring is always better than rewriting code. Another executive told me that every line of code within this decentralized architecture has a clear owner so that there is specific responsibility for each and every software component. To be clear, commercial solutions have an important role to play and should be a part of the software stack. But it’s the own-account software that matters most. ... In contrast, firms that lead with code typically begin by aiming to solve a focused business problem. They build and iterate on new features and products. Executives at these firms told me that until you try something out and see how your customers, suppliers, or employees react, and whether your business improves as a result, you can’t be sure of what to build.



Quote for the day:

"If you don't start somewhere, you're gonna go nowhere." -- Bob Marley

Daily Tech Digest - March 18, 2021

Confidential computing – the next frontier in security

“Confidential computing is a technique for securing data while in use by creating secure spaces that users rather than administrators’ control,” says Martin O’Reilly, director of research engineering at the Alan Turing Institute. “The idea is to create a trusted execution environment (TEE) or secure enclave, where the data is only accessible by a specific application or user, and only as the data is being processed.” ... Confidential computing’s ‘360-degree protection’ enables data to be processed within a limited part of the computing environment, giving organisations the ability to reduce exposure to sensitive data while also providing greater control and transparency, even allowing businesses to share data for joint processing securely. This represents a significant change, says O’Reilly, pointing out that the ability to create secure spaces where the user controls who has access to the data effectively replicates the trust companies might have in their own IT departments. He notes, however, that the advantages should be weighed against the complexities involved in setting up and managing these technologies.


Ethical AI – Can India be the leader?

The warning signs that have emerged should make the leadership team within all the financial institutions sit up and take serious notice. With enough success achieved on customer acquisition, it's now time to focus energies on creating systems that can serve the financial institutions' best interests and use AI as an effective tool for customer engagement rather than coercive intrusions. AI can indeed be moulded into a great tool for enhanced customer success, although the possibilities of this have not been fully explored. Ethical AI is a practice of evaluating the ethical quality of the impact of its prediction on human life. Debt collection is a good place to develop this practice because human debt collectors with performance targets typically find it hard to navigate the moral quagmire involved in dealing with financial stress. India, the land of spirituality, is well-positioned to become a leader in the practice of Ethical AI. The day is not far when the practice of Ethical AI will be a key differentiator for AI Platforms. On the policy side, countries across the globe are also working on Data Protection laws in the model of EU's GDPR, which would give customers the legal Right to Explanation.


The Progressive Web Apps Guide You’ve Been Looking For

PWA, or Progressive Web Apps, is one of the hottest topics in web development these days. It promises users a native-app like experience and features, but still built with common HTML, CSS, and JavaScript (or you can go even further by using any JavaScript frameworks), how cool is that! ... n simpler words, progressive web apps are typical web apps, built using the same old HTML, CSS, and JavaScript but with additional capabilities such as offline mode, A2HC (add to home screen), background syncing, push notifications, etc. They look and feel like a native apps but they ain’t! ... There are a few things to note about service worker: It’s a JavaScript Worker, so it can’t access the DOM directly. Instead, a service worker can communicate with the pages it controls by responding to messages sent via the postMessage interface, and those pages can manipulate the DOM if needed; Service worker uses a lot of promises, since it’s a script that runs in the background. What it mainly does is to listen to events that happen on the browser then provide a callback for each events we interested.


The path to .NET 5 and Blazor WebAssembly with some fun sprinkled in

One extremely interesting announcement was the new release of Blazor WebAssembly. Blazor lets you build interactive web UI wuth C# instead of JavaScript. Blazor WebAssembly allows users to build a completely isolated application entirely in C# that can run in nearly every web environment, including environments that only support static sites (think only HTML, CSS and Javascript). Blazor WebAssembly does this my compiling all C# code needed to run your application (your code and .NET libraries) into native code that will execute on the browser, not a server somewhere. This is valuable in scenarios when your app needs to run offline, or completely decoupled from a server, and your app only needs a server if it requests data outside of the application (similar to how many Javascript-based applications work). Due to the fact that the application runs in the browser, render times of the UI are near instantaneous, allowing for a great end-user experience. To see these benefits for ourselves, we decided to port a heavily used application to .NET 5 and Blazor WebAssembly and not only reap these benefits, but document the process one would take moving their existing .NET Core application using Blazor Server to .NET 5 using Blazor WebAssembly.


Are Data Engineering Jobs Getting More Popular Than Data Science Jobs?

While some of their activities might overlap, data engineers are primarily about moving and transforming data into pipelines for the data science team. Put it simply, data engineers have three critical tasks to perform — design, build and arrange data pipelines. In contrast, data scientists analyse, test, aggregate and optimise data. ... Data engineers essentially collect, generate, store, enrich and process data in real-time or in batches. Data engineering involves building data infrastructure and data architecture. Data engineers require experience in software engineering, programming languages, and a firm grip on core technical skills. Understanding ETL, SQL, and programming languages such as Java, Scala, C++, and Python are desired. ... The data science strategy in an organisation deals with data infrastructure, data warehousing, data mining, data modelling, data crunching, and metadata management, most of which are carried out by data engineers. Studies suggest most data science projects fall through as data engineers and data scientists find themselves at cross purposes. Many companies fail to recognise the importance of hiring data engineers. While most companies are starting to realise the importance of data engineers, the talent shortage is all too real.


Using Machine Learning in Testing and Maintenance

With machine learning, we can reduce maintenance efforts and improve the quality of products. It can be used in various stages of the software testing life-cycle, including bug management, which is an important part of the chain. We can analyze large amounts of data for classifying, triaging, and prioritizing bugs in a more efficient way by means of machine learning algorithms. Mesut Durukal, a test automation engineer at Rapyuta Robotics, spoke at Aginext 2021 about using machine learning in testing. Durukal uses machine learning to classify and cluster bugs. Bugs can be classified according to severity levels or responsible team or person. Severity assignment is called triage and important in terms of prioritization, where the assignment of bugs to the correct team or person prevents a waste of time. Clustering bugs helps to see whether they heap together on specific features. Exploring the available data on bugs with machine learning algorithms gave him more insight into the health of their products and the effectiveness of the processes that were used.


Digital Transformation Requires Redefining Role of Data Governance

Today’s data governance and data management practices must be redefined to support the organization’s business needs and ultimately underpin the organization’s data monetization strategy. A Data Monetization practice must: Evangelize a compelling vision regarding the economic potential of data and analytic assets to power an organization’s digital transformation; Educate senior executives, business stakeholders and strategic customers on how to “Think Like a Data Scientist” in identifying where and how data and analytics can deliver material business value; Apply Design Thinking and Value Engineering concepts in collaborating with business stakeholders to identify, validate, value and prioritize the organization’s high-value use cases that will drive the organization’s data and analytics development roadmap; Champion a Data Science team to “engineer” reusable, continuously-learning and adapting analytic assets that support the organization’s high priority use cases; Develop an analytics culture that synergizes the AI / ML model-to-human collaboration that empowers teams at the point of customer engagement and operational execution.


AI and IoT: Transforming Business

The execution of modern technologies such as Artificial Intelligence and IoT has been changing the entire business world. A recent survey stated that almost 500 IT professionals claiming AI and IoT as most emerging technologies do remodeling business operations and compelling companies to invest more to gain a competitive advantage. And the reasons are simple. The amalgamation of IoT and AI can create smart strategies that can read human preferences and help management make informed decisions with zero error. Not convinced? Let’s check it out with a real-life example. One of the world’s renowned car manufacturers, BMW, has started using AI and IoT in their manufacturing process. It uses censored robots in their premises which help workers while producing innovative cars. They are also leveraging AI for driverless cars for the future. In fact, AI and IoT are affecting the entire transportation industry. Interactive maps and smart route optimization is making it easy for drivers to reach the destination early. It saves fuel cost and reduces journey time. This is why you might have heard why entrepreneurs embrace AI solutions in their taxi app clone development because it plans routes based on peak hours and road construction.


10 Ways Enterprises Can Use the Edge

Edge use cases are expanding across industries as companies move compute and analytics capabilities to the edge. Some companies want to reduce latency. Others want to gain greater insights into what's happening in the field whether people, crops, or oil rigs. "Edge computing enables companies and other types of organizations to analyze large amounts of data on site or on devices in real time," said Shamik Mishra, CTO for Connectivity in the Engineering and R&D Business at global consulting firm Capgemini. "This can enable several new opportunities in terms of new sources of revenue, improved productivity, and decreased costs." In fact, there's an entire world of the Internet of Things (IoT) innovation happening that makes edge use cases even more compelling including smart homes, wearables, AR video games and increasingly intelligent vehicles. Gartner expects the IoT platform market to grow to $7.6 billion by 2024, which represents both on-premises and cloud deployments. The company considers PaaS a key enabler of digital scenarios. Allied Market Research sees the broader opportunity worth $16.5 billion by 2025, driven by the desire to avoid network latency problems and restrictions on bandwidth usage for storing data in the cloud...."


How to Maintain a Healthy Codebase While Shipping Fast

To determine what proportion of your sprint to allocate to tech debt, simply find the overlap between the parts of your codebase you'll modify with your feature work and the parts of your codebase where your worse tech debt lives. You can then scope out the tech debt work and allocate resources accordingly. Some teams even increase the scope of their feature work to include the relevant tech debt clean-up. More in this article 'How to stop wasting time on tech debt.' For this to work, individual contributors need to track medium-sized debt whenever they come across it. It is then the Team Lead's responsibility to prioritize this list of tech debt, and to discuss it with the Product Manager prior to sprint planning so that engineering resources can be allocated effectively. Every once in a while, your team will realize that some of the medium-sized debt they came across is actually due to a much larger piece of debt. For example, they may realize that the reason the front-end code is under-performing is that they should be using a different framework for the job. Left unattended, these large pieces of debt can cause huge problems, and — like all tech debt — get much worse as time goes by.



Quote for the day:

"No man is good enough to govern another man without that other's consent." -- Abraham Lincoln