Daily Tech Digest - February 10, 2022

What is Relational Machine Learning?

Much of the recent deep learning research was then about discovering models and learning representations capturing data in various forms of sets and graphs. However, it is only rarely acknowledged that these structured learning representations have for long been studied in Relational Machine Learning. A relation, as you might recall, is a subset of a cartesian product defined over some sets of objects. Every set is thus simply a degenerated case of some (unary) relation. Every graph can then be seen as an instantiation of a binary relation over the same set of objects (nodes). Tabular data, with more than 2 columns (objects), then correspond to relation of a higher arity, also known as a hypergraph. Add multiple such relations (tables) over the objects, and you have a relational database. Much of the real-world data is then stored in such relational databases, which you have certainly encountered before. Now imagine that your learning samples are not prepared nicely as rows in a single table, but spread across multiple interlinked tables of the database, where different samples consist of different types and numbers of objects, with each object being characterized by a different set of attributes.

Blockchain technology in financial services

Decentralised financial operations have benefitted greatly from networks built on public blockchain. Hosting transaction data on these networks allow for transparency and visibility from all users involved. However, Conor Svensson, founder and CEO of Web3 Labs, believes that regulators need to bring this infrastructure further up in their agendas, for financial service bodies to drive true value. Svensson explained: “We have seen financial institutions offer institutional cryptocurrency product to meet the demands of institutional investors. Whilst on the surface, this may not sound like a key value driver, other than providing additional revenue sources for the firms making such access possible, I believe it is a key step in facilitating mass adoption of some of the key innovations taking place on public blockchain networks such as Ethereum. “With the innovations during the past two years in DeFi, investors are able to get yields that far surpass what is currently available in traditional bank savings accounts. The savvy users of DeFi, can see returns on crypto assets that are pegged to the US dollar achieve yields of 7%+ per annum, far exceeding what people can get in bank accounts. 

The power of AI can be unleashed with a focus on ethics

Consumers’ concerns about AI should not be ignored – and they are not insurmountable. As businesses invest in AI, they need to adopt a wider perspective that is not focused purely on the technology itself. There are three strands to this approach for government, regulators and businesses to tackle, each of which can address ethical questions and improve trust and confidence. The first of these is building trust into the foundations of AI through a human-led approach, starting with the design of AI algorithms. Input data needs to be bias-free, while the people behind AI tools need to have the right training to ensure bias-free outputs. Above all, there should be transparency and accountability in how decisions are made by AI. Citing technological limitations as a reason for a lack of transparency or not owning accountability for AI’s decision-making won’t be accepted by consumers – or regulators. To bridge the trust gap, collaboration is key and AI stakeholders must work as an ecosystem to mitigate risks. Businesses must ensure AI is a board-level agenda item and a core part of their overall strategy, and there must be a proactive dialogue between government, suppliers, consumers and regulators.

Alibaba Open-Sources AutoML Algorithm KNAS

Researchers from Alibaba Group and Peking University have open-sourced Kernel Neural Architecture Search (KNAS), an efficient automated machine learning (AutoML) algorithm that can evaluate proposed architectures without training. KNAS uses a gradient kernel as a proxy for model quality, and uses an order of magnitude less compute power than baseline methods. The algorithm and a set of experiments were described in a paper published in Proceedings of Machine Learning Research. Unlike many AutoML algorithms which require that proposed models undergo the full training process, KNAS can predict which model architectures will perform well without actually training them, thereby potentially saving many hours of compute time. When evaluated on the NAS-Bench-201 computer vision benchmark, KNAS achieved a 25x speedup while producing results with "competitive" accuracies compared to other AutoML methods. When evaluated on text classification tasks, KNAS produced models that achieved better accuracy than a pre-trained RoBERTA baseline model.

Charge DeFi: How Algorithms Create Stability in a Decentralized Way

Charge DeFi is a combination of an algorithmic crypto token and rebase mechanics. A stablecoin is a cryptocurrency whose value is pegged to a single unit of a fiat currency, usually 1 USD. Normally, this is by means of “tethering” in which a company acquires an equivalent amount of say, USD and promises to back each unit of stablecoin 1:1. However, inherent in this mechanism is the requirement to trust the guarantor, which requires constant and often expensive monitoring. An algorithmic crypto token takes stability to the next level. Instead of a fixed peg, an algorithm is used to adjust the price of a token based on pre-set conditions which can be written into a smart contract and launched in a fully decentralized way. Consequently, there is no input by any 3rd party thereafter, with the algorithm executing according to demand, supply and market movements. ... One of the core features of this new ecosystem is the rebase mechanic implemented in the contracts. Rebase mechanics implement price-elastic tokens that adjust the circulating supply to influence a token price. 

The innovation behind AI at Scale

One important approach to advancing model capabilities is the training of expert models with subtasks using an ensemble method, named Mixture of Experts (MoE). The MoE architecture also preserves sublinear computation with respect to model parameters that provides a promising path to improve model quality via scaling out trillions of parameters without increasing training cost. We have also developed MoE models to our pretrained model family accelerated with DeepSpeed. These large-scale pre-trained models become platforms and can be adapted to specific domains or tasks by using domain-specific data in a privacy-compliant manner. We refer to this collection of base and domain-adapted models as “AI models as a platform,” which can be used directly to build new experiences with zero-shot/few-shot learning or used to build more task-specific models through the process of fine-tuning the model with a labeled dataset for the task. In a similar fashion, you can domain adapt or fine-tune these models with your own enterprise data privately within the scope of your tenant and use them in your enterprise applications to learn representations and concepts unique to your business and products.

IT leadership: 8 tips to improve resiliency

There’s no rest for IT, though. The year ahead promises to put technology organizations to the test as they enable enterprises to meet rapidly changing customer demands, create better employee experiences, mitigate complex security challenges, effectively and ethically integrate emerging artificial intelligence (AI) tools, consider the impact of climate change, and ensure resilient business systems in an uncertain global environment. The only way for CIOs to tackle these challenges – and avoid burnout – is to improve their own resilience as leaders and as individuals. “When we’re exhausted and we’re burned out, we cannot think big,” says Adam Markel, author of the forthcoming book, Change Proof: Leveraging the Power of Uncertainty to Build Long-Term Resilience. “And [IT leaders] have to constantly be bigger than the problems that are presented. They have to be able to think outside of the bounds of whatever challenge or problem is coming at them.” To develop resilience, it’s important to understand what it is – and isn’t. “Resilience is actually about how we recharge, not about how we endure,” says Markel. 

TypeScript and the Power of a Statically-Typed Language

TypeScript came about within Microsoft in 2012 as a way to help their C++ or C# developers to write large-scale web applications, recalled Hoban, who was then part of the developer tools group at Microsoft. The natural choice was to use JavaScript, the dominant language for the web. But JavaScript didn’t offer much tooling to help developers organize, assist and check their code writing. Various early solutions worked on by Microsoft and other web-scale companies were mostly efforts to compile traditional languages such as C++ into JavaScript, though this left developers at “an arm’s length from the platform,” Hoban said. They wanted to develop a language that had the “ability to express types into the syntax as a lightweight extension to JavaScript, but … still offer as much sort of inference as possible, so that you need to use those annotations sort of as infrequently as possible,” Hoban said. The answer came in making a superset of JavaScript that would be fully compatible with JavaScript and can be transpiled into JavaScript for full browser compatibility. To add rigor, it came with additional type checking. 

Turn the “great resignation” into the “great renegotiation”

Workers are leaving their employers in droves, seeking greater fulfillment and better pay, among other opportunities. This great resignation, though, could instead be a great renegotiation. Leaders have a chance now to redesign their organizations in a way that’s more exciting and fulfilling for employees. Doing so requires businesses to rethink their fundamental ways of operating. It means putting everything on the table, including roles, schedules, key performance indicators, individual performance metrics, and more. It takes time, energy, and work. But the incentive is clear. ... Internal negotiations are often much more difficult than external ones, INSEAD professor Horacio Falc√£o and financial services professional Alena Komaromi explain in the INSEAD Knowledge blog. People who work together often assume they’ll have similar aims, so they’re more likely to be underprepared and fail to consider one another’s conflicting interests. Internal power dynamics also often hamper negotiations. It helps to train employees on how to negotiate.

Cross-Architecture Capabilities: Thinking With GPUs

Outside of specialized applications, most programmers have traditionally kept their focus on CPUs. Until recently, CPUs were the only substantial processing power in most machines. However, with powerful GPUs now a standard component in a wide variety of devices, the CPU is no longer the only game in town. Developers can leverage the additional processing power of GPUs and other processors in conjunction with traditional CPUs. To do so, they need to adopt parallel programming techniques to create applications with cross-architecture capabilities. Intel’s oneAPI provides a powerful suite of tools for creating software that takes full advantage of cross-architecture resources. oneAPI enables developers to think about the possibilities of enhancing their applications with some extra GPU processing power. One of oneAPI’s critical features is code reusability. Using Intel’s Data Parallel C++, an implementation of C++, and SYCL standards for data parallelism, developers can write code that works with CPUs, GPUs, and FPGAs. 

Quote for the day:

"Hiding from yourself is the surest path to self hatred, self pity and a whole lot of missed potential." -- Jon Westernberg

No comments:

Post a Comment