What is Relational Machine Learning?
Much of the recent deep learning research was then about discovering models and
learning representations capturing data in various forms of sets and graphs.
However, it is only rarely acknowledged that these structured learning
representations have for long been studied in Relational Machine Learning. A
relation, as you might recall, is a subset of a cartesian product defined over
some sets of objects. Every set is thus simply a degenerated case of some
(unary) relation. Every graph can then be seen as an instantiation of a binary
relation over the same set of objects (nodes). Tabular data, with more than 2
columns (objects), then correspond to relation of a higher arity, also known as
a hypergraph. Add multiple such relations (tables) over the objects, and you
have a relational database. Much of the real-world data is then stored in such
relational databases, which you have certainly encountered before. Now imagine
that your learning samples are not prepared nicely as rows in a single table,
but spread across multiple interlinked tables of the database, where different
samples consist of different types and numbers of objects, with each object
being characterized by a different set of attributes.
Blockchain technology in financial services
Decentralised financial operations have benefitted greatly from networks built
on public blockchain. Hosting transaction data on these networks allow for
transparency and visibility from all users involved. However, Conor Svensson,
founder and CEO of Web3 Labs, believes that regulators need to bring this
infrastructure further up in their agendas, for financial service bodies to
drive true value. Svensson explained: “We have seen financial institutions offer
institutional cryptocurrency product to meet the demands of institutional
investors. Whilst on the surface, this may not sound like a key value driver,
other than providing additional revenue sources for the firms making such access
possible, I believe it is a key step in facilitating mass adoption of some of
the key innovations taking place on public blockchain networks such as Ethereum.
“With the innovations during the past two years in DeFi, investors are able to
get yields that far surpass what is currently available in traditional bank
savings accounts. The savvy users of DeFi, can see returns on crypto assets that
are pegged to the US dollar achieve yields of 7%+ per annum, far exceeding what
people can get in bank accounts.
The power of AI can be unleashed with a focus on ethics
Consumers’ concerns about AI should not be ignored – and they are not
insurmountable. As businesses invest in AI, they need to adopt a wider
perspective that is not focused purely on the technology itself. There are
three strands to this approach for government, regulators and businesses to
tackle, each of which can address ethical questions and improve trust and
confidence. The first of these is building trust into the foundations of AI
through a human-led approach, starting with the design of AI algorithms. Input
data needs to be bias-free, while the people behind AI tools need to have the
right training to ensure bias-free outputs. Above all, there should be
transparency and accountability in how decisions are made by AI. Citing
technological limitations as a reason for a lack of transparency or not owning
accountability for AI’s decision-making won’t be accepted by consumers – or
regulators. To bridge the trust gap, collaboration is key and AI stakeholders
must work as an ecosystem to mitigate risks. Businesses must ensure AI is a
board-level agenda item and a core part of their overall strategy, and there
must be a proactive dialogue between government, suppliers, consumers and
regulators.
Alibaba Open-Sources AutoML Algorithm KNAS
Researchers from Alibaba Group and Peking University have open-sourced Kernel
Neural Architecture Search (KNAS), an efficient automated machine learning
(AutoML) algorithm that can evaluate proposed architectures without training.
KNAS uses a gradient kernel as a proxy for model quality, and uses an order of
magnitude less compute power than baseline methods. The algorithm and a set of
experiments were described in a paper published in Proceedings of Machine
Learning Research. Unlike many AutoML algorithms which require that proposed
models undergo the full training process, KNAS can predict which model
architectures will perform well without actually training them, thereby
potentially saving many hours of compute time. When evaluated on the
NAS-Bench-201 computer vision benchmark, KNAS achieved a 25x speedup while
producing results with "competitive" accuracies compared to other AutoML
methods. When evaluated on text classification tasks, KNAS produced models
that achieved better accuracy than a pre-trained RoBERTA baseline model.
Charge DeFi: How Algorithms Create Stability in a Decentralized Way
Charge DeFi is a combination of an algorithmic crypto token and rebase
mechanics. A stablecoin is a cryptocurrency whose value is pegged to a single
unit of a fiat currency, usually 1 USD. Normally, this is by means of
“tethering” in which a company acquires an equivalent amount of say, USD and
promises to back each unit of stablecoin 1:1. However, inherent in this
mechanism is the requirement to trust the guarantor, which requires constant
and often expensive monitoring. An algorithmic crypto token takes stability to
the next level. Instead of a fixed peg, an algorithm is used to adjust the
price of a token based on pre-set conditions which can be written into a smart
contract and launched in a fully decentralized way. Consequently, there is no
input by any 3rd party thereafter, with the algorithm executing according to
demand, supply and market movements. ... One of the core features of this
new ecosystem is the rebase mechanic implemented in the contracts. Rebase
mechanics implement price-elastic tokens that adjust the circulating supply to
influence a token price.
The innovation behind AI at Scale
One important approach to advancing model capabilities is the training of
expert models with subtasks using an ensemble method, named Mixture of
Experts (MoE). The MoE architecture also preserves sublinear computation
with respect to model parameters that provides a promising path to improve
model quality via scaling out trillions of parameters without increasing
training cost. We have also developed MoE models to our pretrained model
family accelerated with DeepSpeed. These large-scale pre-trained models
become platforms and can be adapted to specific domains or tasks by using
domain-specific data in a privacy-compliant manner. We refer to this
collection of base and domain-adapted models as “AI models as a platform,”
which can be used directly to build new experiences with zero-shot/few-shot
learning or used to build more task-specific models through the process of
fine-tuning the model with a labeled dataset for the task. In a similar
fashion, you can domain adapt or fine-tune these models with your own
enterprise data privately within the scope of your tenant and use them in
your enterprise applications to learn representations and concepts unique to
your business and products.
IT leadership: 8 tips to improve resiliency
There’s no rest for IT, though. The year ahead promises to put technology
organizations to the test as they enable enterprises to meet rapidly
changing customer demands, create better employee experiences, mitigate
complex security challenges, effectively and ethically integrate emerging
artificial intelligence (AI) tools, consider the impact of climate change,
and ensure resilient business systems in an uncertain global environment.
The only way for CIOs to tackle these challenges – and avoid burnout – is to
improve their own resilience as leaders and as individuals. “When we’re
exhausted and we’re burned out, we cannot think big,” says Adam Markel,
author of the forthcoming book, Change Proof: Leveraging the Power of
Uncertainty to Build Long-Term Resilience. “And [IT leaders] have to
constantly be bigger than the problems that are presented. They have to be
able to think outside of the bounds of whatever challenge or problem is
coming at them.” To develop resilience, it’s important to understand what it
is – and isn’t. “Resilience is actually about how we recharge, not about how
we endure,” says Markel.
TypeScript and the Power of a Statically-Typed Language
TypeScript came about within Microsoft in 2012 as a way to help their C++ or
C# developers to write large-scale web applications, recalled Hoban, who was
then part of the developer tools group at Microsoft. The natural choice was
to use JavaScript, the dominant language for the web. But JavaScript didn’t
offer much tooling to help developers organize, assist and check their code
writing. Various early solutions worked on by Microsoft and other web-scale
companies were mostly efforts to compile traditional languages such as C++
into JavaScript, though this left developers at “an arm’s length from the
platform,” Hoban said. They wanted to develop a language that had the
“ability to express types into the syntax as a lightweight extension to
JavaScript, but … still offer as much sort of inference as possible, so that
you need to use those annotations sort of as infrequently as possible,”
Hoban said. The answer came in making a superset of JavaScript that would be
fully compatible with JavaScript and can be transpiled into JavaScript for
full browser compatibility. To add rigor, it came with additional type
checking.
Turn the “great resignation” into the “great renegotiation”
Workers are leaving their employers in droves, seeking greater fulfillment and
better pay, among other opportunities. This great resignation, though, could
instead be a great renegotiation. Leaders have a chance now to redesign their
organizations in a way that’s more exciting and fulfilling for employees.
Doing so requires businesses to rethink their fundamental ways of operating.
It means putting everything on the table, including roles, schedules, key
performance indicators, individual performance metrics, and more. It takes
time, energy, and work. But the incentive is clear. ... Internal negotiations
are often much more difficult than external ones, INSEAD professor Horacio
Falcão and financial services professional Alena Komaromi explain in the
INSEAD Knowledge blog. People who work together often assume they’ll have
similar aims, so they’re more likely to be underprepared and fail to consider
one another’s conflicting interests. Internal power dynamics also often hamper
negotiations. It helps to train employees on how to negotiate.
Cross-Architecture Capabilities: Thinking With GPUs
Outside of specialized applications, most programmers have traditionally kept
their focus on CPUs. Until recently, CPUs were the only substantial processing
power in most machines. However, with powerful GPUs now a standard component
in a wide variety of devices, the CPU is no longer the only game in town.
Developers can leverage the additional processing power of GPUs and other
processors in conjunction with traditional CPUs. To do so, they need to adopt
parallel programming techniques to create applications with cross-architecture
capabilities. Intel’s oneAPI provides a powerful suite of tools for creating
software that takes full advantage of cross-architecture resources. oneAPI
enables developers to think about the possibilities of enhancing their
applications with some extra GPU processing power. One of oneAPI’s critical
features is code reusability. Using Intel’s Data Parallel C++, an
implementation of C++, and SYCL standards for data parallelism, developers can
write code that works with CPUs, GPUs, and FPGAs.
Quote for the day:
"Hiding from yourself is the surest
path to self hatred, self pity and a whole lot of missed potential." --
Jon Westernberg
No comments:
Post a Comment