Thursday, April 15, 2021
  • Setup menu at Appearance » Menus and assign menu to Top Bar Navigation
Advertisement
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
No Result
View All Result
Home Machine Learning

Harnessing the value of machine learning in logical data warehouses

April 9, 2019
in Machine Learning
Harnessing the value of machine learning in logical data warehouses
586
SHARES
3.3k
VIEWS
Share on FacebookShare on Twitter

Credit: Google News

Logical data warehouses have been gaining popularity for some time, and their potential benefits are many. They help provide a unified infrastructure for querying across disparate data sources. They also help provide security and metadata management on top of multiple analytical data management systems.

You might also like

Sailthru Announces Machine Learning Features for Improved Lifecycle Optimization

Global Machine Learning Market In-Depth Qualitative Insights & Future Growth Analysis 2021-2027 – The Courier

Seminar on Machine Learning Techniques in Banking – India Education| Global Education |Education News

The LDW is needed because, to make the data actionable, organizations often have to run multiple analytical systems because data volumes are too big and distributed to fit into a single physical system and also because analytic needs are too diverse to be solved by a single engine.

LDWs can not only help traditional analytics; they can also aid artificial intelligence and machine learning initiatives by providing agile, governed access to data no matter where it is located or how it is formatted. Not only do they greatly help with data discovery; LDWs also provide an easy way to expose reusable “logical” data sets to data scientists, so they do not need to take care of complex issues such as data combination from several data sources, complex transformations and performance optimization.

Recent studies have found that data scientists spend roughly 80 percent of their time finding, combining and preparing data for analysis, so this is no small feat. Finally, LDW technology can offer an easy way to publish the results of ML efforts for business users and applications.

Here are three steps to establishing a unified infrastructure for analytics.

Much like how LDWs can aid AI and ML, machine learning technology itself can also be used to automatically tune LDW performance. This is accomplished by leveraging ML strategies to analyze past queries in the system and then automatically detect bottlenecks while identifying optimization opportunities.

This is a key benefit where the data that is processed in LDW queries tend to be huge and distributed across many different systems Therefore, and to improve the quality of service perceived by end users, it’s critical to make the best optimization decisions by following these steps:

Prepare the infrastructure
The first step is to move the processing to the data to avoid latencies associated with transferring large volumes of data at query time. As analyst Rick F. van der Lans points out, data virtualization—when placed at the core of the LDW architecture—is a sophisticated data integration technology that can facilitate this step without physically moving any data. Instead, it accesses the data in its existing location—in real time, as needed.

Additionally, there may be opportunities for big performance gains by selectively moving small datasets from one data source to another. For example, if two datasets are usually combined, and one of them is relatively small, it can make sense to replicate it to speed up queries. ML techniques can be applied to detect these situations and automatically recommend selective data replications with low cost and big impact in overall performance, as the LDW can use this information to automatically perform these selective replications and keep them up-to-date.

Putting machine learning to work
The next step is to leverage the natural capacity of ML to analyze past workloads. For example, ML techniques can automatically identify overlaps between groups of analytic queries, which provide opportunities for acceleration, such as pre-computing last year’s sales data aggregated by the most common dimensions (for example, product, customer or point-of-sale). Not only can this serve as a starting point for many different queries without needing to compute everything from scratch, ML techniques also can identify the optimal system in which to create each intermediate result to maximize data co-location at query time.

Tuning the LDW
At this point, it is possible to implement a tuning process whereby, the ML measures the throughput of the infrastructure, suggests or implements changes, measures again and suggests new changes in an iterative process. Over time—and by continuously measuring, implementing changes, learning and repeating—such a system will gradually improve performance, similar to a tuning expert.

LDWs that are created using data virtualization provide the only viable way to establish a unified infrastructure for analytics across large enterprises. But achieving the optimal configuration for performance can be challenging because of the distributed nature of today’s organizations. By intelligently analyzing previous workloads, ML techniques can automatically suggest (and even implement) optimization actions with the lowest cost and the highest performance gains.

This story originally appeared in Information Management.


Alberto Pan

Alberto Pan

Alberto Pan is chief technical officer at Denodo, a provider of data virtualization software, and an associate professor at University of A Coruña.

Reprints and licensing

For reprint and licensing requests for this article, click here.

Credit: Google News

Previous Post

Microsoft's April Patch Tuesday comes with fixes for two Windows zero-days

Next Post

The Citizen’s Perspective on the Use of AI in Government: Cautious Optimism

Related Posts

Sailthru Announces Machine Learning Features for Improved Lifecycle Optimization
Machine Learning

Sailthru Announces Machine Learning Features for Improved Lifecycle Optimization

April 14, 2021
Global Machine Learning Market In-Depth Qualitative Insights & Future Growth Analysis 2021-2027 – The Courier
Machine Learning

Global Machine Learning Market In-Depth Qualitative Insights & Future Growth Analysis 2021-2027 – The Courier

April 14, 2021
Seminar on Machine Learning Techniques in Banking – India Education| Global Education |Education News
Machine Learning

Seminar on Machine Learning Techniques in Banking – India Education| Global Education |Education News

April 14, 2021
Applying artificial intelligence to science education — ScienceDaily
Machine Learning

Machine learning can help slow down future pandemics — ScienceDaily

April 14, 2021
ML Ops and the Promise of Machine Learning at Scale
Machine Learning

ML Ops and the Promise of Machine Learning at Scale

April 14, 2021
Next Post
The Citizen’s Perspective on the Use of AI in Government: Cautious Optimism

The Citizen’s Perspective on the Use of AI in Government: Cautious Optimism

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recommended

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

January 6, 2019
Microsoft, Google Use Artificial Intelligence to Fight Hackers

Microsoft, Google Use Artificial Intelligence to Fight Hackers

January 6, 2019

Categories

  • Artificial Intelligence
  • Big Data
  • Blockchain
  • Crypto News
  • Data Science
  • Digital Marketing
  • Internet Privacy
  • Internet Security
  • Learn to Code
  • Machine Learning
  • Marketing Technology
  • Neural Networks
  • Technology Companies

Don't miss it

Six courses to build your technology skills in 2021 – IBM Developer
Technology Companies

A brief intro to Red Hat OpenShift for Node.js developers – IBM Developer

April 15, 2021
Microsoft Defender for Endpoint now protects unmanaged BYO devices
Internet Security

Microsoft Defender for Endpoint now protects unmanaged BYO devices

April 15, 2021
New JavaScript Exploit Can Now Carry Out DDR4 Rowhammer Attacks
Internet Privacy

New JavaScript Exploit Can Now Carry Out DDR4 Rowhammer Attacks

April 15, 2021
Sailthru Announces Machine Learning Features for Improved Lifecycle Optimization
Machine Learning

Sailthru Announces Machine Learning Features for Improved Lifecycle Optimization

April 14, 2021
Data Labeling Service — How to Get Good Training Data for ML Project? | by ByteBridge | Apr, 2021
Neural Networks

Data Labeling Service — How to Get Good Training Data for ML Project? | by ByteBridge | Apr, 2021

April 14, 2021
The Search Engine Land Awards are open: Wednesday’s daily brief
Digital Marketing

The Search Engine Land Awards are open: Wednesday’s daily brief

April 14, 2021
NikolaNews

NikolaNews.com is an online News Portal which aims to share news about blockchain, AI, Big Data, and Data Privacy and more!

What’s New Here?

  • A brief intro to Red Hat OpenShift for Node.js developers – IBM Developer April 15, 2021
  • Microsoft Defender for Endpoint now protects unmanaged BYO devices April 15, 2021
  • New JavaScript Exploit Can Now Carry Out DDR4 Rowhammer Attacks April 15, 2021
  • Sailthru Announces Machine Learning Features for Improved Lifecycle Optimization April 14, 2021

Subscribe to get more!

© 2019 NikolaNews.com - Global Tech Updates

No Result
View All Result
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News

© 2019 NikolaNews.com - Global Tech Updates