Friday, April 23, 2021
  • Setup menu at Appearance » Menus and assign menu to Top Bar Navigation
Advertisement
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
No Result
View All Result
Home Big Data

Confluent announces ‘infinite’ Kafka release

July 2, 2020
in Big Data
Confluent announces ‘infinite’ Kafka release
585
SHARES
3.3k
VIEWS
Share on FacebookShare on Twitter

Confluent, Inc. is today announcing infinite data retention as a new feature on its Confluent Cloud-managed Apache Kafka service.  The company, founded by Kafka’s creators, is announcing infinite data retention as part of its “Project Metamorphosis,” which aims to imbue Kafka with modern cloud properties. Infinite retention rolls out this month for Confluent Cloud Standard and Dedicated Clusters on AWS, with other cloud providers coming later. The feature could really change the way Kafka is used.

Must read:

You might also like

TigerGraph and Google Cloud make it a threesome

Redis Labs unveils Redis Database 7.0

GoodData.CN: Veteran analytics provider goes cloud-native

Cloudy Kafka

In a briefing with ZDNet, Confluent CEO Jay Kreps explained that modern cloud properties, like elasticity, fully-managed operations, and separation of compute and storage, have largely eluded Kafka. Instead, companies that have adopted Kafka have had to manage a lot of moving parts and, in particular, have had to manage storage very explicitly and diligently.

Customers using Confluent Cloud dedicated clusters have had to pre-provision the storage they’ve needed and, typically, only a one-week window of data has been kept in Kafka topics. Now data will simply be able to accumulate, without limit. Customers will of course pay for the cloud storage needed to retain this data, but that is much more cost-efficient than using storage on the Kafka cluster nodes themselves. As a result, customers won’t have to pre-provision any such node-level storage, and therefore charges for the clusters themselves won’t change, with costs continuing to be based on the volume of data ingested and processed.

Take off your coat and stay awhile

Since its inception, the very premise of streaming data systems has been that they act as a transfer point for data, and not a permanent home. As a result, they’ve typically served up a short window of recent and real-time data, while historical data has had to be retrieved from other systems. So-called “Lambda architectures” that have sought to integrate real-time and historical/batch data platforms, in order to provide applications with both types of data, might be better-characterized as Rube Goldberg architectures.

But with infinite data retention, Confluent is promoting the idea that Kafka can be used as a database of record, rather than just a conduit. Even Confluent would likely not suggest that Kafka replace data warehouses and data lakes, or act as an analytics database platform. But just as relational OLTP (online transactional processing) platforms act as permanent database stores for transactional data, Confluent is saying Kafka can now do the same for event stream data. Essentially, Kafka can be data’s permanent home, rather than just a hotel where data stays when it first arrives.

It’s a (K)SQL world, we’re just living in it

The relational database analogy is more than just conceptually useful. About three years ago, Confluent introduced KSQL, a SQL query layer for Kafka. This feature allows developers to work with Kafka as if were a relational database and Kreps acknowledged KSQL as a major driver for customer demand that Kafka acts as a repository for historical as well as real-time data. 

Also read: Kafka gets SQL with KSQL

While KSQL isn’t part of Apache Kafka per se, it is available from Confluent on a community-licensed basis for use with open-source Kafka implementations. On Confluent Cloud, KSQL technology is available as a fully managed service, alongside the core Confluent platform.

Streaming data is becoming increasingly important, just as data protection regulations are creating imperatives around the management of historical data. In this context, infinite data retention for Kafka looks to be a game-changing capability that could make IoT, log, social, and clickstream data as easily query-able and manageable as transactional data has been for quite some time.

Credit: Zdnet

Previous Post

Kirobo claims ‘retrievable’ crypto transaction solution

Next Post

A New Ransomware Targeting Apple macOS Users Through Pirated Apps

Related Posts

TigerGraph and Google Cloud make it a threesome
Big Data

TigerGraph and Google Cloud make it a threesome

April 22, 2021
Redis Labs unveils Redis Database 7.0
Big Data

Redis Labs unveils Redis Database 7.0

April 21, 2021
GoodData.CN: Veteran analytics provider goes cloud-native
Big Data

GoodData.CN: Veteran analytics provider goes cloud-native

April 20, 2021
Rockset takes a deeper dive into enterprise data pool
Big Data

Rockset takes a deeper dive into enterprise data pool

April 16, 2021
Chainlink 2.0 brings off-chain compute to blockchain oracles, promotes adoption of hybrid smart contracts
Big Data

Chainlink 2.0 brings off-chain compute to blockchain oracles, promotes adoption of hybrid smart contracts

April 16, 2021
Next Post
A New Ransomware Targeting Apple macOS Users Through Pirated Apps

A New Ransomware Targeting Apple macOS Users Through Pirated Apps

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recommended

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

January 6, 2019
Microsoft, Google Use Artificial Intelligence to Fight Hackers

Microsoft, Google Use Artificial Intelligence to Fight Hackers

January 6, 2019

Categories

  • Artificial Intelligence
  • Big Data
  • Blockchain
  • Crypto News
  • Data Science
  • Digital Marketing
  • Internet Privacy
  • Internet Security
  • Learn to Code
  • Machine Learning
  • Marketing Technology
  • Neural Networks
  • Technology Companies

Don't miss it

King Island connectivity upgrade to include 110km radio link across Bass Strait
Internet Security

King Island connectivity upgrade to include 110km radio link across Bass Strait

April 23, 2021
IoT in Telecommunications: Challenges, Opportunities, Benefits & The Future
Data Science

IoT in Telecommunications: Challenges, Opportunities, Benefits & The Future

April 23, 2021
Your Doctor’s Assistant is AI 
Artificial Intelligence

Your Doctor’s Assistant is AI 

April 23, 2021
Machine learning model generates realistic seismic waveforms
Machine Learning

Machine learning model generates realistic seismic waveforms

April 23, 2021
Website Image Optimization Guide | Infographic
Marketing Technology

Website Image Optimization Guide | Infographic

April 23, 2021
Malware and ransomware gangs have found this new way to cover their tracks
Internet Security

Malware and ransomware gangs have found this new way to cover their tracks

April 23, 2021
NikolaNews

NikolaNews.com is an online News Portal which aims to share news about blockchain, AI, Big Data, and Data Privacy and more!

What’s New Here?

  • King Island connectivity upgrade to include 110km radio link across Bass Strait April 23, 2021
  • IoT in Telecommunications: Challenges, Opportunities, Benefits & The Future April 23, 2021
  • Your Doctor’s Assistant is AI  April 23, 2021
  • Machine learning model generates realistic seismic waveforms April 23, 2021

Subscribe to get more!

© 2019 NikolaNews.com - Global Tech Updates

No Result
View All Result
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News

© 2019 NikolaNews.com - Global Tech Updates