Wednesday, April 14, 2021
  • Setup menu at Appearance » Menus and assign menu to Top Bar Navigation
Advertisement
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
No Result
View All Result
Home Big Data

Where is Confluent going? | ZDNet

April 3, 2019
in Big Data
Where is Confluent going? | ZDNet
586
SHARES
3.3k
VIEWS
Share on FacebookShare on Twitter

Credit: ZDnet

Maybe it was a minor miracle that the weight of a couple ZDnet bloggers didn’t sink the pier on which Kafka Summit floated yesterday, but there’s little question that Kafka technology is very much afloat. Confluent’s most recent annual Kafka survey, published last June, found over 90 percent of survey respondents deemed Kafka as mission-critical to their data infrastructure, and that queries on Stack Overflow grew over 50 percent during the year.

You might also like

Apache Software Foundation retires slew of Hadoop-related projects

Weaviate is an open-source search engine powered by ML, vectors, graphs, and GraphQL

MinIO simplifies onramps to do-it-yourself hybrid cloud object storage

A couple years back, we looked at how Kafka emerged as the big data firehose. Fast forward to the present, and now, even AWS has gotten into the act, introducing its own managed Kafka service to go up against its own Kinesis. As with its recent introduction of DocumentDB and managed Kubernetes services, AWS is not sentimental about protecting its own proprietary offerings when enough customers demand open source. Likewise, Google Cloud, which offers the Dataflow service for building distributed data pipelines, also has a partnership with Confluent for hosting its managed Kafka service.

Big on Data colleague Andrew Brust covered the latest Confluent 5.2 platform release yesterday, showing the latest release adding a number of developer goodies like making C++, Python, Go, and .NET the same first-class citizen status long enjoyed by Java.

So the question is, given the fact that Kafka has become the de facto standard engine for distributed big data messaging, where does Confluent go from here?

Taking note of its popularity, Confluent joined other open source providers in erecting its own licensing walls last year to keep the cloud guys from making money on its own IP. Yes, there is the usual free developer edition, where you can mount the full Confluent stack for a single broker in your sandbox.  Beyond that, there is the Confluent Community license that allows the typical open source privileges – except that you cannot turn around and offer your own SaaS cloud service. That’s for features like KSQL, connectors, schema registry, and REST proxy. And then there is the proprietary Confluent Enterprise license for features like the management console.

Kafka’s success has propelled Confluent into unicorn territory, as Big on Data colleague George Anadiotis reported last January. With a fresh $125 million financing round, the company’s valuation has gotten to a crazy $2.5 billion. That prompts comparisons with Databricks, which itself received another $250 million infusion the following month to bring its valuation to $2.75 billion.

Both companies seem to be following similar trajectories. They’ve both created popular open source technologies that sit in the middle of the stack. But they’ve thrived for different reasons. Of the two, Kafka is clearly the less glamorous – it provides the underlying plumbing for enabling the types of distributed data pipelines that are used for building massively scaled real-time streaming applications. When Kafka works, you don’t see it, but you see the dashboards that work atop it, or you see the models that are consuming all that data. Conversely, Spark gets more of the spotlight, as it includes the compute engines and the vast library of frameworks and algorithms that generate the analytics. While Spark is not the front end, it’s a lot closer to it than Kafka.

As to business strategies, Databricks has followed more of the classic open core model, with the elements outside the runtime being outside the bounds of the Apache project, while Confluent, as noted above, has taken a more complex approach.

Reflecting that more of the spotlight shines on Spark, it has drawn far more competition. Spark’s IOPS-intensive nature means that it is not always the best tool for the job — especially for compute-intensive deep learning models where Spark is having to adapt. There are frameworks like H2O that work with, but also independently of Spark. Likewise, the machine learning services offered by cloud providers typically bypass Spark. And while the Apache Spark project has worked to improve the performance of R and Python programming, many users from the Anaconda and CRAN communities may use other execution engines for scaling their models.

By contrast, Kafka, sitting outside the limelight, has drawn far more modest competition. Sure, the Hadoop folks have either moved message brokering directly into the cluster (MapR Streams) or provide utilities that compete with Confluent (hello Hortonworks Dataflow), but their attention these days is focused more on making their platforms cloud-native, rather than trying to compete with Kafka. Instead, the real competition is with the cloud providers who are making their own accommodation.

Or maybe coopetition is the more suitable term. Confluent runs its cloud on AWS and Google Cloud (we’re waiting for the time when they go live on Azure), but with Google, the relationship is more formal. We’d like to GCP OEM the Confluent technology to make it a fully supported service alongside Dataflow in the same way that Azure Databricks is now a formal part of the Azure portfolio. While we don’t expect any Azure Databricks-like announcements to come from Google Cloud and Confluent next week at NEXT, we’ll be curious to see what they have up their sleeves.

Credit: ZDnet

Previous Post

Machine learning models add teeth to Google’s flood forecasting efforts

Next Post

WordPress iOS App Bug Leaked Secret Access Tokens to Third-Party Sites

Related Posts

Apache Software Foundation retires slew of Hadoop-related projects
Big Data

Apache Software Foundation retires slew of Hadoop-related projects

April 14, 2021
Weaviate is an open-source search engine powered by ML, vectors, graphs, and GraphQL
Big Data

Weaviate is an open-source search engine powered by ML, vectors, graphs, and GraphQL

April 8, 2021
MinIO simplifies onramps to do-it-yourself hybrid cloud object storage
Big Data

MinIO simplifies onramps to do-it-yourself hybrid cloud object storage

April 7, 2021
Trifacta goes all in on the cloud
Big Data

Trifacta goes all in on the cloud

April 6, 2021
Cloudera Data Platform hits Google Cloud
Big Data

Cloudera Data Platform hits Google Cloud

March 31, 2021
Next Post
WordPress iOS App Bug Leaked Secret Access Tokens to Third-Party Sites

WordPress iOS App Bug Leaked Secret Access Tokens to Third-Party Sites

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recommended

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

January 6, 2019
Microsoft, Google Use Artificial Intelligence to Fight Hackers

Microsoft, Google Use Artificial Intelligence to Fight Hackers

January 6, 2019

Categories

  • Artificial Intelligence
  • Big Data
  • Blockchain
  • Crypto News
  • Data Science
  • Digital Marketing
  • Internet Privacy
  • Internet Security
  • Learn to Code
  • Machine Learning
  • Marketing Technology
  • Neural Networks
  • Technology Companies

Don't miss it

Five Top Quality APIs
Learn to Code

Five Top Quality APIs

April 14, 2021
Cybersecurity: Victims are spotting cyber attacks much more quickly – but there’s a catch
Internet Security

Cybersecurity: Victims are spotting cyber attacks much more quickly – but there’s a catch

April 14, 2021
Detecting the “Next” SolarWinds-Style Cyber Attack
Internet Privacy

Detecting the “Next” SolarWinds-Style Cyber Attack

April 14, 2021
Weekly NFT roundup March 23-29: Circle, Klaytn, and more
Blockchain

Weekly NFT roundup April 7–13: Christie’s, Triller, and more

April 14, 2021
Machine learning can help keep the global supply chain moving
Machine Learning

Machine learning can help keep the global supply chain moving

April 14, 2021
Why I Think That Avengers: Age of Ultron is One of the Best Sci-Fi Movies About A.I | by Brighton Nkomo | Apr, 2021
Neural Networks

Why I Think That Avengers: Age of Ultron is One of the Best Sci-Fi Movies About A.I | by Brighton Nkomo | Apr, 2021

April 14, 2021
NikolaNews

NikolaNews.com is an online News Portal which aims to share news about blockchain, AI, Big Data, and Data Privacy and more!

What’s New Here?

  • Five Top Quality APIs April 14, 2021
  • Cybersecurity: Victims are spotting cyber attacks much more quickly – but there’s a catch April 14, 2021
  • Detecting the “Next” SolarWinds-Style Cyber Attack April 14, 2021
  • Weekly NFT roundup April 7–13: Christie’s, Triller, and more April 14, 2021

Subscribe to get more!

© 2019 NikolaNews.com - Global Tech Updates

No Result
View All Result
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News

© 2019 NikolaNews.com - Global Tech Updates