In our first blog, “IoT Overdrive Part 1: Compute Cluster Running Apache Cassandra® and Apache Kafka®”, we explored the project’s inception and Version 1.0. Now, let’s dive into the evolution of the project via a walkthrough of Version 1.4 and our plans with Version 2.0. Some of our initial challenges that need to be addressed include: Insufficient hardware Unreliable WiFi Finding a way to power all the new and existing hardware without as many cables Automating the setup of the compute nodes Upgrade: Version 1.4 Version 1.4 served as a steppingstone to Version 2.0, codenamed Ericht, which n ..read more

NetApp is pleased to announce the immediate general availability of Instaclustr for Apache ZooKeeper® 3.7.2 and 3.8.4 on the Instaclustr Managed Platform. This update is part of our ongoing commitment to provide our users with the latest and most efficient tools for managing their distributed systems, with both releases containing important security fixes.  In the coming weeks we will be transitioning the lifecycle states of the older versions to more restrictive states. For details on the timelines of the lifecycle state changes please refer to this page. For details on the meaning of ea ..read more

Camels drinking from a leaking pipe (Source: Wikimedia)  A few years ago I built a “zero-code” data processing pipeline with Apache Kafka® and related open source technologies like OpenSearch®, PostgreSQL®, and Apache Superset. Here’s the first blog in that series (there were 10 in all; the links to each can be found at the bottom of this page).  It was designed to use an open source Kafka REST source connector to ingest tidal data from the NOAA REST API into Kafka topics for downstream parts of the pipeline to process. I used this REST connector at the time, but when I downloaded i ..read more

Apache Spark is a critical part of AI data engineering pipelines, but it requires expertise to architect, deploy, and manage efficiently in a scalable container environment. Misconfiguration of security settings can expose applications to potential threats.   To solve this, NetApp—a global, cloud-led, data-centric software company—collaborated with Red Hat, the world’s leading provider of open source solutions, to launch a new offering: NetApp Supported Apache Spark on Red Hat OpenShift.  This collaboration aims to provide businesses with a robust, enterprise-grade solution that acce ..read more

Welcome to Part 3 in our series about user interfaces (UI) to help you visualize and monitor data within your Instaclustr for Apache Kafka® cluster! In our previous posts, we explored akhq.io (in Part 1) and UI for Apache Kafka (in Part 2), as well as how to configure the console and connect them to an Instaclustr for Apache Kafka cluster. Now in Part 3, we will follow a similar pattern but with an analysis on a third console we found interesting: Kafdrop. What Is Kafdrop? Kafdrop is a web UI for viewing Kafka topics and browsing consumer groups. The tool displays information such as brokers ..read more

The Instaclustr Managed Platform now supports the zero-downtime scaling-up of local storage (also known as instance store or ephemeral storage) nodes running Apache Cassandra on AWS, Azure and GCP! Enhanced Vertical Scaling  NetApp has released an extension to the Cluster Scaling feature on the Instaclustr Managed Platform, which now supports scaling up the local storage of the Cassandra nodes. This enhancement builds upon the existing network block storage scaling option, offering greater flexibility and control over cluster configurations. Customers can now easily scale up their Cassand ..read more

Overview  Originally developed and open sourced by Uber, Cadence® is a workflow engine that greatly simplifies microservices orchestration for executing complex business processes at scale.  Instaclustr has offered Cadence on the Instaclustr Managed Platform for more than 12 months now. Having made open source contributions to Cadence to allow it to work with Apache Cassandra® 4, we were keen to test that performance with the new features of Cassandra 4.1 with Cadence.  Paxos is the consensus protocol used in Cassandra to implement lightweight transactions (LWT) that can handle ..read more

A radio-controlled saltwater crocodile?  No, a croc with telemetry – sensors + radio communication!   (Source: Wikimedia)  1. Introduction It’s been 5 years since I looked at observability and distributed tracing for Apache Kafka® with open source OpenTracing and Jaeger in this blog.    So, I decided it was worth taking another look at tracing Apache Kafka, particularly after discovering that OpenTracing and OpenCensus have merged to form OpenTelemetry, which handles everything observable: traces, metrics, and logs (although I’m only interested in trace ..read more

Apache Cassandra® version 5.0 Beta 1.0 is now available in public preview on the Instaclustr Managed Platform!   This follows on the heels of the project release of Cassandra version 5.0 Beta 1.0. Instaclustr is proud to be the first managed service platform to release this version for deployment on all major cloud providers or on-premises.   This release is designed to allow anyone to easily undertake any application-specific testing of Apache Cassandra 5.0 (Beta 1.0) in preparation for the forthcoming GA release of Apache Cassandra 5.0.  Apache Cassandra 5.0  The last maj ..read more

Check out all the new features and updates added to our platform over the last few months!      As always, if you have specific feature requests or improvement ideas you’d love to see, please get in touch with us.  Major Announcements:  Apache Cassandra®  AWS PrivateLink for Apache Cassandra® on the Instaclustr Managed Platform  NetApp is excited to announce that the Instaclustr Platform now supports the use of AWS PrivateLink to connect to Apache Cassandra clusters running on the platform. This new feature provides customers running on AWS with a more secu ..read more