Apache Pinot grew popular among companies like Uber for its capability to serve SQL queries to thousands of external users with sub-second latency. Now the commercial open-source vendor backing Pinot, StarTree, announced that it’s expanding the database into its first internal use case, analyzing observability data. StarTree also announced launch of an automated anomaly detection application called ThirdEye, and unveiled the addition of vector support in the open source project. Apache Pinot is a distributed columnar database that was developed at LinkedIn in 2015 to serve the social media com ..read more

Anaconda created a name for itself in the data science community over the past decade by combining hundreds of the most popular Python-based statistical and machine learning packages, such as NumPy, Pandas, and SciPy, into one easy-to-use package. As the GenAI revolution spreads across the land, the Austin, Texas-based company is now looking to find stronger footing to take it to the next decade. In January, Anaconda announced a change at the top of its org chart, with longtime member of the board of directors Barry Libert taking over the CEO position and Peter Wang moving to head up the compa ..read more

The American Privacy Rights Act of 2024 (APRA), a proposed federal data privacy law that was introduced last month by Congresswomen from Washington State, is currently making its way through both halls of Congress. If approved and signed into law by President Biden–which is no guarantee–the APRA would represent a monumental shift in data privacy. The APRA is a proposed new data privacy law introduced on April 7 by U.S. Senator Maria Cantwell (D-WA) and U.S. Representative Cathy McMorris Rodgers (R-WA). If enacted, the APRA would give implement a national data privacy standard, giving American ..read more

Confluent, the data streaming pioneer, announced last month the general availability of Confluent Cloud for Apache Flink, enabling users to experience Apache Kafka and Flink as a unified, enterprise-grade platform.  This week, Confluent announced that the addition of new capabilities to the service. The managed service for Apache Flink now features AI Model Inference, which allows users to clean and process real-time streaming data for AI and ML applications. In addition, Confluent has also announced an auto-scaling cluster type for use cases such as logging and telemetry data without tim ..read more

Amazon Web Services (AWS) says business productivity will soar as a result of Amazon Q, its new generative AI service that became generally available this week. In particular, the capability to deploy semi-autonomous AI agents that can perform various tasks will be a boon for software developers and office workers, says Doug Seven, the company’s general manager and director of AI developer experience. “I feel fairly confident saying that, for customers who are leveraging Q across a number of different use cases, they can see upwards of 80% more productivity in various roles in different jobs ..read more

With its vast user base and the numerous interactions that occur daily, LinkedIn generates an enormous amount of data every day. The billions of data points fuel various applications, from rankings to search. The addition of AI features has added further complexity to the platform.  LinkedIn relies on a massive data lake architecture to handle this vast volume of data. This enables efficient access to datasets generated by the platform. However, managing and utilizing such a large architecture remains a massive infrastructure challenge.  To some extent, utilizing data pipelines has h ..read more

Businesses are eager to deploy generative AI applications, but fears over toxic content, leaks of sensitive data, and hallucinations are giving them pause. One potential solution is to deploy “guard models” alongside GenAI apps that can immediately detect and prevent this kind of behavior. That’s the approach espoused by DataRobot, which today added new AI observability capabilities to its AI Platform that are aimed at preventing large language models (LLMs) from running amok. In addition to a handful of pre-configured guard models, the DataRobot AI Platform gains new alerting and notification ..read more

The recently released Apache Hive 4.0 by Apache Software Foundation (ASF) marks a significant milestone in the progress of data lake and data warehouse technologies.  In the world of big data processing tools, Apache Hive stands out as one of the leading data warehouse tools. It has the ability to query large data sets while offering outstanding flexibility through its SQL-like query language.  Since its inception in 2010, Hive has empowered organizations around the world to perform analytics and scale their data processing capabilities. It has become a critical component in the arch ..read more

Extracting valuable insights from unstructured text is a critical application in the finance industry. However, this task often goes beyond simple data extraction and necessitates advanced reasoning capabilities. A prime example is determining the maturity date in credit agreements, which usually involves deciphering a complex directive like “The Maturity Date shall fall on the last Business Day preceding the third anniversary of the Effective Date.” This level of sophisticated reasoning poses challenges for Large Language Models (LLMs). It requires the incorporation of external knowledge, suc ..read more

In this monthly feature, we’ll keep you up-to-date on the latest career developments for individuals in the big data community. Whether it’s a promotion, new company hire, or even an accolade, we’ve got the details. Check in each month for an updated list and you may even come across someone you know, or better yet, yourself! Hamid Azzawe Hamid Azzawe, who took over as the new CEO of parallel graph database developer TigerGraph in February, was formally announced today as the new chief executive. As a technology and database veteran, Azzawe had stints at IBM, Microsoft, AWS, Meta, and Bloomber ..read more