You could be saving money every month on databases costs with a smarter data retention policy. One of the primary reasons, and a huge benefit of partitioning is using it to automatically archive your data. For example, you might have a huge log table. For business purposes, you need to keep this data for 30 days. This table grows continually over time and keeping all the data makes database maintenance challenging. With time-based partitioning, you can simply archive off data older than 30 days. The nature of most relational databases means that deleting large volumes of data can be very ineff ..read more

In a PostgreSQL database, sequences provide a convenient way to generate a unique identifier, and are often used for key generation. From the community, PostgreSQL provides functions and SQL language to help manage sequence generation, but the sequences themselves are not without limitations in a multi-master environment. Snowflake sequences from pgEdge work seamlessly in a multi-master PostgreSQL cluster to remove those limitations so your data can thrive at the network edge. Why are Sequences an Issue?In a distributed multi-master database system, sequences can get complicated ..read more

We explore PostgreSQL Internals in detail - its processes, architecture, the different types of processes available, and their respective responsibilities. The post PostgreSQL Internals Part 3: Understanding Processes in PostgreSQL appeared first on Stormatics ..read more

Last Friday (April 12, 2024) happened, with a record of 270 registrations, the 8th annual PGConf.DE in Munich, Germany. There definitely was a great lineup: 1 German track, 2 English tracks and 1 sponsors track. While my awesome colleague Valeria was speaking about Navigating the Postgres community, I attended Martín Marqués talk about Postgres native incremental backups. If you’re following the -hackers mailing list, you may have noticed that it is a hot topic at the moment. PostgreSQL 17 should receive the possibility to take incremental backups with pg_basebackup. Even though I think this ..read more

The news today is that GQL:2024, the first version of the GQL standard, has been published by ISO. GQL is a new language for graph databases, like SQL is for relational databases. Here is the link to it on the ISO web site: ISO/IEC 39075:2024 This is the culmination of work that has been going on formally within ISO since 2019 and for several years more before that in preparation. (There was even a manifesto! How can you go wrong?) I suppose this doesn’t affect my usual work with PostgreSQL and SQL much. But having a new database language to work on in the same ISO working group that deals w ..read more

I won’t be able to attend Magnus’ talk at PG Day Chicago because it is scheduled at the same time as my talk, but fortunately, I attended his talk at SCaLe. There are several features I am really excited about, and I can’t wait to start using them! Event trigger on login. Yes, yes, I know, Magnus told us it’s a footgun, but I promise, I know how to use it safely! I have so many potential usages!!! Error handling in COPY. Since I started my new development with pgBadger, my biggest fear is that if just one record is somehow wrong, the whole file won’t load, and I can’t do anything with that ..read more

This post looks at some interesting new capabilities managing Partitioned Tables coming in PostgreSQL 17, expected for release Fall 2024. The current major version is 16. Current Table Partition Commands Prior to Version 17, workflow options for partition management are limited to creating, attaching, and detaching partitions. Once we’ve designed our partition structure, we couldn’t redesign it in place. This applies to all partition types, whether we’re using RANGE, LIST, or HASH. To combine multiple partitions into a single one, or to “subdivide” a single partition into multiples, we’d need ..read more

Any database engine, including PostgreSQL, requires some routine maintenance to ensure optimal performance. Especially when there are massive delete or purge operations to delete rows from a Table, the fragmentation builds up within a table. Reclaim this fragmented space from disk requires a table to be rebuilt. At HexaCluster, we had many Customers asking us […] The post Rebuilding Tables online using pg_repack in PostgreSQL appeared first on HexaCluster ..read more

When you first spin up your app, the emphasis is on getting started and getting data to your clients. But when you don’t have throughput, you are also not going to have enough concurrency to unveil bad queries. But then you have success. And success means data. More users, more interactions, more everything. Suddenly, queries that performed fine are struggling under the load, hurting performance and scalability. This is all going to mean a far worse user experience when much higher costs because of inefficient resource use. This is where mastering query rewriting techniques comes into play. By ..read more

This is the second article in a series that explores advanced strategies for scaling PostgreSQL databases in Kubernetes with the help of CloudNativePG. This article focuses on horizontal table partitioning and tablespaces and how they can be used to manage large datasets. By partitioning tables based on specific criteria and optimising storage with tablespaces, PostgreSQL users can achieve better scalability and performance in cloud-native environments, just like they could in traditional VMs or bare metal deployments ..read more