C compilers do a great job of converting C to assembly language (optionally) and then to primitive CPU instructions. However, some CPU instructions are too specialized or too complex to map to the C language. For example, Postgres has used test and set assembly language CPU instructions for decades to provide good performance for high concurrency workloads. We have such CPU instructions for: i386 x86_64/AMD64 ARM S/390 Sparc, Sparc v7 PowerPC MIPS HP PA-RISC Since 2014, Postgres has supported atomics, which provides test and set, compare and exchange, and atomic addition. We also use compile ..read more

TIMESTAMPs are very precise and flexible in Postgres, but sometimes users want to do index lookups of TIMESTAMP values with less precision, i.e., by date. To illustrate this, let's create a sample table with 100k TIMESTAMP values: CREATE TEMPORARY TABLE date_test (event_time TIMESTAMP WITH TIME ZONE); INSERT INTO date_test SELECT ( SELECT '2023-03-01 00:00:00'::timestamptz + (floor(random() * (extract(EPOCH FROM '2023-04-01'::timestamptz) - extract(EPOCH FROM '2023-03-01'::timestamptz)) + b * 0)::integer || 'seconds')::interval ) FROM gen ..read more

The Postgres mailing lists are full of practical discussions, but two years ago there was a 77-email thread titled "The tragedy of SQL" that questioned the utility of the SQL query language; t is worth a review. While opening with "A fun philosophical discussion," it states: The world's economic output would be substantially higher (5%?) if our industry had settled on almost anything other than SQL for relational databases. It disparages object-relational mappers and suggests Datalog as an alternative query language ..read more

When a single query is run outside of a transaction block, it is clear how transaction visibility should behave — only transactions committed before the query started should be visible. Changes made by other sessions during the query, even if committed during the query, should not be visible. Slide 11 of my MVCC Unmasked presentation illustrates this. For queries run in transaction blocks, the ideal behavior is less clear, so Postgres supports three options for controlling transaction block behavior. The default, READ COMMITTED, causes each new query in a transaction block to get a new visibil ..read more

LATERAL is a powerful SQL feature that allows virtual tables to be created in FROM clauses that reference real or virtual tables that appeared previously in the same FROM clause. Here is a simple example: CREATE TABLE test (x) AS SELECT generate_series(1,3); SELECT * FROM test AS test1 (x) JOIN LATERAL (SELECT test1.x + 1) AS test2 (x) ON (true); xx ---+--- 12 23 34 ..read more

My common table expression (CTE) talk shows how CTEs can be used to combine individual queries into a single CTE query. This explores the downsides of such combinations, and when combining is unreasonable. Its conclusions are: Queries combined into a CTE behave as though they were in a REPEATABLE READ transaction block Changes made to table rows are not visible to other parts of the CTE The most glowing report of CTE combining is from this saying, "I have yet to regret replacing a transaction with a CTE over the past decade ..read more

SQL is a powerful declarative language — you tell it what you want, and the optimizer determines the fastest way to produce the desired result. Of course, the language is not perfect. I already blogged about the non-sequential order in which SELECT clauses are executed. The unusual behavior I would like to explain here concerns UNION, INTERSECT, and EXCEPT. These three clauses allow queries to be connected with the result being either combined, intersected, or subtracted, respectively. In most aspects of the SQL language, you add keywords to cause additional processing, e.g., add UNIQUE to a c ..read more

I wrote Explaining the Postgres Query Optimizer in 2011, and though I have presented it 22 times, it has changed little over the years. (I created a followup presentation this year.) Periodically, I check my slides to make sure they match the output of the current Postgres version. I did this recently using the file associated with the presentation and found that the EXPLAIN plans for the final LIMIT queries were different. I updated my slides, and then went to find the optimizer improvement responsible for this change. After applying a patch to enable compiling of older branches, I found the ..read more

Having 61 presentations on my website, I am always looking for efficient ways to create them. For presentations that focus on SQL features, I have found the most efficient method is to first write the SQL queries in the order I want them to appear in my presentation. I then run the queries through psql, and write its output to a text file. Next, I import the file into my document editor, LyX. I then add slide headings, markup, colors, and supporting text. Sometimes there are too many SQL queries to efficiently manage in a single file so I use multiple files and name them with numeric prefixes ..read more

While Postgres excels at supporting many workloads, including modern ones, there are still some workloads it cannot handle. Two areas that Postgres is currently unable to handle involve large data volumes: Data sets which are so large that they cannot be efficiently processed by a single server Write-heavy workloads that cannot be efficiently written by a single server Postgres is a great general-purpose database, and it might be fine if Postgres were deemed unacceptable for such workloads, especially if supporting such workloads required a major redesign of Postgres. However, could Postgres ..read more