This post describes our approach to implementing a disk-backed hashmap in Rust for indexing high-cardinality predicates in our diagnostics product, Seq. If you've ever wondered how a hashmap works, or what makes a disk-backed datastructure different from an in-memory one, you might find it interesting. Seq is a database for diagnostic events. It's designed to store large volumes of immutable event records with limited ingestion overhead. Records are initially ingested into unordered write-ahead-logs based on their timestamp. After some grace period has passed, multiple logs are sorted and coal ..read more

TL;DR: With Seq 2024.3, Seq gains user-defined high-cardinality indexes, its first new index type since the introduction of signal indexes back in 2018! If you're eager to skip to the binaries and dig around for yourself, you can get MSIs at https://datalust.co/download, or pull datalust/seq:latest from Docker Hub. Debugging is unpredictable. Because of this, log analysis is unpredictable too. Unlike a typical database system, executing the very same query millions of times in a day, Seq might be called on to run dozens of new, novel queries in the course of a single debugging session. Debug ..read more

Now in preview, Seq 2024.3 introduces a new index type on event properties, known as expression indexes. Under the right circumstances, expression indexes reduce search time from minutes to milliseconds. Expression indexes are created using the context menu on an event property. A property like OrderId is a strong candidate for an index because different events are likely to have different OrderIds. Once the index has been built, searches of the form OrderId = <value> will use the index. If the indexed property is well suited to indexing it can be possible to search terabytes in less th ..read more

TL;DR: for the latest UX and performance improvements, pull datalust/seq:latest from Docker Hub, or grab the MSI from datalust.co/download. Seq 2024.2 is ready to go! Few releases come together as smoothly as this one has. We allocated some time after the major tracing-related changes in 2024.1 to incorporate feedback, and to work on polish and performance. First, the new trace view has been significantly refined. If you're using Seq to capture distributed, hierarchical trace data from OpenTelemetry or SerilogTracing, you'll appreciate the quality-of-life improvements here. Next, searches tha ..read more

Introducing the new Seq.App.Mail.* packages for SMTP and Microsoft365 email. A lot has changed since the first email notification plugin was written for Seq! SMTP is alive and well, but .NET's SmtpClient is all but dead and gone. Services like Amazon SES and Microsoft 365 provide alternative email APIs alongside SMTP, with advantages for some, but with using different protocols and message formats. The new Seq.App.Mail.* packages are our answer to these changes. First, they're wholly modern codebases using MailKit on .NET 6+. Second, perhaps more importantly, they build on a general-purpose Se ..read more

Seq 2024.1 introduced a change to the default display of nested event properties. Previous versions of Seq rendered collapsed nested structures that could be expanded to access a context menu on the nested properties. The newer scheme flattens nested properties into dotted names and makes the regular property context menu available for all nested properties. Before After Flattening nested properties has the advantage of making the event property actions menu easily available for all properties. It also works well with the OpenTelemetry data model, which includes a lot of nested data. If you p ..read more

Distributed tracing makes Seq even better for monitoring and troubleshooting the behavior and performance of complex distributed systems. The emerging standard for logging and tracing is OpenTelemetry, which Seq supports via the OpenTelemetry Protocol (OTLP). If you are using Serilog another option you can consider is SerilogTracing. SerilogTracing is a new project that adds distributed, hierarchical tracing to applications that use Serilog. Traces can be created programmatically or generated by .NET components including HttpClient and ASP.NET Core. Traces can be sent to Seq or to any OTLP com ..read more

NET Aspire is a toolkit for development and deployment of microservice applications. It is a local orchestrator with support for deployment to production-grade orchestrators like Azure Container Apps or Kubernetes. One great thing about Aspire is the built-in local development dashboard. Aspire's built-in local development dashboard It shows the resources that are included in the solution as well as the observability signals generated by the resources. There are pages for structured logs, traces and metrics with basic search and analysis features. With no additional effort developers get a da ..read more

OpenTelemetry has emerged as an important standard for generating observability data. It provides a consistent model across all environments and reduces vendor lock-in. As such, we think it is important for Seq to support OpenTelemetry clients in addition to the other ingestion options that Seq provides. 2024.1 extends Seq's OpenTelemetry support to include traces as well as logs. Distributed tracing makes Seq even better for monitoring and troubleshooting the behavior and performance of complex distributed systems. A distributed trace including spans and logs It is now possible to add the Ope ..read more

Seq uses the OpenTelemetry service.name attribute, or Serilog-style Application property, to determine which color should be used for the spans belonging to a service. This sometimes works brilliantly - we put some effort into the palette! - but it can also fall short, if you happen to be unlucky and hit collisions. Take for example the colours for the two services in this trace: The birthday paradox takes no prisoners: the names "weather" and "Example.WeatherService" end up hashing (mod the array length) to adjacent indices in the color array, and both look roughly the same shade of purple ..read more