A few weeks ago, the owners of Redis changed its licensing from BSD to a more limited source available license. As far as I know this was done in order to prevent service providers from adding their own closed source, proprietary changes to their Redis service offerings. In principle I like this change. My Faktory project uses the AGPL license for the same purpose: Faktory is open source and anyone who offers Faktory as a service with proprietary changes should have to open source those changes too ..read more

Ruby’s JSON library allows you to convert Ruby datatypes into a JSON document, as long as those types are native to JSON: String, bool, int, float, nil, Hash and Array. Everything else converts to a String by default; if you use any other core datatypes, they will not survive a JSON.generate/JSON.parse round trip. Here I pass a Range, Symbol and Time; notice how the end result is three Strings instead of the actual types ..read more

2024-02-04 This article was originally published on DanSvetlov.me and is republished here with permission of the author. This article is relevant to Sidekiq v7. Sidekiq is one of the most ubiquitous1 Ruby background job processors out there. To anybody who has worked with Ruby on and off Rails, it needs no introduction. Sidekiq has a 10+ year track record of being an efficient, battle-tested and simple-to-use solution for offloading the execution of application logic into the background ..read more

For about 15 years, Redis has been the dominant choice for background job infrastructure in Ruby. First used by Resque, Sidekiq adopted it also as a pragmatic choice which had already gained mass acceptance within the community. And so for the last 15 years, we’ve used Redis widely across the industry for caching, jobs and many different data munging tasks. However the only constant is change. Dragonfly brings competition and new capabilities ..read more

Recently I decided to add support for Kubernetes HTTP health checks to Sidekiq Enterprise. This means, within each Sidekiq worker process, we need to implement an HTTP server which listens on port X and simply returns 200/OK if the process is alive. Notice we have really basic requirements here: no need for serving files or arbitrary dynamic content. We only respond to “/” so there’s no need for routes, paths or query parameter handling ..read more

Recently I made some minor changes to Sidekiq Enterprise 7.1 in order to greatly increase the scalability of the Rate Limiter feature. Redis has a little-known feature which allows you to safely use MULTI transactions with multiple keys in a cluster. This was a real learning experience for me so I thought other people might find this interesting ..read more

I’m proud to announce, after nearly a year of work, Sidekiq 7.0 is now available. Sidekiq is the most popular background job system for Ruby, used by thousands of companies around the world. This release is our biggest, most splendiforous release ever! What’s New? Metrics One thing I know: everybody loves big beautiful graphs! Sidekiq 7.0 has a major new feature for tracking and visualizing job execution times. Big thanks to @adamlogic of Rails Autoscale for implementing the new graphs and I hope you’ll agree he did a fantastic job ..read more

Sidekiq 7.0 can be embedded within another process so you do not need to run a separate Sidekiq process. This is called embedding. Why? Why would you want to do this? Embedding can make for a simpler deployment: you deploy one larger process instead of two separate processes, now you only need to monitor and manage one process. Embedding also requires less total memory since both subsystems will share most of the Ruby data structures and code in memory ..read more

Sidekiq 7.0 introduces a new subsystem for gathering job execution data along with a new Metrics tab within the Web UI to visualize this data. The data is designed to help you answer a few questions: Which types of jobs executed? Which types of jobs took the most amount of time to execute? How has the performance of Job X changed recently? When did any deploys occur during this time? The graphs only show data for the last hour ..read more

Yesterday I released the first public beta of Sidekiq 7.0. This is a major new release with several huge new features, several features removed, APIs refactored and updated requirements. Job Performance Metrics! Embedded mode! Capsules! Strict args! But with all these awesome new features, we refactored a LOT of internal APIs and likely broke a bunch of things. In other words, we really need people to test it and open issues in order to get a smooth 7 ..read more