A Tale of Technical Debt Introduction Early on in CUDA’s development, I made a change to the driver API that moved the context from being a parameter to every function, to being stored in a TLS (thread local storage) slot. The interface change resulted in the context parameter being removed from almost every CUDA entry point. For example, the function to create a CUDA array changed from: CUresult cuArrayCreate( ctx, array, descriptor ); to: CUresult cuArrayCreate( array, descriptor ); and the context was taken from a TLS slot. TLS slots, described here for the GNU toolchain and here for Micros ..read more

or: How I Learned To Stop Worrying And Deprecate cudaGetLastError() Every API designer has struggled with the question of how best to propagate errors to their callers, since before the term “API” was invented. Even decades ago (say 30+ years), interface designers knew to separate the error return from the payload, in functions that return other results to their caller. Since it is sometimes useful to know what not to do: My favorite example of an antipattern in this department is in the venerable atoi() function in the C runtime, which converts a string to an integer: int atoi (const char ..read more

Fatalism Can Be Useful Sometimes Introduction A wise man once told me: “Scientists build in order to learn. Engineers learn in order to build.” One of the most exciting experiences for an engineer is realizing that new discoveries or technologies enable new possibilities, but part of an engineer’s job also is to save work by eliminating possibilities. For example, it sometimes comes in handy to know that nothing can travel faster than the speed of light in a vacuum. I call this principle “The Utility of Futility,” and it is an ethos that more software engineers would do well to embrace. As a s ..read more

When the time came for us to gather requirements and start designing CUDA 2.0 (c. 2008), I was surprised to hear from our product manager that the key Next Biggest Thing in CUDA was… drum roll… 3D textures[1]! The reason this priority was surprising is because adding texturing in the first place had been somewhat controversial. Why have graphics features in a general purpose computing ecosystem[2]? Early versions of CUDA hardware did not have caches to mitigate the overhead of uncoalesced memory traffic, though, so applications that really needed misaligned reads had no choice but to use the t ..read more

This question on StackExchange was put on hold as primarily opinion-based: “…answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.” The content of StackExchange is usually high quality, but in this case, while the design decision was based on opinion, the answer to the question needn’t be… you just need to ask the people who know! And the inimitable talonmies, who is poised to crack 30k on StackExchange’s points-based reputation system, compounded the problem by saying that CUdeviceptr is a handle to a device memory al ..read more

As the industry absorbs the implications of the Google v. Oracle ruling, which essentially put proprietary APIs on the same footing as standardized ones, understanding the differences between standardized and proprietary APIs will become more important. The differences between closed and open source software have been well-plumbed; the differences between proprietary and standardized APIs, less so. I’m a veteran of two API competitions that shed light on the tensions between the philosophies of proprietary and standardized APIs and the way they intersect with market realities: OpenGL/Direct3D ..read more

As a professional software developer, I am not accustomed to court cases, let alone Supreme Court cases, addressing the tradecraft of my profession; and when they do, I’m usually disappointed in the outcome. But this week, the Supreme Court issued a decision that directly addresses, not just computer programming in general, but API design, which I have worked on for decades, and as it happens, I agree with the Court’s ruling and look forward to the wide-reaching impact it will have on my profession[1]. This ruling will reverberate throughout techdom for many years to come: The Supreme Court ru ..read more

Wow. Has it really been two months since my last post? The calendar says yes. I’ve been putting finishing touches on the manuscript, and as part of that exercise, I did a line count to see how much code we can say accompanies the book. I was surprised to see it’s slightly more than 25,000 lines! As a reminder, the source code resides in this Git repository. The code frequently offers multiple versions of the same algorithm, but with this book, that’s the whole point. In some cases (e.g. the Reduction chapter), readers are walked through different versions of functionally-identical code, a ..read more