Data and AI Engineering Maturity - Fix our problems before we hit the buffers
Endjin Blog
by
7h ago
Something has happened over the last 10 years: business has started to view "data" as the engine of change. In the 1980s and 1990s "information technology" and "the micro on every desktop" held that position. Over 10-15 years "the IT department" evolved methodologies for providing and managing those services more efficiently (with varying degrees of success!). Today, we have automated, virtualized infrastructure (often provided by mega-vendors in "the Cloud"). Instead of provisioning hardware, we have Bring-Your-Own-Device policies (from laptops to smartphones). Working-from-home has taken fac ..read more
Visit website
SQLbits 2024 - The Best Bits
Endjin Blog
by
7h ago
For the third year in a row I attended SQLbits - the largest expert led data conference in Europe. This time the journey took me from Scotland to the exhibition centre in Farnborough in Hampshire, South West of London. Farnborough has a long association with aviation, and hosts a world renowned international air show every year. SQLbits took advantage of the massive exhibition space which is on offer at Farnborough. As you would expect, the theme for the conference this year was aviation, and it was embraced throughout the conference by the vendors, in the various sessions and in the after pa ..read more
Visit website
The Heart of Reactive Extensions for .NET
Endjin Blog
by
7h ago
Reactive Extensions for .NET (Rx.NET) is profoundly useful, and developers often end up loving. Why is that? Endjin Technical Fellow Ian Griffiths explains how Rx.NET's deep roots in mathematical foundations make it both powerful and beautiful. To learn more about Rx.NET, check out the free book Introduction to Rx.NET 2nd Edition (2024). Ian Griffiths Hi, I'm Ian Griffiths, Technical Fellow at endjin. Back at the start of 2023, endjin took over as primary maintainers of the Reactive Extensions for .NET, usually known as Rx.NET, an open source .NET project for event driven programming. In this ..read more
Visit website
How to Build Navigation into Power BI
Endjin Blog
by
2w ago
At endjin we have been working in the Power BI space for a number of years. We're continually looking to customize our report design through themes, branding, and data visualization to give the UI a unique look and feel. In this post we are going to overlay UX practices and build on them and make reporting look more considered and intentional. We need to build stories and journeys into our reporting because data has a story. It’s about exposing and surfacing these topics and meaningful outcomes from the data we are presenting. One way to add formal UX practices is by integrating navigation. T ..read more
Visit website
How to access multi-select choice column choice labels from Azure Synapse Link for Dataverse with PySpark or SQL
Endjin Blog
by
2w ago
When you export a table from Dataverse into Synapse using the Azure Synapse Link for Dataverse, the multi-select choice columns in your linked tables will display a list of numerical choice values. This isn't very meaningful to the end-user, what we really want to display in our choice columns is the associated text choice labels. This blog post explains how to access the multi-select choice column choice labels from Azure Synapse Link for Dataverse using PySpark, Spark SQL or T-SQL through SQL Serverless. Multi-select choice columns in Dataverse A multi-select choice column, is a type of colu ..read more
Visit website
Introduction to Python Logging in Synapse Notebooks
Endjin Blog
by
2w ago
Notebooks are an incredibly flexible and useful part of Synapse (and other similar data platforms). They are great for getting started with exploring new datasets, as well as providing maximum flexibility when processing and analyzing data. As a result, it's quite common for a notebook to evolve from that initial exploratory code to be part of a productionised data processing pipeline. During the process of productionisation, it's necessary to ensure that errors in production can be effectively diagnosed. Depending on the complexity of the notebook, this might be simple - if it fails, you can ..read more
Visit website
No-code/Low-code is software DIY - how do you avoid DIY disaster?
Endjin Blog
by
3w ago
TLDR; Much like how DIY tools empower individuals to undertake home improvements without professional intervention, No-code/Low-code platforms aim to democratise the process of building software, empowering users without the need for extensive coding expertise. They can be incredibly powerful and may be a perfect fit for your needs, but before you dive in to embrace these platforms fully it's important to understand the broader context of software engineering. For example, critical factors such as testing, security and regulations can often be an afterthought in this type of tooling, so it's i ..read more
Visit website
.NET JsonElement Error Handling
Endjin Blog
by
1M ago
At endjin, we are fans of System.Text.Json in the .NET runtime libraries. In particular, the JsonDocument and JsonElement types, are able to provide astonishingly high performance. Not quite as fast as the streaming Utf8JsonReader, but surprisingly close given how much easier it is to use. There's one small trap for the unwary, however: if your inputs aren't in exactly the form you expect, there are two subtly different ways in which it can tell you. This means that your code might not be as robust in the face of malformed input as you thought. For more information about how to use these APIs ..read more
Visit website
SQLBits 2024 - Do those numbers look right? How to ensure quality, and avoid inaccuracies in your data insights.
Endjin Blog
by
2M ago
Join James Broome at the world's leading data conference, SQLBits, on the free-to-attend Community Saturday, 23rd March to deep dive into data quality. The need to validate business rules and security boundaries within a data solution is important, as well as the need for ensuring that quality doesn't regress over time as it evolves. Data insights are useless or even dangerous if they can't be trusted, and in this session, James will explain how and why they should treated just like any other software project with respect to testing - by building automated quality gates into the end to end dev ..read more
Visit website
2x Microsoft MVPs renewed for 2023
Endjin Blog
by
2M ago
Endjin Technical Fellow, Ian Griffiths, has been awarded a Microsoft Most Valuable Professional award for 2023 in the Developer Technologies category, making this his 16th year of being an MVP! Endjin co-founder, Howard van Rooijen, has been renewed as a Microsoft Most Valuable Professional award for 2023; his 8th year as a MVP. Together, they are maintainers for the Reactive Extensions for .NET and Reaqtor, Howard curates the free Azure Weekly newsletter, which he has been publishing since 2014, and Ian is the author of the Programming C# book series for O'Reilly Media, and speaks at many use ..read more
Visit website

Follow Endjin Blog on FeedSpot

Continue with Google
Continue with Apple
OR