At a recent DC Data Engineering Meetup, a community group Silectis created and sponsors, we had the pleasure of having Tim Tutt, CEO of Night Shift Development as our guest speaker. Through Tim’s presentation about using data engineering to enable data analytics through the masses, our teams started to notice how complementary our products are. We spoke the same language and knew our fellow data practitioners do, too. We decided to do another event together, this time a co-hosted webinar all about telling a compelling story in a product demo. Yesterday, Drew Hayes, COO of Night Shift Developme ..read more

Should you build your own tooling, take a “best of breed” approach, or buy a turnkey data engineering platform? We’ve got you covered. Data Engineering Platforms: Build, Best of Breed, or Buy? Every company wants to be data-driven. Modern organizations that thrive based on data have a common strength: a solid data engineering practice. How can you best enable your data engineering team to keep up with the demands of the business? Enter the Data Engineering Platform. As they’re relatively new to the market, teams may be challenged to determine whether it’s best to buy a platform off-the-shelf ..read more

Ever wondered what it’s like to work at Silectis? We’re spotlighting our employees to give you a peek into our lives in and outside of work. For our first spotlight, we hear from Brendan Freehart, a true Silectis veteran who’s been with the company for almost 3 years. Brendan is a Data Engineer at Silectis, meaning he partners with our clients to help them get productive with Magpie, our data engineering platform, faster. He’ll explain more about what this work looks like below. Get to Know Brendan Brendan Freehart, Data Engineer @ Silectis What do you do at Silectis? Where were you befor ..read more

The importance of data engineering is on the rise, with organizations increasingly investing in talent and infrastructure. Here at Silectis, we are in the fortunate position of working with a wide range of enterprises across multiple industries. I caught up with a few members of the team to take note of some of the data engineering trends we anticipate seeing more of this year and beyond.  1. Scarce skills drive product evolution & unification There continues to be a serious shortage in data engineering talent. This has been driven by the relatively recent emergence of “data engineeri ..read more

Data lineage can be a tremendously useful tool for data engineering and analytics, but is often treated as an afterthought both because of the challenges in implementation and the fact that it has not been broadly available within organizations. Many practitioners have never had access to data lineage information and may not know what they are missing.  Assessing the value of lineage for your team requires defining its scope, understanding the specific benefits it delivers, and putting it in context within the broader architecture of tools used for data management. In this blog post (and ..read more

In this blog post, we’ll walk you through how data science and data engineering are complementary disciplines. We’ll also delineate a third category: data analysis. We’ll explore how both data engineering and data science should be marshaled to make better decisions. Organizations often struggle to strike the right balance between engineering, analysis, and data science skills within data teams. To identify gaps in talent, resources, or tools, it is useful to conceptualize how data engineering and data science function within an organization. Articles with titles like “Data Science vs Data Eng ..read more

We’re excited to share that Silectis has released a new suite of automated data lineage features within Magpie, the end-to-end data engineering platform. These features equip users with knowledge of where data originates, when it was published, and who it was published by. This additional context provides users the transparency and accountability necessary to trust their data and react to inevitable data quality issues. Below we’ll cover the basics of data lineage, why it is important, and how Magpie enables teams to trust their data with this important new release.  What is Data Lineage ..read more

For those using a robust analytics database, such as the Snowflake® Data Cloud, adding the power of a data engineering platform can help maximize the value you’re getting out of that database. In this demo, we’ll show you how native tools in the Magpie data engineering platform play well with Snowflake, ultimately, allowing your team to do more in a centralized data engineering environment. Data Warehouses Have Boundaries Data warehouses do what they’re meant to, they provide a high-performance environment for data analytics. This is an important piece of a modern and reliable data architectur ..read more

If you’re new to data engineering or are a practitioner of a related field, such as data science, or business intelligence, we thought it might be helpful to have a handy list of commonly used terms available for you to get up to speed. This data engineering glossary is by no means exhaustive, but should provide some foundational context and information. Advanced Analytics The process of discovering deeper insights in data than typically enabled by most business intelligence (BI) tools. Advanced analytics can be performed by employing sophisticated tools and techniques, including machine le ..read more

If you’ve been following along with Silectis over the past couple of years, you are familiar with our data engineering platform, Magpie. You’re aware of the many outcomes it puts at the fingertips of data engineers, and teams of data practitioners more largely. If you’re new around here, not to worry. We can catch you up quickly. We are excited to share our brand new explainer video with you! Learn about Magpie in 1 minute and 28 seconds Magpie Data Engineering Platform from Silectis on Vimeo. A Unified Data Engineering Solution Your journey with Magpie starts with a seamless integration with ..read more