Jobs in the field of Data Engineering are in great demand. But how do you get a job like this? 1) Set your personal Goals The first thing you have to do is set personal goals. What do you actually want to achieve? Where do you see yourself in the next few years? What exactly do you want to do? What kind of company do you want to work for? Where and how do you want to live? Do you possibly want to go abroad? These are all questions you have to ask yourself first and then define your goals. 2) Research Job postings Afterwards you should research job advertisements. Go to the best search platf ..read more

You want to become a Data Engineer, but don't know how to set up a data engineering project? I will show you! Do not make this mistake! First of all you should not make the mistake that unfortunately many people make! Often people want to build the whole thing from the beginning. They say: "Okay I need to do a project. I need to make a big thing. I don't even know what data and what tools I want to use." But they want to do the full chain right away. Start small! I always say start small! Start by selecting a few data that make sense and that interest you. And then start with one tool. Then ..read more

What is the difference between data science and data engineering? Data Science Data Scientists work with the data and are basically doing the analytics part. They take the data and create new insights. They do the analytics for instance by using Machine Learning techniques or do just simple standard math. These results are then shown in reports or user interfaces. That’s the essence of the Data Scientist job. But that's only one part of the whole thing. Data Engineering The other part, around science, is the whole engineering part - the part of data Engineers. They are responsible for buildi ..read more

When you watch the blueprint I also use in my cookbook you see the different phases: Connect, Processing Framework, Store and Buffer. At the beginning you think about where you validate the data? To make sure that the data is okay and makes sense. Do you validate the data directly on the API - before you put it in the buffer or when you take it out of the buffer and process it? Or do you validate the data where you store it? I think this is a really interesting question because it gives a lot of depth, like thinking about an operations part. Validation in the connect phase When you think ab ..read more

I get often asked: What is the best or easiest cloud platform to start with for data engineering? Here are my thoughts on this. A general overview. Market shares Amazon Web Services (AWS) is globally the biggest cloud provider (approx. 50% of the market). This goes for the US and the EU and so on, except China, because they're not in China. Microsoft Azure is number two on the market. Google Cloud Platform (GCP) comes in third place regarding market share. Job Preferences Generally, when you compare the overall tools and functionalities that are on these platforms, it's most of the time the ..read more

In one of my livestreams, a viewer asked me the question: Scala or PySpark? Which one I prefer and why, I'll answer you in this video: https://youtu.be/64iMuH2QLjE Let me know in the comments how you think about it! See you later. Andreas Check out our Data Engineering Academy to learn the Plumbing of Data Science ..read more

How to draw architecture diagrams for AWS, Google Cloud and Azure in an easy and time-saving way? Check out this video to find out! The tool is really great! https://youtu.be/hNoVd-XKbmI Let me know in the comments how you think about it! See you later. Andreas Check out our Data Engineering Academy to learn the Plumbing of Data Science ..read more

Jobs as a Junior Data Engineer are in great demand. But what do you really earn as such? Check out this video to find out:https://youtu.be/Yve3Yl37mrE Let me know in the comments how you think about it! See you later. Andreas Check out our Data Engineering Academy to learn the Plumbing of Data Science ..read more

You're an aspiring Data Engineer and don't know exactly how to proceed to best acquire all the skills you need? Then be sure to watch this video. There you will get a roadmap on how to proceed: https://youtu.be/LKhdmm4CzbU Let me know in the comments how you think about it! See you later. Andreas Check out our Data Engineering Academy to learn the Plumbing of Data Science ..read more

SQL and Python - two skills everyone talks about. But what other skills should you have as a Data Engineer? Check out this video to find out: https://youtu.be/8f1AL_BW6nQ Let me know in the comments how you think about it! See you later. Andreas Check out our Data Engineering Academy to learn the Plumbing of Data Science ..read more