Azure Data Engineer (ID: 4098)
Key Skills
Job Description
- Design, develop, and maintain scalable data pipelines using Microsoft Azure data services.
- Develop data processing and transformation solutions using Python.
- Build, configure, and manage data workflows using Azure Data Factory (ADF).
- Develop and optimize data processing solutions using Azure Databricks and Apache Spark.
- Integrate data from multiple sources using APIs and other data integration techniques.
- Work with Azure Databricks and Azure data services to process, transform, and prepare data for analytics and reporting.
- Implement data ingestion, transformation, and loading processes to support business requirements.
- Monitor data pipelines, troubleshoot failures, and resolve data processing issues.
- Optimize pipeline performance, reliability, and scalability.
- Collaborate with cross-functional teams to understand data requirements and deliver effective technical solutions.
- Follow coding standards, version control practices, testing procedures, and Agile ways of working.
- Maintain technical documentation and support data platform improvements.
- 4–6 years of professional experience in data engineering, Azure data solutions, or related technical roles.
- Hands-on experience with Microsoft Azure data engineering services.
- Strong programming knowledge in Python for data processing and transformation.
- Experience developing and maintaining data pipelines using Azure Data Factory.
- Practical experience with Azure Databricks and Apache Spark.
- Experience integrating data from external systems using APIs.
- Understanding of data ingestion, transformation, ETL/ELT processes, and data integration concepts.
- Knowledge of data pipeline monitoring, troubleshooting, and performance optimization.
- Familiarity with Agile methodologies and collaborative development practices.
- Strong analytical, problem-solving, and communication skills.
- Develop reliable and scalable data pipelines using Python, Azure Data Factory, and Azure Databricks.
- Extract, transform, and load data from different sources into target data platforms.
- Build API-based integrations to ingest data from external applications and systems.
- Process and transform large datasets using Databricks and Spark.
- Troubleshoot pipeline failures, data quality issues, and performance bottlenecks.
- Optimize data processing workflows to improve performance and reliability.
- Collaborate with technical teams and business stakeholders to translate requirements into data engineering solutions.
- Follow established development standards and participate in Agile development activities.
- Document data pipelines, integration processes, and technical solutions.
- An opportunity to work on Azure-based data engineering projects.
- Exposure to Python development, cloud data integration, and large-scale data processing.
- Opportunities to contribute to data pipeline development and optimization.
- Collaboration with technical teams in an Agile working environment.
- A six-month project engagement based in Amsterdam, Netherlands.
Core Responsibilities
Design, develop, and maintain scalable data pipelines using Microsoft Azure services and Python. Collaborate with cross-functional teams to optimize data processing workflows and ensure reliable data integration.
Requirements
Requires 4–6 years of professional experience in data engineering with hands-on expertise in Azure, Python, and Databricks. Candidates must possess strong analytical skills and experience with ETL processes and API-based data integration.
About STAFIDE
Industry: Staffing and Recruiting
Company size: 11-50 employees
Headquartered in the Netherlands, STAFIDE specializes in Niche Tech Recruitment across Europe, connecting top talent with outstanding opportunities in the IT sector. Our data-driven recruitment model ensures the ideal match between candidates and employers, enhancing both skill alignment and cultural fit. With a strong focus on Skill-Based Hiring, we provide candidates with an in-depth understanding of the organization—whether for permanent positions or temporary assignments (interim opdrachten) like secondment (detachering)—strengthening employer branding for our clients as a recruitment company (uitzendbureau). In today’s fast-evolving tech landscape, we recognize the unique challenges companies face in sourcing highly skilled talent. Our advanced, AI-driven talent intelligence system (ATS) is specifically designed to prioritize Skill-Based Hiring for niche IT roles, empowering organizations to address talent shortages by identifying professionals with the expertise to drive innovation and growth. Whether you’re a company in search of exceptional talent or a professional seeking your next ICT role, STAFIDE is your strategic recruitment (werving en selectie) partner. Our deep expertise in the Dutch market, combined with a steadfast commitment to excellence, makes us the preferred choice for all your recruitment needs.