
Senior Azure Data Engineer (ID: 4146)
Key Skills
Job Description
- Design, develop, and maintain complex data pipelines using Azure Data Factory (ADF) and Azure Databricks.
- Build, implement, and optimize large-scale data processing solutions using PySpark and Databricks.
- Develop robust ETL processes using Python for data extraction, transformation, and loading.
- Design and implement CI/CD pipelines using Azure DevOps to automate testing, integration, and deployment.
- Work with SQL to query, process, transform, and manage large datasets.
- Monitor, troubleshoot, and optimize data pipelines to ensure performance, scalability, and reliability.
- Collaborate with cross-functional teams to understand business and data requirements and deliver effective data solutions.
- Ensure data security, quality, and compliance with established standards and best practices.
- Stay up to date with emerging technologies and best practices in Azure cloud data engineering.
- 8–10 years of overall experience in IT, with strong experience in data engineering and development.
- Hands-on experience with Azure Data Factory for data integration, orchestration, and pipeline automation.
- Strong expertise in Azure Databricks for scalable data processing and engineering solutions.
- Proficiency in Python and PySpark for big data processing and ETL development.
- Experience implementing CI/CD pipelines using Azure DevOps.
- Good knowledge of SQL and experience working with large datasets.
- Experience developing, maintaining, and optimizing complex data pipelines.
- Understanding of data security, data quality, and cloud data engineering best practices.
- Design and develop scalable, reliable, and high-performance data engineering solutions.
- Troubleshoot complex data pipeline issues and improve processing efficiency.
- Work independently as well as collaboratively within cross-functional teams.
- Analyze technical and business requirements and translate them into effective data solutions.
- Apply problem-solving and analytical skills to overcome technical challenges.
- Communicate technical concepts clearly and maintain proper technical documentation.
- Adapt to evolving Azure technologies and modern data engineering practices.
- An opportunity to work on enterprise-scale Azure data engineering solutions.
- Exposure to modern cloud data technologies, including Azure Data Factory, Azure Databricks, and PySpark.
- An opportunity to contribute to large-scale data processing and automation initiatives.
Core Responsibilities
Design, develop, and maintain complex data pipelines using Azure Data Factory and Databricks. Collaborate with cross-functional teams to build scalable data processing solutions and ensure data quality and security.
Requirements
Requires 8-10 years of experience in IT with strong expertise in Azure cloud data engineering. Proficiency in Python, PySpark, SQL, and CI/CD pipeline implementation is essential.
About STAFIDE
Industry: Staffing and Recruiting
Company size: 11-50 employees
Headquartered in the Netherlands, STAFIDE specializes in Niche Tech Recruitment across Europe, connecting top talent with outstanding opportunities in the IT sector. Our data-driven recruitment model ensures the ideal match between candidates and employers, enhancing both skill alignment and cultural fit. With a strong focus on Skill-Based Hiring, we provide candidates with an in-depth understanding of the organization—whether for permanent positions or temporary assignments (interim opdrachten) like secondment (detachering)—strengthening employer branding for our clients as a recruitment company (uitzendbureau). In today’s fast-evolving tech landscape, we recognize the unique challenges companies face in sourcing highly skilled talent. Our advanced, AI-driven talent intelligence system (ATS) is specifically designed to prioritize Skill-Based Hiring for niche IT roles, empowering organizations to address talent shortages by identifying professionals with the expertise to drive innovation and growth. Whether you’re a company in search of exceptional talent or a professional seeking your next ICT role, STAFIDE is your strategic recruitment (werving en selectie) partner. Our deep expertise in the Dutch market, combined with a steadfast commitment to excellence, makes us the preferred choice for all your recruitment needs.