Ubique Systems logo

Data Engineer

Ubique Systems

Amsterdam
Contractor
5-10 years experience
On-site

Key Skills

Azure Databricks
PySpark
Python
Azure Data Factory
SQL
Spark SQL
Azure DevOps
CI/CD
ETL/ELT
Data Pipeline Orchestration
Azure Data Lake Storage
Delta Lake
Azure Synapse Analytics
Databricks Workflows
Git
Medallion Architecture

Job Description

Job Description: Azure Data Engineer – Azure Databricks & PySpark Job Overview: We are looking for an experienced Azure Data Engineer with strong expertise in Azure Databricks, PySpark, Python, and Azure Data Factory to design, develop, and maintain scalable data pipelines and ETL/ELT solutions. The candidate should have hands-on experience in big data processing, cloud data engineering, pipeline optimization, and CI/CD implementation using Azure DevOps. Key Responsibilities: Design, develop, and maintain data pipelines using Azure Data Factory and Azure Databricks. Build and optimize large-scale data processing solutions using PySpark and Python. Develop ETL/ELT workflows for data extraction, transformation, and loading. Work with Azure Data Lake Storage (ADLS) and Delta Lake. Implement CI/CD pipelines using Azure DevOps. Write complex SQL queries and optimize data processing performance. Monitor, troubleshoot, and optimize data pipelines. Ensure data quality, security, and compliance. Collaborate with cross-functional teams to deliver effective data solutions. Required Skills (Must Have): Strong hands-on experience with Azure Databricks. Expertise in PySpark and Python programming. Experience with Azure Data Factory (ADF) and pipeline orchestration. Strong knowledge of SQL and Spark SQL. Experience with Azure DevOps and CI/CD pipelines. Understanding of ETL/ELT processes and data integration. Good problem-solving, communication, and documentation skills. Good-to-Have Skills: Azure Data Lake Storage (ADLS Gen2). Delta Lake and Delta Tables. Azure Synapse Analytics. Databricks Workflows and Jobs. Git and version control. Medallion Architecture (Bronze, Silver, Gold).

Core Responsibilities

Design, develop, and optimize scalable data pipelines and ETL/ELT solutions using Azure Data Factory, Azure Databricks, PySpark, and Python. Monitor and troubleshoot pipelines, implement CI/CD with Azure DevOps, ensure data quality and security, and collaborate with cross-functional teams.

Requirements

Requires strong hands-on experience with Azure Databricks, PySpark, Python, Azure Data Factory, SQL, Spark SQL, Azure DevOps, and CI/CD. Candidates should understand ETL/ELT and data integration and have strong problem-solving, communication, and documentation skills.

About Ubique Systems

Industry: IT Services and IT Consulting

Company size: 501-1,000 employees

Ubique Systems is a global IT recruitment, technology consulting and workforce solutions company helping organisations build skilled teams, modernise operations and deliver business-critical technology initiatives. Founded in 2000, we support clients across the United Kingdom, Europe, India, Singapore and the Middle East through a combination of local market expertise and global delivery capabilities. Our services include: • IT recruitment and staffing • Technology consulting and software development • Managed workforce and outsourcing solutions • Human resource and vendor management • Enterprise integration and business intelligence • International mobility support We work with organisations across multiple industries, helping them access specialist talent, improve workforce flexibility and deliver technology solutions efficiently. Ubique Systems is ISO 27001 certified, reflecting our commitment to information security, structured processes and responsible service delivery. Whether you are looking for specialist technology professionals, a recruitment partner or support for a technology transformation initiative, our teams are ready to help. Visit our website to explore our services, current opportunities and global presence

Added Today