Senior Data Engineer
Key Skills
Job Description
Technical skills we are looking for: - At least 8 years of relevant experience in data engineering, cloud engineering, data analytics or a closely related technical field. - Proven hands-on experience designing and building complex production-grade data pipelines using ETL and ETL patterns. - Strong programming skills in Python and SQL, with practical experience using PySpark for distributed data processing. - Hands-on experience with Apache Spark and workflow orchestration tools such as Apache Airflow. - Experience with cloud data platforms, preferably Google Cloud Platform and BigQuery, and working knowledge of Azure or AWS. - Strong understanding of data modelling, normalization and denormalization, data integration, data lakes and data warehouses. - Experience with relational and non-relational data stores, including technologies such as MS SQL, Hive and dbt, plus object storage such as Amazon S3. - Experience with containerization, orchestration and automated delivery tooling, including Docker, Kubernetes and CI/CD pipelines. - Experience managing and optimizing distributed systems and clusters for large-scale batch processing. - Good command of software engineering practices, including modular design, testing, code review, documentation, security and production support.
Core Responsibilities
Design and build complex, production-grade data pipelines using ETL patterns and distributed processing technologies. Manage and optimize large-scale batch processing systems while applying sound software engineering practices and supporting production workloads.
Requirements
Requires at least 8 years of relevant technical experience, strong Python and SQL skills, and hands-on experience with PySpark, Apache Spark, and workflow orchestration. Candidates should also have cloud data platform experience, knowledge of data modeling and storage systems, and familiarity with containers, CI/CD, testing, security, and production support.
About Virtusa
Industry: IT Services and IT Consulting
Company size: 10,001+ employees
Virtusa is a global product and platform engineering services company that makes experiences better with technology. We help organizations grow faster, more profitably, and more sustainably by reimagining enterprises through domain-driven solutions. We combine strategy, design, and engineering, backed by unmatched expertise at the intersection of industry, business, and technology to generate real-world business impact for clients. Headquartered in Massachusetts with global delivery centers, Virtusa provides a broad range of services, solutions, and assets, including strategy and design, AI advisory and services, digital engineering, data and analytics, digital assurance, cloud and security, cx transformation and managed services across industries such as financial services, healthcare, communications, media, entertainment, travel, manufacturing, and technology.