305 King St W
Suite 1100
Kitchener, ON N2G 1B9
Canada
Lead Data Engineer (Databricks) Athens, Greece
Lead Data Engineer (Databricks) Description
We are seeking an experienced Lead Data Engineer to join our dynamic team working on a cutting-edge supply-chain data analytic platform.
As a Lead Data Engineer, you will be at the forefront of designing and developing our data graph, which serves as the backbone of our platform. You will collaborate closely with our Machine Learning team and take charge of implementing system-level improvements and data engineering components across the organization.
#LI-DNI#LI-IK2
Responsibilities
- Lead the design, development, and maintenance of scalable data pipelines and architectures
- Collaborate with Machine Learning engineers to integrate AI-driven insights and recommendations
- Develop and optimize data models and ETL processes using Databricks and other technologies
- Implement data quality checks and monitoring to ensure high data integrity
- Mentor and guide junior data engineers, fostering a collaborative team environment
- Stay updated with emerging trends and technologies in data engineering and propose adoption of new tools where beneficial
- Ensure compliance with data governance and security policies
- Troubleshoot and resolve data-related issues in a timely manner
- Participate in code reviews and maintain high standards of code quality
Requirements
- Experience as a Lead Data Engineer in a project employing the Databricks platform
- Solid understanding of data architectures, data modeling skills; experience in designing and building ETL pipelines with Databricks using external orchestrators like Airflow
- Hands-on experience with Databricks (Delta Lake, workflows, Delta Live Tables, deployment, and versioning)
- Expertise in Python, PySpark, and SQL
- Proficiency in cloud-native technologies and software engineering best practices (containers, unit tests, linting, and code style checks)
- Engineering experience with either AWS, Azure, or both
- Experience with big data and performance optimization of data-intensive applications
- Proactivity and client-facing experience
- Ability to deal with ambiguity and work independently without constant direction
- Desire to work in a transparent and fast-moving startup environment
- Fluent English communication skills at a B2+ level
Nice to have
- Experience setting up or maintaining CI/CD pipelines on Azure DevOps
- Understanding of Data Observability and Data Quality Monitoring; experience integrating data quality checks in data pipelines
- Familiarity with Docker and ETL/ELT Solutions
- Knowledge of Ingestion Pipelines, PostgreSQL, and Terraform
We offer
- For you:
- Paid annual vacation
- Paid sick leave days
- Private health insurance
- Stable income
- Meal and home office compensation
- For your comfortable work:
- Remote and hybrid work opportunities
- Corporate laptop
- Possibility to work on your own device
- Free licensed software
- Relocation opportunities
- Free wellbeing activities
- For your growth:
- Possibility to create a Personal Development Plan from the first day in the company
- Free trainings for technical and soft skills
- Free access to LinkedIn Learning platform
- Free access to internal and external e-Libraries
- Certification opportunities
- Language courses
- Internal technical and non-technical communities
- Possibility to contribute in internal, opensource products
EPAM is a leading global provider of digital platform engineering and development services. We are committed to positively impacting our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to learn and grow continuously. You will join a dedicated, creative, and diverse community that will help you discover your fullest potential.
EPAM is committed to providing our global team of 52,800+ EPAMers with inspiring careers. EPAMers lead with passion and honesty and think creatively. Our people are the source of our success, and we value collaboration, try always to understand our customers' business, and strive for the highest standards of excellence.
