305 King St W
Suite 1100
Kitchener, ON N2G 1B9
Canada
Lead Data Software Engineer Malaysia or Remote
Lead Data Software Engineer Description
We are seeking a Lead Data Software Engineer to take ownership of designing and implementing advanced data engineering solutions.
Our client specializes in delivering cutting-edge property insights and data analytics, enabling clients in real estate, financial services and insurance to make informed decisions backed by comprehensive data. This role requires both hands-on expertise and the ability to guide a team of engineers. You will work in a dynamic and challenging environment, collaborating with a transitioning client team to ensure seamless knowledge transfer and successful project delivery.
The ideal candidate will have extensive experience with Python, Spark and cloud platforms and strong leadership and problem-solving skills.
#LI-DNI#REF_MY_PrR
Responsibilities
- Lead the design and development of scalable data pipelines and architectures using Python, Spark, Airflow and Kafka
- Architect and implement robust data storage and processing solutions leveraging AWS (DynamoDB, Glue, S3, Lambda, Fargate) or GCP and Snowflake
- Provide technical leadership and mentorship to a team of engineers
- Drive knowledge transfer from the client’s onshore team and ensure continuity in project delivery
- Collaborate with stakeholders to define technical requirements and deliver effective data solutions
- Oversee the implementation of containerized solutions using Docker, with a focus on scalability and reliability
- Ensure adherence to best practices in code quality, performance, and security
- Work within an Agile framework (e.g., Scrum, Kanban) and lead the team in adopting DevOps practices such as CI/CD pipelines and automated testing
- Use GitFlow or similar version control workflows for development and deployment
Requirements
- Extensive experience with Python, Spark and SQL in large-scale data environments
- Proven ability to lead technical teams and deliver complex projects
- Deep understanding of cloud-based ecosystems and distributed data systems
- Strong interpersonal and stakeholder management skills to address challenges in a transitioning client team
- Exposure to advanced technologies, such as serverless architecture and data lake design
- Highlight time zone flexibility to collaborate effectively with an Australian client
- Understanding and proficiency in general big data concepts
We offer
- By choosing EPAM, you're getting a job at one of the most loved workplaces according to Newsweek 2021 & 2022&2023
- Employee ideas are the main driver of our business. We have a very supportive environment where your voice matters
- You will be challenged while working side-by-side with the best talent globally. We work with top-notch technologies, constantly seeking new industry trends and best practices
- We offer a transparent career path and an individual roadmap to engineer your future & accelerate your journey
- At EPAM, you can find vast opportunities for self-development: online courses and libraries, mentoring programs, partial grants of certification, and experience exchange with colleagues around the world. You will learn, contribute, and grow with us
Life at EPAM
- EPAM is a leader in the fastest-growing segment (product development/digital platform engineering) of the IT industry. We acquired Just-BI in 2021 to reinforce our leading position as a global Business Intelligence services provider and have been growing rapidly. With a talented multinational team, we provide data and analytics expertise
- We are currently involved in end-to-end BI design and implementation projects in major national and international companies. We are proud of our entrepreneurial start-up culture and are focused on investing in people by creating continuous learning and development opportunities for our employees who deliver engineering excellence for our clients
