Senior Data Software Engineer Portugal or Remote

Responsibilities

• Develop, maintain, and optimize efficient and scalable data pipelines using Python and Apache Spark
• Build streaming and batch solutions leveraging Delta Lake tables and Spark Structured Streaming
• Collaborate closely with business partners, IT experts, and architects to gather requirements, design solutions, and refine processes
• Participate in peer code reviews to ensure code quality and adherence to best practices
• Utilize relational data models and SQL to manage, query, and process data effectively
• Implement automated testing strategies for data solutions to ensure high reliability and maintainability
• Monitor, troubleshoot, and resolve complex data processing issues to support operational stability
• Document technical designs, processes, and decisions with clarity and precision
• Proactively identify areas for improvement, recommending and implementing solutions
• Engage with product owners and team members to prioritize and deliver high-impact solutions

Requirements

• 3+ years of experience in a software engineering or data engineering role working with large-scale data systems
• Significant experience in Python, Apache Spark, and Delta Lake
• Knowledge of relational data models, SQL, and using them for complex data handling
• Expertise in cloud-native technologies within the Microsoft Azure ecosystem including Databricks and Azure DevOps
• Familiarity with automated testing practices and the ability to design reliable test strategies
• Proficiency in analyzing complex problems, breaking them into actionable steps, and delivering elegant solutions
• Fluency in English and strong communication skills to work effectively within distributed and collaborative teams

Nice to have

• Understanding of Spark Structured Streaming for real-time and near real-time data processing solutions
• Showcase of working on financial data or similar high-complexity data domains
• Competency in identifying and resolving performance bottlenecks in distributed data systems
• Familiarity with documenting data pipeline architecture and best practice implementation approaches

We offer

• Competitive compensation depending on experience and skills
• Variety of projects within one company
• Being a part of a project following engineering excellence standards
• Individual career path and professional growth opportunities
• Internal events and communities
• Flexible work hours