About this role
Accountabilities:
- Own, maintain, and continuously improve change data capture (CDC) and data synchronization pipelines, ensuring accurate, reliable, and timely data delivery from market-level databases into Snowflake.
- Optimize pipeline performance, scalability, and infrastructure efficiency by reducing processing latency, controlling operational costs, and supporting increasing data volumes across multiple markets.
- Design, develop, and evolve data infrastructure by onboarding new data sources, tables, and markets while managing schema changes and ensuring compatibility across systems.
- Establish and maintain service level objectives (SLOs), implement monitoring and alerting mechanisms, and develop automated data quality checks to ensure data accuracy, availability, and reliability.
- Strengthen engineering standards by developing unit and integration tests, maintaining CI/CD workflows, implementing infrastructure-as-code, and introducing safe deployment strategies such as dry runs and blue-green deployments.
- Troubleshoot production incidents, lead root cause analyses, and translate findings into improvements in monitoring, testing, technical documentation, and operational runbooks.
- Participate in architectural and technical design discussions, prepare and review technical design documents, and contribute to code quality through constructive peer reviews.
- Collaborate with Data Science, Analytics, Data Platform, Product, Credit, and Business Development teams to understand data requirements and provide dependable access to critical information.
- Take end-to-end ownership of production data systems, from development and deployment through ongoing maintenance and operational improvement.
Requirements:
- At least 4 years of professional experience in backend engineering or data engineering, ideally supporting consumer-scale applications.
- Minimum 2 years of hands-on professional experience developing with Python.
- Proven ability to independently build, operate, troubleshoot, and optimize production-grade data pipelines.
- Practical experience with change data capture (CDC) and stream processing technologies such as Kafka, Amazon MSK, AWS Database Migration Service (DMS), or comparable solutions.
- Strong proficiency in Apache Spark, particularly Spark Structured Streaming.
- Experience with modern data lakehouse architectures and table formats, including Apache Iceberg, Delta Lake, or Apache Hudi, as well as Parquet.
- Strong database knowledge across relational and analytical platforms, including MySQL, PostgreSQL, Amazon RDS, and Snowflake.
- Hands-on experience with AWS cloud infrastructure, ideally including EMR, S3, MSK, DMS, RDS, and EKS.
- Experience with infrastructure-as-code and CI/CD technologies such as Terraform, Terragrunt, Helm, and Jenkins.
- Familiarity with data pipeline orchestration and job scheduling tools such as Apache Airflow or Python-based services running on Kubernetes.
- Strong understanding of distributed systems, scalable algorithms, and large-scale data processing architectures.
- Excellent problem-solving and analytical skills, with the ability to investigate complex technical challenges and deliver reliable solutions.
- Strong written and verbal communication skills, including the ability to explain complex technical concepts to engineering teams and business stakeholders.
- Ability to work independently, take ownership of technical initiatives, and collaborate effectively across multidisciplinary teams.
Nice to Have:
- Experience with Trino or Presto query engines.
- Practical knowledge of Kubernetes and containerized applications.
- Familiarity with AWS Glue.
- Experience with additional stream processing technologies such as Debezium, Amazon Kinesis, or Apache Flink.
- Exposure to modern data lakehouse technologies, including Delta Lake and Apache Hudi.
- Advanced degree in Computer Science, Mathematics, or another relevant technical discipline.
Benefits:
- Annual salary: $130,000–$200,000, depending on qualifications, experience, and other relevant factors.
- Fully remote, remote-first working environment.
- Opportunity to collaborate with an international team across multiple global markets.
- Exposure to modern cloud infrastructure, AI-driven technology, and large-scale distributed data systems.
- Opportunity to contribute to innovative financial technology solutions with meaningful global impact.
- Inclusive and diverse working environment that values different backgrounds, perspectives, and professional experiences.
- Opportunities to influence technical architecture, engineering standards, and the evolution of critical data infrastructure.
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1