Senior Python Engineer - Data ( Remote, Full-Time) [AS340]

Responsibilities

- Design, build and maintain reliable data pipelines in Python on AWS.

- Build and maintain backend services and APIs using Django and FastAPI.

- Model, query and optimise data across PostgreSQL and search/NoSQL databases, including OpenSearch, Elasticsearch and DynamoDB.

- Deploy and run services on Kubernetes, working alongside the DevOps engineer.

- Take end-to-end ownership of assigned work, including design, testing, code reviews, releases and production support.

- Work independently on assigned tasks while collaborating closely with the engineering team and DevOps/SRE engineer on infrastructure and releases.

- Maintain high standards of data quality, correctness and testing while understanding how data pipelines contribute to the wider product.

- Follow established engineering practices, including code reviews and CI/CD, while adapting to changing priorities.

- Proactively identify problems, propose solutions and challenge technical decisions when necessary rather than simply executing tickets.

- Prioritise thoroughness, reliability and production stability when developing and maintaining data pipelines and data stores.

Requirements

- 6+ years of experience in software engineering, including 5+ years of commercial Python development.

- 5+ years of Python backend development experience, with strong hands-on engineering capabilities.

- 3+ years of experience in data pipelines and data engineering, including ETL/ELT processes and batch processing. Relevant technologies include Airflow, Dagster, Prefect, AWS Glue and Step Functions.

- 3+ years of AWS experience, with practical experience developing and maintaining cloud-based backend services and data pipelines. Strong GCP or Azure experience may be considered on a case-by-case basis.

- 3+ years of experience with Python web frameworks, particularly Django or FastAPI. Flask is an acceptable alternative.

- 3+ years of PostgreSQL experience, including data modelling, querying and optimisation. MySQL or SQL Server experience may be considered where strong SQL expertise is demonstrated elsewhere.

- 2+ years of experience with non-relational or search databases, such as Elasticsearch, OpenSearch or DynamoDB. Relevant alternatives include MongoDB, Redis and Cassandra.

- 2+ years of Kubernetes experience, including deploying and running services. Docker combined with ECS or EKS exposure may be considered as an alternative.

- Ability to independently own technical tasks from design through implementation, testing, release and production support.

- Strong attention to data quality, correctness, testing and production reliability.

- Ability to collaborate with engineering and infrastructure teams while independently proposing solutions and challenging decisions where appropriate.

- Ability to follow engineering practices and adapt to changing priorities.

Nice to Have

- 1+ year of experience with workflow orchestration, using Airflow or similar technologies. Alternatives include Celery and cron-based schedulers.

- 1+ year of experience with streaming or event-processing technologies, such as Kafka, Kinesis or SQS. RabbitMQ is an acceptable alternative.

- 1+ year of experience with Infrastructure as Code and CI/CD, including Terraform and GitHub Actions. Alternatives include Pulumi and GitLab CI.

- Previous experience leading a technical project or taking complete end-to-end ownership of a data pipeline or backend service