About this role
What you’ll do as a Data Engineer;
- Build & optimise data pipelines: Design, construct, and maintain robust, scalable ETL/ELT pipelines using dbt and Airflow to integrate new data sources and manage changes to SQL jobs.
- Troubleshoot issues with existing SQL/ETL processes and data loads to drive to solution.
- Design and build extensible, well-structured data models, data products and data apps using various tools such as Snowflake functionality, Airflow, Soda, DBT, AWS, s3 and Bedrock.
- Implement and enforce best practices for data quality, standardised pipelines, testing, and documentation to ensure data is accurate, consistent, and trustworthy.
- Continuously optimise the Snowflake data warehouse, refining data models and architecture to improve query performance, scalability, and cost-efficiency.
- Ability to partner with P&T Engineering teams to co-develop functionality for the business.
What you’ll bring to the team:
- 3+ years of experience in a Data Engineering or similar role.
- Hands-on experience with building data pipelines, data models and implementing data quality.
- Experience accessing and manipulating structured or semi-structured data (JSON) in various cloud data environments.
- Experience with technologies such as Snowflake, Redshift, BigQuery, Hadoop, Spark, AWS and Cloud Storage.
- Expert proficiency in Python for data manipulation and scripting, advanced SQL and database stored procedures for complex querying and data modelling.
- Experience with orchestration and monitoring tools such as Airflow, dbt.
- Solid understanding of ETL/ELT design patterns, data modelling principles (e.g. Kimball, Inmon), and database architecture.
- Experience of full stack development, implementing continuous integration and automated tests (e.g. GitHub, dbt, Jenkins).
- Ability to work effectively in a fast-paced environment.
- Proven ability to communicate complex analysis in a clear, precise, and actionable manner.
- Proven ability to work creatively and analytically in a fast-paced, problem-solving environment.
- Excellent communication (verbal and written) and interpersonal skills.