Data Engineer

Data Engineer

Purpose:  Build, automate, secure, and operate the Google Cloud data platform, focusing on Infrastructure as Code, CI/CD, environment management, security, observability, and FinOps across Development, Test, and Production environments.

Responsibilities

- Build and maintain reusable Terraform modules for GCP infrastructure, including VPC, VPC-SC, BigQuery, Cloud Storage, Composer, Datastream, Dataplex, IAM, KMS, and related services.
- Provision and maintain Dev, Test, and Production environments using consistent Infrastructure as Code patterns.
- Implement Terraform remote state, policy-as-code controls, and automated drift detection.
- Build and manage Cloud Build CI/CD pipelines for data ingestion, transformation, data quality rules, and semantic models.
- Implement automated unit, integration, infrastructure, and data-quality testing gates.
- Automate non-production environment refreshes with appropriate PII masking/tokenisation.
- Configure Cloud Logging, Cloud Monitoring, dashboards, SLA monitoring, and failure alerts.
- Implement FinOps controls, including billing exports to BigQuery, budgets, alerts, resource labelling, and cost reporting.
- Support cloud security hardening and remediation of infrastructure-level penetration-test findings.

Required skills and experience

- 6+ years of experience in Cloud Infrastructure, Platform Engineering, DevOps, DataOps, or similar roles.
- Minimum 3+ years of hands-on GCP experience.
- Proven experience supporting production-grade cloud or enterprise data platforms.
- Strong understanding of cloud infrastructure, networking, security, CI/CD, and observability.
- Experience managing multiple environments such as Dev, Test, and Production.
- Hands-on expertise with Google Cloud Platform (GCP).
- Proficiency in Terraform, including module design, remote state, environment management, and policy-as-code.
- Experience with Cloud Build / CI/CD.
- Strong Python skills for automation and tooling.
- Experience with BigQuery and Cloud Storage.
- Knowledge of GCP networking: Shared VPC, Private Service Connect, Private Google Access.
- Understanding of security: IAM, KMS, Secret Manager, VPC Service Controls.
- Experience with Cloud Logging and Cloud Monitoring.
- Ability to implement automated testing and deployment gates for data pipelines.
- Familiarity with Git workflows, code reviews, branch protection, tagged releases, and rollback.
- Container fundamentals: Docker, Cloud Run, GKE.
- Good-to-have skills include Cloud Composer, Dataplex, Datastream, GKE/Kubernetes, Cloud Interconnect and hybrid networking, data masking/tokenisation, FinOps, GCP Billing Export to BigQuery, Power BI, penetration-test remediation, and GitHub Actions or GitLab CI/CD.

Certifications

- Required (at least one):  Google Cloud Professional Cloud DevOps Engineer; Google Cloud Associate Cloud Engineer.
- Preferred:  HashiCorp Terraform Associate; Google Cloud Professional Cloud Security Engineer.