Data Platform Engineer (2137)

📌 Position: Senior Data Platform Engineer (Snowflake, Kafka & AWS)
Location: Remote from LATAM

Contract Type: Full-time vendor (Contract directly with In All Media)

Time Zone Alignment: US Central Time (CT) overlap (6+ hours)

🧭 About In All Media

In All Media is a global technology and design firm focused on building impactful digital solutions through remote, distributed teams across LATAM. We partner with international clients across industries, providing long-term technical expertise, product innovation, and team augmentation. Contract Details: This is a 100% remote contractor position open to talent anywhere in LATAM. Your contract and engagement will be directly managed with In All Media.

🚀 Project Overview

In this role, you will partner with an enterprise engineering organization executing a high-impact data platform modernization and infrastructure migration initiative. The project centers on migrating data platform workloads, pipelines, and storage from legacy corporate systems into a standardized, modern cloud environment in AWS (us-east-2), anchored by Snowflake and Confluent Kafka. Operating within enterprise security standards, change control frameworks, and secure cloud workspace environments (AWS WorkSpaces), you will bridge DevOps and platform engineering practices to modernize batch workflows, automate multi-environment deployments, and ensure high-availability operational reliability.

🔍 Key Responsibilities

- Cross-Organization Infrastructure Migration: Lead and execute the hands-on migration of data platform workloads, pipelines, and storage from legacy corporate infrastructure to target enterprise cloud infrastructure.
- Regional Cloud Architecture: Migrate workloads and infrastructure to AWS us-east-2 (USE2), ensuring required cross-network connectivity, security prerequisites, and deployment operational standards are met.
- Modernization & Standardization: Modernize legacy batch ecosystems (Hadoop, Hive, Airflow) by transitioning workloads toward modern cloud-native architectures (Snowflake, Confluent Kafka, AWS-managed services) and enterprise platform standards.
- Data Accessibility & Governance: Implement governed, secure access models to expose platform data to downstream systems and business consumers across entities while adhering to strict compliance and least-privilege principles.
- Platform Operations & Reliability: Operate, maintain, and monitor production and staging environments across Snowflake, Confluent Kafka, and AWS infrastructure to reduce platform operational support overhead.
- Infrastructure as Code (IaC) & Auditing: Utilize Terraform and GitLab CI/CD to ensure all configurations, platform changes, and environment setups are codified, auditable, and automated.
- Incident Response & Runbooks: Participate in production troubleshooting, on-call incident response, root cause analyses (RCA), and author comprehensive operational runbooks and migration documentation.

💡 Must-Have Skills

- Senior-Level Experience: 3+ years of hands-on, senior-level experience in production data platform engineering with a strong DevOps / Platform Engineering orientation.
- Programming & Scripting: Strong programming proficiency in Python, shell scripting (Bash), and advanced SQL.
- Core Platform Operation (Snowflake + Kafka): Proven production operational/DevOps experience with both Snowflake and Confluent/Kafka, meeting either of the following profiles:
- Deep expertise in Snowflake and AWS, coupled with solid working operational knowledge of Kafka (topics, schema registries, connectors); OR
- Deep expertise in Confluent/Kafka and AWS, coupled with solid working operational knowledge of Snowflake.

- Lifecycle & Multi-Environment Delivery: Proven track record operating, deploying, and migrating production-grade data pipelines across development, staging, and production tiers.
- Security & Operations: Demonstrated proficiency in least-privilege security architectures, enterprise change control, secure secrets management, and audited deployment practices.
- Operational Excellence: Strong debugging, production incident-handling skills, runbook authoring, and clear cross-functional stakeholder communication.
- Environment & Connectivity (Mandatory Compliance): Ability and requirement to perform all development work within an enterprise-provisioned Virtual Desktop Infrastructure (AWS WorkSpaces). Candidates must verify reliable network latency/connectivity to an AWS WorkSpace region by completing the official health check and submitting a screenshot of passing results alongside their profile submission.

🌟 Nice-to-Have Skills

- Legacy Big Data Modernization: Hands-on experience with Apache Spark, Hadoop, Hive / HQL, and migrating legacy batch jobs or Airflow orchestrations to cloud-native platforms.
- Lakehouse & Data Quality: Familiarity with modern data lakehouse formats (Snowflake-to-Iceberg patterns), Snowpark, dbt alternatives, AWS Glue, or data observability platforms (e.g., Monte Carlo).
- Corporate Integration / M&A: Prior experience navigating technical integrations, acquisitions, or system alignments across merged organizations.

🌐 Time Zone & Collaboration

This role requires a minimum overlap of 6 hours with US Central Time (CT) business hours during a standard 8-hour workday to collaborate effectively with distributed engineering teams.

💬 Language

All interviews, technical documentation, daily team syncs, and operational communications will be conducted in English.