Data Engineer GCP AWS & Databricks at iShare Inc

About this role

Data Engineer – GCP, AWS & Databricks

Location: Remote – United States
Employment Type: Contract
Experience Required: 6+ Years
Position Overview
We are seeking an experienced Data Engineer to design, develop, and maintain scalable cloud-based data pipelines and data platforms. The ideal candidate will have strong hands-on experience with GCP, AWS, Databricks, Apache Spark, PySpark, Python, and SQL.
This role requires expertise in building reliable ETL/ELT pipelines, integrating data from multiple sources, and optimizing cloud data solutions for performance, security, scalability, and cost.
Key Responsibilities
• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build cloud-based data solutions using GCP and AWS services.
• Use Databricks, Apache Spark, and PySpark for large-scale data processing.
• Integrate structured, semi-structured, and unstructured data from multiple sources.
• Develop and optimize batch and real-time data-processing workflows.
• Improve pipeline performance, reliability, scalability, and cost efficiency.
• Implement data-quality checks, monitoring, security, and governance standards.
• Design and support cloud data warehouses and data lakes.
• Troubleshoot production issues and perform root-cause analysis.
• Collaborate with data architects, analysts, application teams, and business stakeholders.
• Create and maintain technical documentation for pipelines, data models, and workflows.

Required Qualifications
• 5+ years of professional data engineering experience.
• Strong hands-on experience with both GCP and AWS.
• Expertise in Databricks, Apache Spark, and PySpark.
• Strong programming skills in Python and SQL.
• Proven experience developing ETL/ELT pipelines and cloud data platforms.
• Experience with data warehouses, data lakes, and dimensional data modeling.
• Experience with orchestration tools such as Apache Airflow or Google Cloud Composer.
• Understanding of data security, governance, monitoring, and quality frameworks.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and cross-functional collaboration skills.

Preferred Qualifications
• Experience with GCP services such as BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, and Cloud Composer.
• Experience with AWS services such as S3, Glue, EMR, Redshift, Lambda, and Kinesis.
• Familiarity with Delta Lake and Databricks Lakehouse architecture.
• Experience with streaming technologies such as Apache Kafka.
• Familiarity with CI/CD, Git, Terraform, and cloud infrastructure automation.
• Experience working in Agile development environments.