Senior Data Engineer (Pyspark, Databricks)

Senior Data Engineer (PySpark)

About the Role

We are looking for a Senior Data Engineer with 5+ years of experience to join our Data Engineering team. The ideal candidate should have strong hands-on experience with PySpark, Spark, SQL, and data pipelines, along with the ability to work with large-scale datasets.

You will play a key role in building and optimizing data pipelines while improving their reliability, efficiency, scalability, and performance. This is also an opportunity to work on AI-ready datasets and modern data platforms, with exposure to technologies such as Databricks and Spark.

Key Responsibilities

- Build and maintain end-to-end data pipelines.
- Develop and implement best practices for data modeling and pipeline development.
- Create AI-ready analytical datasets with appropriate structure, metadata, documentation, and business context.
- Solve complex data pipeline challenges using PySpark and SQL.
- Work with stakeholders to understand requirements and incorporate business logic into data pipelines.
- Work with large-scale datasets and optimize data processing for performance and reliability.
- Contribute to the development and improvement of data engineering standards and processes.
- Gain hands-on exposure to Databricks, Spark, AI technologies, and modern ETL tools.
- Collaborate closely with engineering and business stakeholders to deliver high-quality data solutions.

What We're Looking For

- 5+ years of experience in Data Engineering or a related technical role.
- Strong hands-on experience with PySpark, Spark, and SQL.
- Proven experience building and maintaining data pipelines.
- Experience working with large-scale datasets.
- Strong understanding of data transformations, modeling, and pipeline architecture.
- Hands-on experience with Databricks, Delta Lake, or similar technologies.
- Ability to understand business requirements and translate them into effective data solutions.
- Strong problem-solving and analytical skills.
- Excellent verbal and written communication skills.
- Self-motivated, collaborative, and comfortable taking ownership.
- Willingness and ability to learn new technologies quickly.

Nice to Have

- Apache Airflow
- dbt
- Snowflake
- Modern ETL/ELT tools
- AI/ML data pipelines

Education

Bachelor’s or Master’s degree in Computer Science

Culture of Belonging: At SSI, we are committed to fostering a culture of belonging where everyone feels valued, respected, and empowered to contribute and grow.