Data Engineer

In this position you will…

… design, build, and optimize scalable data products, pipelines, and platform capabilities using the Databricks Lakehouse Platform and related technologies. Develop production-grade data solutions that are reliable, maintainable, and built to scale across markets and regions. Establish strong data engineering standards and best practices to improve quality, governance, and operational efficiency. Partner closely with business and technical teams to transform complex datasets into trusted and actionable insights. Help ensure bolttech’s data ecosystem remains secure, governed, and scalable across countries, data centers, and AWS regions.

You will be responsible for…

- Designing, developing, testing, and operating scalable data pipelines and data products on the Databricks Lakehouse Platform.

- Building batch and streaming data-processing solutions using PySpark, Spark SQL, and Delta Lake.

- Creating and managing production pipelines using Delta Live Tables, including data-quality controls, pipeline dependencies, and operational monitoring.

- Orchestrating data pipelines and platform workloads using Databricks Workflows.

- Managing DABs to perform CICD operations and change management processes.

- Implementing governed data access, cataloguing, lineage, and permissions using Unity Catalog.

- Applying MLflow to support experiment tracking, model lifecycle management, and collaboration between data engineering and data science teams.

- Designing efficient Delta Lake data structures, including appropriate approaches to schema management, data quality, performance, and workload optimization.

- Building and maintaining reliable extraction, transformation, and loading processes across a wide variety of data sources.

- Integrating Databricks with relevant AWS services, including Amazon S3, EC2, EMR, RDS, Redshift, and AWS Glue.

- Designing secure data solutions that respect national boundaries, regional requirements, and data-separation controls across multiple data centers and AWS regions.

- Building large and complex datasets that satisfy functional and non-functional business requirements.

- Implementing reusable data-engineering frameworks, common libraries, monitoring capabilities, and development standards.

- Performing root-cause analysis of data, pipeline, performance, and platform issues and implementing sustainable corrective actions.

- Improving engineering processes by automating manual activities, optimizing data delivery, and redesigning solutions for scalability, reliability, and maintainability.

- Building data tools and curated datasets that support analytics, reporting, data science, and operational use cases.

- Applying software-engineering practices such as version control, code reviews, automated testing, deployment automation, and environment management.

- Working with executive, product, data, partner, and engineering stakeholders to resolve data-related technical issues and support their data infrastructure requirements.

- Collaborating with data and analytics specialists to improve the functionality, usability, and business value of bolttech’s data platforms.

For you to be successful…

…we expect you to be able to demonstrate the following key competencies:

Customer Focus

- Actively seeks to understand customer feedback and needs and uses this in decision making and solutioning

- Responds quickly and effectively to new customer ideas and request.

- Focused on simplifying and improving customer journeys, leveraging automation

Impactful

- Proactive in identifying what needs to be done, and taking action, before being asked, or before the situation escalates

- Evaluates data and makes decisions, including differing stakeholder perspectives and/or some missing information. Understands when an “80% solution” is sufficient and acts accordingly

- Takes accountability and self-motivated to deliver results even in situations which are not straight-forward

Collaborative

- Partners with a range of people to create trust, and co-create and deliver mutually beneficial outcomes

- Develops collaborative and dynamic working relationships to achieve the best possible outcomes

- Resolves disputes using a range of tactics to prioritize positive outcomes

Communication

- Effective and articulate communicator actively and respectfully listens to and synthesizes others’ perspectives

- Keeps relevant people accurately informed and up-to-date of both positive and potentially negative information

- Concise in communicating and references relevant information tailored to the audience to support points

Adaptable

- Looks to understand bigger picture rationale for changes and adapts in a flexible and nimble manner

- Flexible in successfully juggling multiple requirements, ambiguity and competing demands. Perseveres in challenging circumstances

- Adapts style and approach to best meet different situations

You will require the following qualifications and skills

- At least five years of professional data-engineering experience, including designing, building, deploying, and supporting production-grade data platforms and pipelines.

- Strong hands-on experience with the Databricks Lakehouse Platform, including Delta Lake, Unity Catalog, Databricks Workflows, Databricks Asset bundles (DABs), Delta Live Tables (DLT), Delta Share and MLflow.

- Strong development experience with Python, PySpark, and advanced SQL, including complex data transformations and performance optimization.

- Proven experience designing and operating scalable batch and streaming data pipelines, including monitoring, observability, error handling, recovery, and data-quality controls.

- Strong understanding of data architecture, modelling, metadata, lineage, dependencies, distributed processing, workload optimization, change management and code versioning.

- Experience implementing data governance, access controls, security, cataloguing, and lineage within enterprise data environments.

- Strong knowledge of AWS, particularly S3 and related services such as EC2, EMR, RDS, Redshift, and Glue.

- Experience integrating Databricks with cloud storage, databases, APIs, event streams, and enterprise data sources, with a good understanding of networking, IAM, encryption, and secrets management.

- Strong software-engineering practices, including Git, automated testing, CI/CD, clean and reusable code, environment management, and deployment automation.

- Experience working with relational and NoSQL databases and with structured, semi-structured, and unstructured datasets at scale.

- Strong analytical and troubleshooting skills, with the ability to perform root-cause analysis and resolve complex data, pipeline, platform, and performance issues.

- Strong stakeholder management and communication skills, with the ability to translate business requirements into secure, reliable, and scalable technical solutions.

- Ability to work effectively across cross-functional and internationally distributed teams, demonstrating ownership, adaptability, prioritization, mentoring, and knowledge sharing.