Senior Data Engineer

At HexaIO we are looking for a Senior Data Engineer with strong software engineering skills to join an international e-invoicing project for an enterprise client in the insurance sector.

We are looking for someone who can deliver a stable production application, not only ETL pipelines.

It is a strong advantage if you efficiently use AI-based development tools and agents (e.g., GitHub Copilot, ChatGPT, Cursor, or similar solutions) on a daily basis, and can apply them productively to support the development process.

As a Data Engineer, you will:

- Build the Python backend that retrieves invoices from an external API and validates their content.

- Implement API-based content-based routing that forwards validated invoices to downstream procurement and ERP systems.

- Develop and deploy the solution as a Databricks App using Workflows, Delta Lake, and Unity Catalog.

- Build and extend the operational UI in Streamlit or React to show processing status and error details.

- Integrate REST and JSON APIs secured with OAuth 2.0, including token handling and retry logic.

- Cover the application with unit and integration tests, and maintain CI/CD pipelines in Git.

- Set up structured logging and monitoring so production incidents can be traced and resolved quickly.

- Work directly with the client's data and procurement stakeholders to clarify validation rules and routing targets.

Requirements

- At least 5 years of professional experience in Python backend or data engineering

- Hands-on Databricks experience: Databricks Apps, Workflows, Delta Lake, Unity Catalog

- REST API integration with JSON and OAuth 2.0 authentication

- Production software engineering practices: Git, CI/CD, unit and integration testing, logging and monitoring

- Frontend development with Streamlit or React inside Databricks Apps

- Fluent English

Development knowledge of as many of the following technologies as possible:

- Python frameworks and tooling: FastAPI, Pydantic, pytest

- PySpark, SQL

- Databricks Asset Bundles, Terraform

- Azure or AWS cloud services

- Monitoring and observability tooling, e.g. Databricks system tables, Grafana

Nice to Have

- E-invoicing standards such as UBL or PEPPOL

- SAP Ariba or other procurement platform integrations

- ERP integration experience, e.g. SAP or Oracle

- Insurance or financial services domain background

- Databricks Apps already delivered to production