Senior Data Engineer

Public Sector Resourcing (PSR)London, EnglandRemoteContractListed 7 hours ago

Apply now

About this role

### Posting ID
PIP12296

On behalf of BEIS, we are looking for a Senior Data Engineer Inside IR35 for a 4 months contract based REMOTELY.

As a Senior Data Engineer , your main responsibilities will be:

PLEASE NOTE: SC/DV Clearance is an essential requirement for this role, as a minimum you must be willing & eligible to undergo checks. Please note, due to the exceptional requirements of this position (short-term nature of this role and speed at which we require a postholder in situ) preference may be given to candidates who meet all of the essential criteria and hold active security clearance.

Key Responsibilities

- Develop, test and maintain data ingestion and transformation pipelines using Microsoft Fabric, OneLake and Azure services
- Build and support reusable data engineering components and data integration patterns
- Design and optimise data models, datasets and data products for reporting, analytics and AI use cases
- Integrate data from Dynamics 365, Dataverse and other enterprise applications
- Develop API-based and event-driven data integration solutions
- Implement data quality, validation and reconciliation processes within pipelines
- Monitor, troubleshoot and optimise data platform performance
- Support CI/CD, automated testing and DataOps practices
- Maintain technical documentation, metadata and data lineage information
- Collaborate with technical teams, business stakeholders and third-party suppliers
- Investigate and resolve data quality, integration and platform issues
- Contribute to continuous improvement of engineering standards and platform capabilities

Essential Skills & Experience

- Strong commercial experience as a Data Engineer or Senior Data Engineer
- Hands-on experience with Microsoft Fabric and OneLake, or closely related Azure data technologies
- Strong SQL development and database optimisation skills
- Experience building and supporting ETL/ELT pipelines
- Experience working with Azure data services and cloud-based data platforms
- Knowledge of Dynamics 365, Dataverse or similar enterprise business systems
- Experience with APIs, event-driven integrations and change-data capture techniques
- Strong understanding of data modelling, transformation and integration best practices
- Experience implementing data quality, governance and security controls
- Knowledge of CI/CD, version control and DataOps methodologies
- Ability to troubleshoot, investigate and resolve production data issues

Desirable Experience

- Azure Data Factory
- Azure Synapse Analytics
- Azure Data Lake
- Microsoft Fabric Lakehouse and Warehouse solutions
- Fabric Notebooks
- Python or PySpark
- AI and advanced analytics platforms
- Enterprise-scale reporting and management information environments

We're Looking For Someone Who

- Takes ownership of technical delivery and engineering quality
- Enjoys solving complex data integration challenges
- Can work effectively with both technical and non-technical stakeholders
- Has a proactive approach to identifying risks, issues and improvement opportunities
- Thrives in collaborative, delivery-focused environments
- Is passionate about modern data engineering and cloud technologies

Key Technologies

- Microsoft Fabric
- OneLake
- Azure Data Services
- Dynamics 365
- Dataverse
- SQL
- APIs
- CI/CD
- DataOps
- Azure DevOps / Git

Please be aware that this role can only be worked within the UK and not Overseas.

In applying for this role, you acknowledge the following "this role falls in scope of the Off Payroll Working in the Public Sector legislation. Any rates of payment quoted will reflect the gross rate per day for the assignment and will be subject to appropriate taxes and statutory costs. As such the payment to the intermediary and your income resulting from this contract will be different".