About this role
Area(s) of responsibility
- Experience in Perform Design, Development & Deployment using Azure Services ( Data Factory, Azure Data Lake Storage, Databricks, PySpark, SQL)
- Develop and maintain scalable data pipelines and build out new Data Source integrations to support continuing increases in data volume and complexity.
- Experience in create the Technical Specification Design, Application Interface Design.
- Files Processing – XML, CSV, Excel, ORC, Parquet file Formats
- Develop batch processing, streaming and integration solutions and process Structured and Non-Structured Data
- Good to have experience with ETL development both on-premises and in the cloud using SSIS, Data Factory, and related Microsoft and other ETL technologies (Informatica preferred)
- Demonstrated in depth skills with Azure Data Factory, Azure Databricks, PySpark, ADLS (must have) with the ability to configure and administrate all aspects of Azure SQL DB.
- Collaborate and engage with BI & analytics and business team
- Deep understanding of the operational dependencies of applications, networks, systems, security and policy (both on premise and in the cloud; VMs, Networking, VPN (Express Route), Active Directory, Storage (Blob, etc.),