Sr. Data Engineer

How is this team contributing to vision of Providence?

- Ensure a continual and seamless service for customers regardless of which services they access. What will you be responsible for?

- Design, develop, and maintain end to end data pipelines and ETL/ELT workflows using Python, SQL, Azure Data Factory, Snowflake, Databricks, and related services.

- Build scalable data ingestion frameworks to extract, parse, transform, and load structured, semi-structured, and unstructured data from files, APIs, databases, and enterprise applications.

- Drive improvements in data architecture, performance tuning, optimization, and reduce technical debt across data systems

- Monitor, troubleshoot, and enhance existing data pipelines and systems; proactively resolve data issues and production defects.

- Developing Ad hoc and delivered reports using SQL backend queries with Power BI to display the data.

- Displaying advanced analytics skills for providing insights to business owners.

- Communicate clearly when faced with unclear requirements or difficult situations so blockers can quickly be removed.

- Maintain strong documentation of data processes, ER diagrams, and architecture.

- Stay current with emerging data technologies, tools, and industry best practices, and advocate improvements within the team. What would your day look like?

- Understand business and feature requirements.

- Create design documents and architecture documents along with principal engineer and architects.

- Build and enhance data pipelines using Azure Data Factory, Databricks, Python, and Snowflake.

- Onboard different data sources for integration using ADF, Databricks, Snowflake.

- Work in Agile mode and work on assigned user stories.

- Ensure code is always checked in and ensure source control standards are followed.

- Collaborate with different teams to understand end to end requirement for new data/report.

- Working on AI use cases/initiatives. Who are we looking for?

- 5–8 years of professional experience in Data Engineering.

- Strong programming skills in Python for ETL, data ingestion, and data transformation.

- Hands on expertise in Azure Cloud (ADF, Azure Databricks, Azure Storage, Azure SQL, Synapse, etc.).

- Strong knowledge on Power BI and advanced analytics.

- Deep knowledge of SQL including complex queries, performance tuning, stored procedures.

- Experience with Snowflake—warehouse design. Solid understanding of data modeling (Star/Snowflake schemas), data warehousing concepts, workflows, and metadata management.

- Experience working with structured, semi structured (JSON, Parquet), and unstructured data.

- Strong knowledge of Agile methodologies, version control (Git), and CI/CD practices, Azure DevOps.

- Excellent analytical and problem-solving skills, with the ability to work in highly collaborative and fast paced environments.

- Experience with data testing, understanding requirements, working with product owner.

- Good to have:

o Understanding of AI/ML and Generative AI concepts, including data preparation for AI use cases.

o Exposure to Snowflake Cortex AI, Azure OpenAI, Document Intelligence, or AI-powered document extraction solutions.