About this role
Key Responsibilities
Data Pipeline Development and Management
- Design, build, and maintain reliable, scalable ETL/ELT pipelines for ingesting, transforming, and delivering data.
- Develop Apache Airflow DAGs to orchestrate pipeline tasks, schedules, dependencies, and data-processing workflows.
- Integrate data from sources such as databases, APIs, and files, applying appropriate transformations and preserving data consistency.
- Develop SQL transformations and Snowflake data structures to support downstream data use.
- Build reusable pipeline components and automate recurring processes to improve maintainability and reduce manual effort.
- Support pipeline configuration and deployments across development, test, and production environments.
Snowflake Development and Optimization
- Develop and maintain Snowflake database objects, including schemas, tables, and views.
- Implement data loading and transformation processes using Snowflake capabilities and the organization’s approved integration tools.
- Optimize SQL queries and Snowflake workloads by evaluating query performance, warehouse sizing and utilization, and cost.
- Apply appropriate Snowflake data modeling and performance optimization techniques, including clustering when beneficial for the workload.
- Implement and maintain Snowflake roles and permissions in accordance with organizational and customer security requirements.
Workflow Orchestration and Automation
- Configure Airflow schedules, task dependencies, retries, timeouts, and failure handling to support reliable data workflows.
- Monitor workflow runs, investigate failures, and implement corrective actions to prevent recurring issues.
- Configure alerts and operational logging to support timely detection and troubleshooting of pipeline issues.
- Develop Python scripts and automation to support pipeline operations; use SQL where appropriate for data transformations.
- Maintain workflow and operational documentation, including dependencies, recovery steps, and troubleshooting guidance.
Data Quality, Testing, and Security
- Develop and maintain automated data validation checks and tests to assess accuracy, completeness, consistency, and integrity.
- Investigate and resolve data-quality issues, documenting their causes and corrective actions.
- Follow established data governance, privacy, security, and access-management requirements when handling data.
- Document data lineage and transformation logic to support traceability and maintainability.
- Follow established procedures for error handling and data recovery, and support compliance with applicable customer, organizational, and regulatory requirements.
Collaboration and Continuous Improvement
- Work with technical teams and stakeholders to clarify data requirements and deliver practical engineering solutions.
- Participate in code reviews, technical design discussions, and development planning.
- Contribute to shared engineering practices for version control, automated testing, and CI/CD.
- Identify and propose improvements to the reliability, scalability, maintainability, and cost efficiency of data solutions.
- Share knowledge with teammates and provide technical guidance when appropriate.
- Perform other related duties as assigned.
Minimum Requirements
- Bachelor's degree in Computer Science, Data Engineering, Information Systems, Software Engineering, or Associates degree plus two years' experience.
- At least 3 years of professional experience in data engineering, data integration, ETL/ELT development, or a related technical field.
- At least 3 years of hands-on experience with Snowflake and Apache Airflow in data engineering or data pipeline development.
- Strong proficiency in SQL, including query development, data transformation, and performance optimization.
- Experience developing, scheduling, monitoring, and troubleshooting data pipelines using Apache Airflow.
- Experience designing and maintaining data structures, transformations, and workflows in Snowflake.
- Proficiency in Python or a comparable scripting language for automation and data processing.
- Understanding of relational databases, data modeling, data integration patterns, and data quality practices.
- Experience with version control tools such as Git and collaborative software development practices.
- Strong analytical, troubleshooting, and problem-solving skills.
- Ability to work independently on assigned engineering tasks while collaborating effectively with technical teams and stakeholders.
- Must live in the Continental US.
- Be either a US citizen or Permanent Resident due to the work we do.
- Depending on agency requirements, you may need to possess or obtain a Public Trust designation or security clearance. Regardless of the specific requirement, all positions require the successful completion of a background check, background investigation, and the standard onboarding process.
Preferred Qualifications
- Experience supporting data engineering solutions in a government contracting, healthcare (CMS (Center of Medicare & Medicaid) or in the health industry, or other regulated environment.
- Experience with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform.
- Familiarity with cloud-based storage services and data ingestion tools.
- Experience implementing CI/CD pipelines and automated testing for data workflows.
- Knowledge of data governance, data lineage, metadata management, and role-based access controls.
- Experience working with REST APIs, JSON, and other structured or semi-structured data formats.
- Familiarity with containerization technologies such as Docker.
- Qualified individuals living in HUBZone-certified areas are strongly encouraged to apply. To find out if you live in a HUBZone go to: https://maps.certify.sba.gov/hubzone/map#center=44.722800,-103.249700&zoom=4 .
The likely salary range for this position is $95,000-$106,000. This is not, however, a guarantee of compensation or salary. Rather, salary will be set based on experience, geographic location and possibly contractual requirements and could fall outside of this range.