Lead Infrastructure Engineer

Job Description

If you are a hands-on infrastructure engineering leader who thrives at the intersection of platform reliability, automation, and technical strategy, this role offers the scope and scale to make a lasting impact. You will own one of JPMorganChase's most critical observability platforms end to end — setting direction, driving cost efficiency, and staying credible in a terminal and a code review. This is a rare opportunity to lead at a senior level while remaining deeply technical, with visibility across engineering, security, and infrastructure teams.

As a Sr Manager of Infrastructure Engineering at JPMorganChase within the Corporate Technology Observability and Logging Platforms team, you will serve as the senior technical authority for our enterprise Splunk estate — a large, multi-cluster deployment running on AWS and managed entirely as code through SaltStack. You will lead platform architecture, capacity planning, incident root-cause analysis, and the automation tooling that keeps the estate performant, predictable, and cost-efficient at scale. You will partner with application, security, and infrastructure teams while growing and mentoring a team of platform engineers.

Job responsibilities

- Define and own the architecture, availability, performance, and recovery targets for indexer clusters, search head clusters, forwarders, HTTP Event Collector ingestion, deployment server and deployer, and SmartStore storage
- Lead recurring capacity assessments across clusters — covering ingestion volume, indexer compute and memory load, disk input/output and space, indexing queues, pipeline processors, and search load — producing severity-graded findings and remediation matrices
- Own the SaltStack state tree that configures and deploys the entire Splunk estate, driving safe, reviewable, branch-based change management with pillar-driven configuration and repeatable cluster operations
- Build and maintain Python automation for operational tasks, reporting, and integrations, including modernization of legacy tooling such as version migrations, dependency hygiene, and authentication flows
- Lead deep root-cause investigations across the estate, correctly attributing write-path versus read-path pressure and search-head-dispatch versus indexer-execution behavior
- Use Splunk internal, introspection, metrics, audit, and license data to understand and forecast platform behavior; build dashboards and reporting that make capacity and health decisions data-driven
- Establish and enforce logging standards, onboarding patterns, retention policies, and scheduled-search hygiene including concurrency and skip-ratio controls
- Right-size infrastructure and reduce license and compute spend without compromising reliability or platform availability
- Grow and mentor a team of platform engineers, set technical strategy and roadmap, and represent the platform to senior stakeholders and partner teams

Required qualifications, capabilities and skills

- Formal training or certification on software engineering concepts and 5+ years applied experience. In addition, 2+ years of experience leading technologists to manage and solve complex technical items within your domain of expertise
- Deep, hands-on operational experience running Splunk at scale, including distributed indexer clusters, search head clusters, HTTP Event Collector, forwarders, SmartStore, knowledge objects, and scheduled search management
- Expert-level proficiency in Splunk Processing Language, including use of tstats and metadata searches and analysis of internal, introspection, audit, and license data for platform diagnostics
- Expert-level experience managing infrastructure fleets with SaltStack, including states, pillars, Jinja templating, gitfs with saltenv and pillarenv, and safe change rollout practices
- Strong Python development experience for production automation, tooling, and integrations, including maintaining and modernizing existing codebases
- Strong experience running production workloads on AWS, including compute, S3 storage (including SmartStore remote storage), identity and access management, networking, and cost management across compute and storage
- Proven track record in incident management, root-cause analysis, and producing clear, actionable technical assessments for both engineering teams and senior leadership
- Experience leading engineering teams, owning a platform roadmap, and maintaining hands-on technical depth alongside leadership responsibilities

Preferred qualifications, capabilities and skills

- Experience reducing Splunk license and ingest costs through logging optimization and source-side volume reduction strategies
- Familiarity with artificial intelligence-assisted operational tooling and chat operations integrations for platform automation
- Experience with continuous integration tooling such as Jenkins, test automation frameworks such as pytest, and build workflow tooling
- Security and compliance exposure in a regulated enterprise environment, including authentication flows such as Security Assertion Markup Language, OpenID Connect, and single sign-on troubleshooting
- Experience building self-service dashboards and reporting for platform capacity, health, and cost visibility