Staff Software Engineer, Network Observability

TeslaFremont, CaliforniaOn-siteFull-timeStaff, 8–12 yearsListed 3 hours ago

Apply now

About this role

You will design and build the backbone of Tesla's network monitoring platform, the systems that collect, store, query, and act on telemetry from tens of thousands of network devices across global sites. Metrics pipelines, alerting, device inventory, config management, and the APIs that feed both dashboards and our AI diagnostic system; you own all of it.

This platform runs 24/7. When it's down, infastructure is running blind . You'll own the architecture, the uptime, and the scaling challenges that come with it.
You will work closely with network engineering to understand operational needs, and with the AI platform team to ensure monitoring data is structured and accessible for intelligent diagnostics.

- Build metrics collection pipelines that pull data from thousands of devices: polling, streaming, APIs and land it in time series and log storage
- Own the device inventory and topology system. Discovery, reconciliation, config tracking, change detection, kept accurate at scale
- Build and run the alerting stack. Threshold checks, baseline comparisons, anomaly detection, catching problems before they cascade
- Design the APIs that expose device state, metrics, alerts, and relationships to dashboards, CLI tools, and AI agents
- Build config management and automation tooling. Generate configs from templates, validate them, push them, roll back when things go wrong
- Keep the monitoring platform itself healthy. Capacity planning, fault tolerance, storage lifecycle, data retention. Monitor the monitors
- Make sure telemetry data flows cleanly to everything downstream, Grafana, reports, the AI diagnostic engine
- Lead projects end-to-end. Design docs, code reviews, architecture decisions, mentoring. You're a technical leader, not just a contributor

- 8+ years building and running production infrastructure software. Not prototypes; things that page you at 2am when they break
- Experience in Go, Java, C++, Rust, or Python. You've built services that sustain high throughput ingest and serve queries under load
- Distributed systems are second nature; you know the real world tradeoffs of consistency, replication, partitioning, and backpressure
- You've worked with time series databases (Prometheus, Mimir, InfluxDB, VictoriaMetrics) and understand what high cardinality metrics do to storage and query performance
- You've worked with MetricWeave or similar telemetry normalization frameworks for abstracting multi-vendor device data
- You've operated log aggregation at scale, Splunk, Elasticsearch, Loki, or similar
- You're comfortable owning production, CI/CD, containers, Kubernetes, and instrumenting your own services so you know when they're sick
- You can whiteboard a collection pipeline for 100K devices, point to where it'll fall over, and design around it
- You've led technical projects from "we should build this" through to production and iteration

Benefits

Along with competitive pay, as a full-time Tesla employee, you are eligible for the following benefits at day 1 of hire:

- Medical plans > plan options with $0 payroll deduction
- Family-building, fertility, adoption and surrogacy benefits
- Dental (including orthodontic coverage) and vision plans, both have options with a $0 paycheck contribution
- Company Paid (Health Savings Accounts) HSA Contribution when enrolled in the High-Deductible medical plan with HSA
- Healthcare and Dependent Care Flexible Spending Accounts (FSA)
- 401(k) with employer match, Employee Stock Purchase Plans, and other financial benefits
- Company paid Basic Life, AD&D
- Short-term and long-term disability insurance (90 day waiting period)
- Employee Assistance Program
- Sick and Vacation time (Flex time for salary positions, Accrued hours for Hourly positions), and Paid Holidays
- Back-up childcare and parenting support resources
- Voluntary benefits to include: critical illness, hospital indemnity, accident insurance, theft & legal services, and pet insurance
- Weight Loss and Tobacco Cessation Programs
- Tesla Babies program
- Commuter benefits
- Employee discounts and perks program
Expected Compensation $120,000 - $396,000/annual salary + cash and stock awards + benefits
Pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. The total compensation package for this position may also include other elements dependent on the position offered. Details of participation in these benefit plans will be provided if an employee receives an offer of employment.