**Principal Software Engineer - Telemetry/Observability**
As a **Principal Software Engineer**, you'll lead telemetry/observability engineering for JPMorganChase. Responsibilities include creating scalable coding frameworks, building secure production-level code, and designing end-to-end telemetry pipelines. You'll leverage advanced Python skills and strong Linux systems knowledge to diagnose cross-system issues in distributed environments.
This senior role requires at least 10 years of software engineering experience, with a focus on platform services in production environments. Deep expertise in Python development for telemetry and operations workflows is expected. Preferred candidates will have experience with OpenTelemetry, SRE principles, and infrastructure as code.
Collaborate with agile teams to drive best-in-class outcomes, enhance internal product portfolios, and provide technical expertise across the organization. Foster a culture of diversity, opportunity, and respect.
Full Job Description
Location: Seattle, WA, United States
If you are looking for a game-changing career, working for one of the world's leading financial institutions, youve come to the right place.
As a Principal Software Engineer at JPMorganChase within the Core Foundational Platforms, you provide expertise and engineering excellence as an integral part of an agile team to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. Leverage your advanced technical capabilities and collaborate with colleagues across the organization to drive best-in-class outcomes across various technologies to support one or more of the firms portfolios.
Job responsibilities
Creates complex and scalable coding frameworks using appropriate software design frameworks, with a focus on observability/telemetry platforms for Virtual Private Cloud (VPC) environmentsDevelops secure and high-quality production code, and reviews and debugs code written by others across telemetry collection, enrichment, storage, and visualization servicesLeads the design and implementation of end-to-end telemetry pipelines (metrics, logs, traces, events) including standards, schemas, and instrumentation patterns for platform and application teamsEngineers and evolves CI/CD pipelines to support reliable delivery, automated testing, policy-as-code, and safe progressive rollouts for observability servicesDesigns and implements solutions that operate effectively in software-defined networking (SDN) and cloud networking environments, including troubleshooting across layered network paths and dependenciesBuilds and maintains monitoring, alerting, and SLO/SLI practices using tools such as Splunk and/or Grafana and/or Prometheus (including dashboarding, alert tuning, and operational runbooks)Advises cross-functional teams on technological matters within your domain of expertise, including telemetry strategy, platform integrations, and operational readinessServes as the functions go-to subject matter expert for observability and telemetry across the VPC platform, influencing standards and best practicesCreates durable, reusable software frameworks that are leveraged across teams and functions (SDKs, libraries, templates, reference architectures, and golden paths)Influences leaders and senior stakeholders across business, product, and technology teams by communicating tradeoffs, risk, and roadmap decisions for platform reliability and resiliencyChampions the firms culture of diversity, opportunity, inclusion, and respectRequired qualifications, capabilities, and skills
10+ years of software engineering experience, including building and operating platform services in production environmentsHands-on practical experience delivering system design, application development, testing, and operational stability, ideally for platform/infrastructure or developer productivity productsExpert-level Python development experience, including building services, automation, tooling, and integrations for telemetry and operations workflowsStrong Linux systems and networking expertise (TCP/IP, DNS, TLS, routing, NAT, packet capture/troubleshooting), and the ability to diagnose cross-system issues in distributed environmentsPractical experience with CI/CD (e.g., automated build/test/release pipelines) and modern delivery practices (progressive delivery, automated quality gates, artifact/version management)Advanced knowledge of software application development and technical processes with considerable in-depth knowledge in one or more technical disciplines (e.g., cloud platforms, platform engineering, reliability engineering, networking, observability)Experience with observability/telemetry tooling such as Splunk and/or Grafana and/or Prometheus (including instrumentation, queries, dashboards, alerts, and operationalization)Experience designing or operating telemetry data pipelines (collection agents, exporters, scraping, indexing, retention, cardinality management, performance/cost controls)Experience applying expertise and new methods to determine solutions for complex technology problems in one or more technical disciplinesWorking knowledge of software-defined networking concepts and environments, including integrations and operational considerations for VPC-like network platformsPreferred qualifications, capabilities, and skills
Experience building an observability platform or paved road telemetry offering used by multiple engineering teams (multi-tenant design, onboarding patterns, guardrails)Experience with OpenTelemetry (instrumentation standards, collectors, exporters) and distributed tracing practicesStrong understanding of SRE principles (SLOs/SLIs, error budgets, incident response, post-incident reviews, toil reduction)Experience with infrastructure as code and GitOps-style workflows (e.g., Terraform-like patterns, configuration as code, policy enforcement)Experience supporting regulated environments with strong security and control requirements (secure-by-default design, auditability, change management) Provide expertise and engineering excellence to enhance, build and deliver market-leading technologies within the firm