Job Title:

Contract Type:

Time Type:

Job Description:

As a Senior Data Engineer specializing in Natural Gas and Power, you will support the design, development, and optimization of advanced data infrastructure and products that support critical trading, risk, and market analytics functions.

You will collaborate closely with traders, quantitative analysts, risk managers, and IT stakeholders to ensure data systems are scalable, secure, and aligned with the strategic goals of the energy business. In addition to hands-on engineering, you will play a pivotal role supporting the data architecture roadmap, mentoring engineers, and driving innovation in data-driven decision-making across the Natural Gas and Power value chain. This role reports to the Data Engineer Lead.

Main Responsibilities

Data Engineering Leadership

• Contribute to the definition of the target data architecture (lake/lakehouse, streaming, event-driven) and technology choices for high-volume time-series, market, and transactional data.

• Build and operate end-to-end pipelines (ingest quality transform serve) with strong SLAs/SLOs, lineage, and observability.

• Establish coding, testing, CI/CD, and infrastructure-as-code standards; drive adoption of data governance, cataloging, and access controls.

• Own capacity, performance, reliability, and cost efficiency of data platforms across environments (dev/test/prod).

• Ensure data quality, lineage, and observability across all layers of the data stack.

Strategic Collaboration & Business Alignment

• Delivery high quality code efficiently.

• Partner with trading desks, quantitative teams, and risk functions to translate business needs into data solutions that enhance decision-making and operational efficiency.

• Act as a senior liaison between engineering and business stakeholders, ensuring alignment on data priorities and delivery timelines.

• Prioritize a value-based backlog (e.g., faster close/settlement, improved forecast accuracy, reduced balancing penalties) and measure business impact.

• Align data models and domain ownership with business processes (bids/offers, nominations, positions, exposures,outages).

• Coordinate with Cybersecurity, Compliance, and Legal on sector-specific controls (e.g., REMIT/NERC-CIP considerations, data retention, segregation).

Innovation & Product Development

• Incubate and industrialize data products: curated marts, feature stores, real-time decision APIs, and event streams for forecasting and optimization.

• Introduce modern patterns (CDC, schema evolution, Delta/Iceberg, streambatch unification) to improve freshness and resilience.

• Evaluate and integrate external data (weather, fundamentals, congestion, capacity postings), internal and external vendor systems (ETRM) safely and at scale.

• Collaborate with quantitative analysts to productionize ML pipelines (forecasting load/renewables, anomaly detection, etc.. ) with monitoring and rollback.

Mentorship & Technical Oversight

• Lead incident reviews and architectural forums; provide pragmatic guidance on trade-offs (latency vs. cost, simplicity vs. flexibility).

• Develop growth paths and learning plans focused on energy domain fluency and modern data engineering practices.

Operational Excellence

• Implement robust monitoring/alerting, runbooks, and on-call rotations; drive MTTR down and availability up for critical data services.

• Enforce data quality contracts (SLAs/SLOs), lineage, and reconciliation for market submissions, settlements, and reporting.

• Optimize cloud spend and storage/compute footprints; plan capacity for market events and seasonal peaks.

• Ensure security and compliance by design: least-privilege access, secrets management, encryption, auditability, and disaster recovery testing.

Profile

• Masters or Bachelors degree in Computer Science, Data Engineering, Applied Mathematics, or a related technical field.

• 5+ years of experience in data engineering, with at least 3 years in a senior role.

• Proven experience in the energy trading sector, ideally with exposure to Natural Gas and Power markets, balancing mechanisms, and regulatory frameworks (e.g., REMIT, EMIR).

• Strong experience building and operating data platforms on AWS is required.

• Azure: ADLS Gen2, Event Hubs, Synapse Analytics, Azure Databricks (Spark), Azure Functions, Azure Data

• Factory/Databricks Workflows, Key Vault, Azure Monitoring/Log Analytics; IaC with Terraform/Bicep; CI/CD with Azure DevOps or GitHub Actions.

• Snowflake (on Azure or multi-cloud): Warehousing design, Streams & Tasks, Snowpipe/Snowpipe Streaming, Time

• Travel & Fail-safe, RBAC & row/column security, external tables over ADLS, performance tuning & cost governance.

• Programming & Engineering Practices: Strong OOP in Python and Java/Scala; SDLC leadership, DevOps mindset, TDD/BDD, code reviews, automated testing (unit/integration/contract), packaging and dependency management, API design (REST/gRPC).

• Web & Data Acquisition: Robust web scraping and ingestion with Scrapy, Requests, Playwright/Selenium; scheduling, retries/exponential backoff, change-data capture; ethical/legal collection practices (robots.txt, terms).

• Orchestration & Quality: Airflow/ADF/Databricks Jobs, data contracts, Great Expectations (or similar), lineage/catalog (e.g., Purview), metrics/observability (Prometheus/Grafana/Application Insights).

• Dataframe oriented programming: pandas, spark/snowpark dataframes, SQL data transformation

• Additional Skills

• Designing low-latency pipelines for sub-second to minute-level telemetry, weather and market data; tuning Spark

• Structured Streaming/Flink/Kafka Streams.

• Quality & Reconciliation for telemetry and market submissions (gap fill, resampling, deduplication, anomaly detection, schema evolution).

• Serving Patterns: time-series stores and query layers (e.g., Delta Lake over ADLS, Iceberg, materialized views in Snowflake), APIs and event streams for downstream consumption.

• English (fluent), any additional language is an asset

Please submit a concise CV (maximum 2 pages) in English, clearly stating your current work authorization status in the U.S. and whether you require visa sponsorship now or in the future.

Please note that this is a fully on-site position.

If you think the open position you see is right for you, we encourage you to apply!

Our people make all the difference in our success.

Location: Houston