
at J.P. Morgan
Bulge Bracket Investment BanksPosted 8 days ago
No clicks
**Lead Software Engineer - Data Engineer** Build and lead data engineering solutions at JPMorgan Chase. Automate data operations using AI-driven workflows in AWS cloud (S3, Lambda, EKS, Step Functions). Design and deploy scalable ingest-to-analytics data pipelines using Apache Spark, Kafka, Flink, and Iceberg. Implement Infrastructure as Code (IaC), drive team's adoption of AI-assisted practices, and ensure data quality and security. Requires 5+ years of experience, formal software engineering training, and deep expertise in data ingestion, transformation, and analytics using relevant tools and AWS services.
- Compensation
- Not specified
- City
- Not specified
- Country
- India
Currency: Not specified
Full Job Description
Location: Hyderabad, Telangana, India
Job responsibilities
- Build automated, self-healing data operations using agentic frameworks and AI-driven workflows for intelligent monitoring, optimization, and remediation
- Design and deploy data solutions using AWS services including S3, Lambda, EKS (Elastic Kubernetes Service), and Step Functions
- Implement Infrastructure as Code (IaC) practices for reproducible and scalable deployments
- Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team.
- Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.
- Build and optimize scalable data ingestion, transformation, and analytics solutions using Apache Spark, AWS Glue, and Apache Iceberg
- Develop and manage both batch and real-time data pipelines leveraging Apache Kafka and Apache Flink for streaming data processing. Implement Data Lake architectures with efficient data organization, partitioning, and governance strategies
- Design and architect end-to-end data solutions from ingestion to analytics, ensuring scalability, reliability, and performance at enterprise scale
- Define data architecture standards and best practices for batch and real-time data processing pipelines
- Formal training or certification on software engineering concepts and 5+ years applied experience
- Demonstrated experience leading effective use of approved AI-assisted software development tools (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security.
- Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching engineers on safe, compliant adoption within delivery practices
- Experience with modern data engineering practices including DataOps, data observability, and data governance
- Deep expertise in data ingestion, transformation, and analytics using Apache Spark, AWS Glue, Apache Iceberg, and Data Lake architectures
- Proven experience designing and managing batch and real-time data pipelines with Apache Kafka and Apache Flink
- Strong hands-on experience with AWS cloud services including S3, Lambda, EKS, and Step Functions
- Experience building automated solutions using agentic frameworks and AI-driven workflows for intelligent data operations
- Advanced proficiency in programming languages such as Python, Scala, or Java with strong SQL skills
- Demonstrated ability to architect end-to-end data solutions from ingestion to analytics at scale




