LOG IN
SIGN UP
Canary Wharfian - Online Investment Banking & Finance Community.
Sign In
Forgot password?
Don't have an account?
or
Join Canary Wharfian
By signing up, you agree to our Terms & Conditions and Privacy Policy.
or

Senior Data Engineer

ExperiencedNo visa sponsorship
Ernst & Young logo

at Ernst & Young

Big Four

Posted 15 days ago

No clicks

**Senior Data Engineer** Design and manage large-scale data infrastructure, fostering business growth through insightful data solutions. Key responsibilities involve architecting complex data structures, fostering collaborative cross-functional teams, and applying technical prowess to solve challenging business problems. Proficient in **Python development**, **data processing & analytics** (Pandas, Polars), and **Azure data engineering** (Azure Databricks, Data Factory, Synapse Analytics). Strong foundation in **Apache Spark (PySpark)** and **SQL**, with experience working with diverse databases. Previous exposure to **metadata management**, **NoSQL databases**, and **graph databases** (Neo4j) considered advantageous. Build a **generative AI** platform using RAG and GraphRAG, and drive **DataOps and MLOps** standards on Azure Kubernetes Service (AKS). Demonstrate strong analytical skills, leadership, and a customer-focused approach to thrive in our dynamic, globally connected teams.

Compensation
Not specified

Currency: Not specified

City
Not specified
Country
India

Full Job Description

At EY, youll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And were counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. 

 

 

 

 

The opportunity 

 

Role: Senior Data Engineer

 

 

Your key responsibilities

 

  • Developing large and complex data architecture, composed of models, policies, rules or standards that govern which data is collected and how it is stored, arranged, integrated and put to use in data systems, including the design, build and management of data infrastructure to address business requirements
  • Developing relationships across the business to understand data requirements, applies deep technical knowledge of data management to solve business problems in areas where solutions may not currently exist, necessitating new solutions/ways of working/technologies and proactively articulating these to the business
  • Collaborating and providing innovative and practical designs for the design and integration of new data architecture for the enterprise, applying sophisticated technical capabilities
  • Creating, maintaining and supervising on-premise or cloud infrastructure, such as the design of end-to-end data platforms, development and operations of cloud management and design of distributed systems, handling structured and unstructured data
  • Working across multidisciplinary teams and with the business to define and delivering analytics solutions that will deliver business value
  • Contributing to data management standards to promote optimization and consistency

 

 

Required Skills (MUST HAVE)

 

Python Development

 

  • Strong Python programming skills with the ability to design, develop, test, and deploy end-to-end applications and data processing solutions.
  • Proficiency in object-oriented programming, data structures, design patterns, and asynchronous programming.
  • Strong understanding of Python language features including classes, inheritance, collections, decorators, context managers, generators, exception handling, package management, and virtual environments.
  • Experience developing scalable, maintainable, and production-grade applications following coding standards and best practices.

 

Data Processing & Analytics

 

  • Hands-on experience with data processing frameworks such as Pandas and Polars.
  • Ability to perform medium to complex data transformations, data cleansing, aggregation, enrichment, and optimization of large datasets.
  • Strong understanding of memory-efficient data processing techniques and performance tuning.

 

Azure Data Engineering

 

  • Experience designing and implementing data engineering solutions on Azure platforms including Azure Databricks, Azure Data Factory (ADF), Azure Synapse Analytics, Azure Data Lake Storage (ADLS), Azure Service Bus, and Azure Event Hub.
  • Ability to translate business requirements into scalable end-to-end ETL/ELT pipelines.
  • Experience processing batch and streaming data using Kafka, Azure Event Hub, or similar messaging platforms.
  • Understanding of data orchestration, monitoring, and operationalization of data pipelines.

 

Spark & Database Technologies

 

  • Strong foundational knowledge of Apache Spark (PySpark) and SQL, including query optimization and performance tuning.
  • Experience working with relational databases such as Azure SQL, Oracle, DB2, Teradata, PostgreSQL, and SQL Server.
  • Understanding of data modeling, partitioning strategies, indexing, and distributed data processing concepts.

 

 

Additional Skills (GOOD TO HAVE)

 

  • Experience in metadata management and data governance with Unity Catalog or Microsoft Purview. 
  • Experience working with NoSQL databases such as MongoDB, Cassandra, HBase, Couchbase, and Neo4j.
  • Graph Databases: Should have extensive experience designing and modelling enterprise solutions on the Neo4j graph database, including defining optimal graph schemas, relationship structures, and query patterns. Must possess a deep understanding of Neo4j internals and configurations, such as clustering, memory tuning, performance optimization, and operational best practices. Strong proficiency in Cypher is required, along with handson expertise using Neo4j Graph Data Science libraries to build advanced analytics and AIdriven graph solutions. Should be able to define architectural standards, integration patterns, and governance models for scalable, secure graphbased systems.
  • Generative AI: Lead the endtoend architecture, design, and governance of enterprisegrade Generative AI platforms leveraging RAG (Retrieval-Augmented Generation) and GraphRAG methodologies. Must demonstrate deep expertise in designing scalable retrieval and reasoning systems, orchestrating complex AI pipelines using LangChain and LangGraph, and integrating heterogeneous enterprise data ecosystemsincluding vector stores, graph databases, relational systems, and unstructured data repositories. Understanding of Model Context Protocol (MCP) servers, with the ability to define standards, patterns, and modular components that enable secure, extensible AI capabilities across distributed environments. Need to provide technical leadership in defining solution blueprints, ensuring that AI systems meet enterprise benchmarks for scalability, reliability, observability, compliance, and interoperability across business platforms.
  • DataOps and MLOps: Must possess deep expertise in modern microservicesbased design and be capable of engineering robust, scalable deployment patterns leveraging Azure Kubernetes Service (AKS). This includes defining CI/CD standards for data and ML pipelines, enabling reproducibility through containerization, governing model lifecycle management, and establishing best practices for automated testing, versioning, lineage, quality, and observability.

 

 

What we look for

 

  • Strong analytical skills and problem-solving ability
  • A self-starter, independent-thinker, curious and creative person with ambition and passion
  • Excellent inter-personal, communication, collaboration, and presentation skills
  • Customer focused
  • Excellent time leadership skills
  • Positive and constructive minded
  • Takes ownership for continuous self-learning
  • Takes the lead and makes decisions in critical times and tough circumstances
  • Attention to detail
  • High levels of integrity and honesty

 

 

What we offer you

 

At EY, well develop you with future-focused skills and equip you with world-class experiences. Well empower you in a flexible environment, and fuel you and your extraordinary talents in a diverse and inclusive culture of globally connected teams. 

 

 

EY | Building a better working world 

 

EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.

 

Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.

 

Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.  

Senior Data Engineer

Compensation

Not specified

City: Not specified

Country: India

Ernst & Young logo
Big Four

15 days ago

No clicks

at Ernst & Young

ExperiencedNo visa sponsorship

**Senior Data Engineer** Design and manage large-scale data infrastructure, fostering business growth through insightful data solutions. Key responsibilities involve architecting complex data structures, fostering collaborative cross-functional teams, and applying technical prowess to solve challenging business problems. Proficient in **Python development**, **data processing & analytics** (Pandas, Polars), and **Azure data engineering** (Azure Databricks, Data Factory, Synapse Analytics). Strong foundation in **Apache Spark (PySpark)** and **SQL**, with experience working with diverse databases. Previous exposure to **metadata management**, **NoSQL databases**, and **graph databases** (Neo4j) considered advantageous. Build a **generative AI** platform using RAG and GraphRAG, and drive **DataOps and MLOps** standards on Azure Kubernetes Service (AKS). Demonstrate strong analytical skills, leadership, and a customer-focused approach to thrive in our dynamic, globally connected teams.

Full Job Description

At EY, youll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And were counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. 

 

 

 

 

The opportunity 

 

Role: Senior Data Engineer

 

 

Your key responsibilities

 

  • Developing large and complex data architecture, composed of models, policies, rules or standards that govern which data is collected and how it is stored, arranged, integrated and put to use in data systems, including the design, build and management of data infrastructure to address business requirements
  • Developing relationships across the business to understand data requirements, applies deep technical knowledge of data management to solve business problems in areas where solutions may not currently exist, necessitating new solutions/ways of working/technologies and proactively articulating these to the business
  • Collaborating and providing innovative and practical designs for the design and integration of new data architecture for the enterprise, applying sophisticated technical capabilities
  • Creating, maintaining and supervising on-premise or cloud infrastructure, such as the design of end-to-end data platforms, development and operations of cloud management and design of distributed systems, handling structured and unstructured data
  • Working across multidisciplinary teams and with the business to define and delivering analytics solutions that will deliver business value
  • Contributing to data management standards to promote optimization and consistency

 

 

Required Skills (MUST HAVE)

 

Python Development

 

  • Strong Python programming skills with the ability to design, develop, test, and deploy end-to-end applications and data processing solutions.
  • Proficiency in object-oriented programming, data structures, design patterns, and asynchronous programming.
  • Strong understanding of Python language features including classes, inheritance, collections, decorators, context managers, generators, exception handling, package management, and virtual environments.
  • Experience developing scalable, maintainable, and production-grade applications following coding standards and best practices.

 

Data Processing & Analytics

 

  • Hands-on experience with data processing frameworks such as Pandas and Polars.
  • Ability to perform medium to complex data transformations, data cleansing, aggregation, enrichment, and optimization of large datasets.
  • Strong understanding of memory-efficient data processing techniques and performance tuning.

 

Azure Data Engineering

 

  • Experience designing and implementing data engineering solutions on Azure platforms including Azure Databricks, Azure Data Factory (ADF), Azure Synapse Analytics, Azure Data Lake Storage (ADLS), Azure Service Bus, and Azure Event Hub.
  • Ability to translate business requirements into scalable end-to-end ETL/ELT pipelines.
  • Experience processing batch and streaming data using Kafka, Azure Event Hub, or similar messaging platforms.
  • Understanding of data orchestration, monitoring, and operationalization of data pipelines.

 

Spark & Database Technologies

 

  • Strong foundational knowledge of Apache Spark (PySpark) and SQL, including query optimization and performance tuning.
  • Experience working with relational databases such as Azure SQL, Oracle, DB2, Teradata, PostgreSQL, and SQL Server.
  • Understanding of data modeling, partitioning strategies, indexing, and distributed data processing concepts.

 

 

Additional Skills (GOOD TO HAVE)

 

  • Experience in metadata management and data governance with Unity Catalog or Microsoft Purview. 
  • Experience working with NoSQL databases such as MongoDB, Cassandra, HBase, Couchbase, and Neo4j.
  • Graph Databases: Should have extensive experience designing and modelling enterprise solutions on the Neo4j graph database, including defining optimal graph schemas, relationship structures, and query patterns. Must possess a deep understanding of Neo4j internals and configurations, such as clustering, memory tuning, performance optimization, and operational best practices. Strong proficiency in Cypher is required, along with handson expertise using Neo4j Graph Data Science libraries to build advanced analytics and AIdriven graph solutions. Should be able to define architectural standards, integration patterns, and governance models for scalable, secure graphbased systems.
  • Generative AI: Lead the endtoend architecture, design, and governance of enterprisegrade Generative AI platforms leveraging RAG (Retrieval-Augmented Generation) and GraphRAG methodologies. Must demonstrate deep expertise in designing scalable retrieval and reasoning systems, orchestrating complex AI pipelines using LangChain and LangGraph, and integrating heterogeneous enterprise data ecosystemsincluding vector stores, graph databases, relational systems, and unstructured data repositories. Understanding of Model Context Protocol (MCP) servers, with the ability to define standards, patterns, and modular components that enable secure, extensible AI capabilities across distributed environments. Need to provide technical leadership in defining solution blueprints, ensuring that AI systems meet enterprise benchmarks for scalability, reliability, observability, compliance, and interoperability across business platforms.
  • DataOps and MLOps: Must possess deep expertise in modern microservicesbased design and be capable of engineering robust, scalable deployment patterns leveraging Azure Kubernetes Service (AKS). This includes defining CI/CD standards for data and ML pipelines, enabling reproducibility through containerization, governing model lifecycle management, and establishing best practices for automated testing, versioning, lineage, quality, and observability.

 

 

What we look for

 

  • Strong analytical skills and problem-solving ability
  • A self-starter, independent-thinker, curious and creative person with ambition and passion
  • Excellent inter-personal, communication, collaboration, and presentation skills
  • Customer focused
  • Excellent time leadership skills
  • Positive and constructive minded
  • Takes ownership for continuous self-learning
  • Takes the lead and makes decisions in critical times and tough circumstances
  • Attention to detail
  • High levels of integrity and honesty

 

 

What we offer you

 

At EY, well develop you with future-focused skills and equip you with world-class experiences. Well empower you in a flexible environment, and fuel you and your extraordinary talents in a diverse and inclusive culture of globally connected teams. 

 

 

EY | Building a better working world 

 

EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.

 

Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.

 

Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.