LOG IN
SIGN UP
Canary Wharfian - Online Investment Banking & Finance Community.
Sign In
Forgot password?
Don't have an account?
or
Join Canary Wharfian
By signing up, you agree to our Terms & Conditions and Privacy Policy.
or

Lead Data Engineer

ExperiencedNo visa sponsorship

Posted a month ago

No clicks

**Lead Data Engineer** Lead a critical role in our billing transformation program, converting raw telemetry usage data into rated, validated, and invoice-ready datasets. Manage data ingestion pipelines, implement rating logic, and ensure data accuracy through comprehensive quality checks. Key skills: AWS Glue, Snowflake, Python, SQL, CI/CD, and usage-based billing. Bring 8+ years of experience, excellent problem-solving skills, and collaboration prowess to this senior associate level role.

Compensation
Not specified

Currency: Not specified

City
Not specified
Country
India

Full Job Description

Role Overview:

We are seeking an experienced Lead Data Engineer to join our billing transformation program. The engineer will be responsible for sourcing telemetry data, building scalable data pipelines, supporting rating logic, and preparing billing-ready outputs for SAP invoicing.

The role requires strong experience with AWS Glue, Snowflake, Python, SQL, and CI/CD practices. The candidate will also be responsible for implementing data quality checks, reconciliation controls, audit trails, and operational monitoring to ensure billing accuracy.

This is a critical role supporting a transformation from raw telemetry usage data to rated, validated, and invoice-ready billing data.

Key Responsibilities:

Data ingestion pipelines from telemetry systems.

Curated usage datasets in Snowflake.

Rating logic implementation and billing-ready datasets.

SAP invoice feed generation.

Data quality and reconciliation framework.

Exception and error-handling process.

CI/CD pipelines for data engineering deployments.

Monitoring, alerting, and operational dashboards.

Technical documentation and runbooks.

Required Skills:

Strong hands-on experience in data engineering, ETL/ELT development, and large-scale data pipeline design.

Experience with AWS Glue, PySpark, Python, SQL, and cloud-based data processing.

Strong experience working with Snowflake as a data warehouse or enterprise data platform.

Good understanding of usage-based billing, rating engines, invoice feeds, or financial data processing.

Experience building data quality checks, reconciliation frameworks, audit controls, and exception handling processes.

Hands-on experience with CI/CD tools such as GitHub Actions, GitLab CI, Jenkins, AWS Code Pipeline, or similar.

Strong SQL skills for data analysis, validation, troubleshooting, and performance optimization.

Experience working with batch and event-driven data ingestion patterns.

Understanding of data modelling, partitioning, performance tuning, and cost optimization.

Soft Skills:

Strong analytical skills and attention to detailespecially important for financial accuracy.

Ability to collaborate with cross-functional teams including Finance, Engineering, and Analytics.

Good communication skills to explain defects, logic, and test outcomes clearly.

High ownership and adaptability, with willingness to take on both QA and minor development tasks.

Benefits

We are an equal opportunity employer with hybrid working.

Annual Leave and Parental Leave benefits.

Upskilling opportunities with job-relevant training and certifications.

Dedicated Employee Assistance and Wellbeing support.

What youll get in return

High-impact projects: We work on a variety of pioneering AI products and leverage extensive datasets to solve complex, high-value challenges.

Competitive benefits: You will enjoy a strong compensation package, comprehensive benefits, and ongoing investment in career growth and skill development.

Industry leadership: This is an opportunity to be a founding member of the organization thats delivering brand-new products that democratize modeling and analytics solutions.

Collaborative environment: We provide opportunities for continuous learning and professional development in a work environment of dedicated, highly experienced teams.

We recognize that to attract the best talent, we need to be flexible, and we are open to discussing work arrangements with you. We take a hybrid approach to the workplace; this role is considered Blended, which requires attending the office at least three day per week while some teams and colleagues choose to collaborate in the office more frequently.

We're proud to have been recognised as a Great Place to Work in India 25

Learn more about life and purpose of our company directly from India colleagues video: Bengaluru, India | Where We Work | LSEG

Career Stage:

Senior Associate

London Stock Exchange Group (LSEG) Information:

Join us and be part of a team that values innovation, quality, and continuous improvement. If you're ready to take your career to the next level and make a significant impact, we'd love to hear from you.

LSEG is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth.

Our purpose is the foundation on which our culture is built. Our values of Integrity, Partnership, Excellence and Change underpin our purpose and set the standard for everything we do, every day. They go to the heart of who we are and guide our decision making and everyday actions.

Working with us means that you will be part of a dynamic organisation of 25,000 people across 65 countries. However, we will value your individuality and enable you to bring your true self to work so you can help enrich our diverse workforce.

We are proud to be an equal opportunities employer. This means that we do not discriminate on the basis of anyones race, religion, colour, national origin, gender, sexual orientation, gender identity, gender expression, age, marital status, veteran status, pregnancy or disability, or any other basis protected under applicable law. Conforming with applicable law, we can reasonably accommodate applicants' and employees' religious practices and beliefs, as well as mental health or physical disability needs.

You will be part of a collaborative and creative culture where we encourage new ideas. We are committed to sustainability across our global business and we are proud to partner with our customers to help them meet their sustainability objectives. Our charity, the LSEG Foundation provides charitable grants to community groups that help people access economic opportunities and build a secure future with financial independence. Colleagues can get involved through fundraising and volunteering.

LSEG offers a range of tailored benefits and support, including healthcare, retirement planning, paid volunteering days and wellbeing initiatives.

Please take a moment to read this privacy notice carefully, as it describes what personal information London Stock Exchange Group (LSEG) (we) may hold about you, what its used for, and how its obtained, your rights and how to contact us as a data subject.

If you are submitting as a Recruitment Agency Partner, it is essential and your responsibility to ensure that candidates applying to LSEG are aware of this privacy notice.

Location: IND-Hyderabad-CapitaLand

Time Type: Full time

Lead Data Engineer

Compensation

Not specified

City: Not specified

Country: India

London Stock Exchange logo
Other

a month ago

No clicks

at London Stock Exchange

ExperiencedNo visa sponsorship

**Lead Data Engineer** Lead a critical role in our billing transformation program, converting raw telemetry usage data into rated, validated, and invoice-ready datasets. Manage data ingestion pipelines, implement rating logic, and ensure data accuracy through comprehensive quality checks. Key skills: AWS Glue, Snowflake, Python, SQL, CI/CD, and usage-based billing. Bring 8+ years of experience, excellent problem-solving skills, and collaboration prowess to this senior associate level role.

Full Job Description

Role Overview:

We are seeking an experienced Lead Data Engineer to join our billing transformation program. The engineer will be responsible for sourcing telemetry data, building scalable data pipelines, supporting rating logic, and preparing billing-ready outputs for SAP invoicing.

The role requires strong experience with AWS Glue, Snowflake, Python, SQL, and CI/CD practices. The candidate will also be responsible for implementing data quality checks, reconciliation controls, audit trails, and operational monitoring to ensure billing accuracy.

This is a critical role supporting a transformation from raw telemetry usage data to rated, validated, and invoice-ready billing data.

Key Responsibilities:

Data ingestion pipelines from telemetry systems.

Curated usage datasets in Snowflake.

Rating logic implementation and billing-ready datasets.

SAP invoice feed generation.

Data quality and reconciliation framework.

Exception and error-handling process.

CI/CD pipelines for data engineering deployments.

Monitoring, alerting, and operational dashboards.

Technical documentation and runbooks.

Required Skills:

Strong hands-on experience in data engineering, ETL/ELT development, and large-scale data pipeline design.

Experience with AWS Glue, PySpark, Python, SQL, and cloud-based data processing.

Strong experience working with Snowflake as a data warehouse or enterprise data platform.

Good understanding of usage-based billing, rating engines, invoice feeds, or financial data processing.

Experience building data quality checks, reconciliation frameworks, audit controls, and exception handling processes.

Hands-on experience with CI/CD tools such as GitHub Actions, GitLab CI, Jenkins, AWS Code Pipeline, or similar.

Strong SQL skills for data analysis, validation, troubleshooting, and performance optimization.

Experience working with batch and event-driven data ingestion patterns.

Understanding of data modelling, partitioning, performance tuning, and cost optimization.

Soft Skills:

Strong analytical skills and attention to detailespecially important for financial accuracy.

Ability to collaborate with cross-functional teams including Finance, Engineering, and Analytics.

Good communication skills to explain defects, logic, and test outcomes clearly.

High ownership and adaptability, with willingness to take on both QA and minor development tasks.

Benefits

We are an equal opportunity employer with hybrid working.

Annual Leave and Parental Leave benefits.

Upskilling opportunities with job-relevant training and certifications.

Dedicated Employee Assistance and Wellbeing support.

What youll get in return

High-impact projects: We work on a variety of pioneering AI products and leverage extensive datasets to solve complex, high-value challenges.

Competitive benefits: You will enjoy a strong compensation package, comprehensive benefits, and ongoing investment in career growth and skill development.

Industry leadership: This is an opportunity to be a founding member of the organization thats delivering brand-new products that democratize modeling and analytics solutions.

Collaborative environment: We provide opportunities for continuous learning and professional development in a work environment of dedicated, highly experienced teams.

We recognize that to attract the best talent, we need to be flexible, and we are open to discussing work arrangements with you. We take a hybrid approach to the workplace; this role is considered Blended, which requires attending the office at least three day per week while some teams and colleagues choose to collaborate in the office more frequently.

We're proud to have been recognised as a Great Place to Work in India 25

Learn more about life and purpose of our company directly from India colleagues video: Bengaluru, India | Where We Work | LSEG

Career Stage:

Senior Associate

London Stock Exchange Group (LSEG) Information:

Join us and be part of a team that values innovation, quality, and continuous improvement. If you're ready to take your career to the next level and make a significant impact, we'd love to hear from you.

LSEG is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth.

Our purpose is the foundation on which our culture is built. Our values of Integrity, Partnership, Excellence and Change underpin our purpose and set the standard for everything we do, every day. They go to the heart of who we are and guide our decision making and everyday actions.

Working with us means that you will be part of a dynamic organisation of 25,000 people across 65 countries. However, we will value your individuality and enable you to bring your true self to work so you can help enrich our diverse workforce.

We are proud to be an equal opportunities employer. This means that we do not discriminate on the basis of anyones race, religion, colour, national origin, gender, sexual orientation, gender identity, gender expression, age, marital status, veteran status, pregnancy or disability, or any other basis protected under applicable law. Conforming with applicable law, we can reasonably accommodate applicants' and employees' religious practices and beliefs, as well as mental health or physical disability needs.

You will be part of a collaborative and creative culture where we encourage new ideas. We are committed to sustainability across our global business and we are proud to partner with our customers to help them meet their sustainability objectives. Our charity, the LSEG Foundation provides charitable grants to community groups that help people access economic opportunities and build a secure future with financial independence. Colleagues can get involved through fundraising and volunteering.

LSEG offers a range of tailored benefits and support, including healthcare, retirement planning, paid volunteering days and wellbeing initiatives.

Please take a moment to read this privacy notice carefully, as it describes what personal information London Stock Exchange Group (LSEG) (we) may hold about you, what its used for, and how its obtained, your rights and how to contact us as a data subject.

If you are submitting as a Recruitment Agency Partner, it is essential and your responsibility to ensure that candidates applying to LSEG are aware of this privacy notice.

Location: IND-Hyderabad-CapitaLand

Time Type: Full time