at Capgemini
ConsultanciesPosted 15 days ago
No clicks
**Senior Data Engineer (Microsoft Fabric/Databricks)** Leverage 5+ years of data engineering experience to build, optimize, and manage Apache Spark pipelines on Microsoft Fabric or Databricks. Tune performance using Spark UI and Delta statistics, optimize Delta Lake strategies, and create reusable Python libraries. Proficient in SQL and Apache Spark, with strong experience in data pipeline optimization and hands-on knowledge of tools like Python, Delta Lake, and delta-optimize. Collaborate on meaningful projects, fostering sustainability and inclusion as part of Capgemini's global team. Mexican state-based work-from-home opportunities.
- Compensation
- Not specified
- City
- Not specified
- Country
- Mexico
Currency: Not specified
Full Job Description
Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way youd like, where youll be supported and inspired by a collaborative community of colleagues around the world, and where youll be able to reimagine whats possible. Join us and help the worlds leading organizations unlock the value of technology and build a more sustainable, more inclusive world.
Your Role
- Build and optimize Spark pipelines on Microsoft Fabric or Databricks.
- Diagnose performance using Spark UI and Delta statistics.
- Manage Delta Lake design, Fabric capacity consumption, compute sizing, throttling exposure, and operational reliability.
- Create reusable Python libraries and Spark modules for ingestion, governance, security, monitoring, and optimization.
Your Profile
- 5+YOE working as a data engineer, ETL engineer or similar roles
- Strong experience with Microsoft Fabric or Databricks in production environments, including performance tuning and operational support.
- Apache Spark troubleshooting, including skew, spills, OOM errors, task execution analysis, and query optimization.
- Hands-on experience optimizing Delta Lake using partitioning, liquid clustering, Z-Order, OPTIMIZE, VACUUM, and MERGE strategies.
- SQL and Data Engineering skills with the ability to design, optimize, and troubleshoot large-scale data pipelines.
- Python development and building reliable, production-grade data solutions with strong testing, logging, monitoring, and documentation practices.
- Fluent English level
What you'll love
- Work on meaningful projects that use technology to solve real-world challenges.
- Be part of a company that values sustainability, inclusion, and digital equity.
- Contribute to building a better future for people, planet, and society.
Need to know
This role can be working from any state of Mexico
About Capgemini
Capgemini is an AI-powered global business and technology transformation partner, delivering tangible business value. We imagine the future of organizations and make it real with AI, technology and people. With our strong heritage of nearly 60 years, we are a responsible and diverse group of over 420,000 team members in more than 50 countries. We deliver end-to end services and solutions with our deep industry expertise and strong partner ecosystem, leveraging our capabilities across strategy, technology, design, engineering and business operations.
At Capgemini Mexico, we aim to attract the best talent and are committed to creating a diverse and inclusive work environment, so there is no discrimination based on race, sex, sexual orientation, gender identity or expression, or any other characteristic of a person. All applications welcome and will be considered based on merit against the job and/or experience for the position.
