Job Opportunity: Data Engineer (Databricks Specialist)
Manchester | Employment Type : Full-time.
Are you a passionate Data & AI expert with a strong Databricks background?
A leading consulting firm is seeking a person skilled in Data Engineering to join their growing team.
In this senior role, you'll work with clients to design and implement scalable, high-performance Databricks-based platforms that drive impactful business outcomes
As a Data Engineer, you will be responsible for designing, implementing, and optimizing data workflows that drive critical insights across the business.
You'll work with cutting-edge technologies to build scalable, high-performance data solutions that meet the evolving needs of our clients.
Data Pipeline Development: Data Integration: Develop scalable data ingestion processes from multiple sources and integrate them into Delta Lake or data lakes for unified storage and analysis.
Partner closely with data architects, data scientists, and business analysts to understand requirements and deliver data solutions that align with business objectives and enhance analytics capabilities.
Performance Optimization: Optimize Databricks workflows for performance, scalability, and cost efficiency, including Spark tuning and cluster management .
Build cloud-native data engineering solutions on leading cloud platforms like Azure, AWS, or GCP to ensure seamless, secure, and efficient data management.
Governance & Quality: Implement and maintain data governance best practices, ensuring data consistency, quality, and security across the organization.
Automation: Automate repetitive tasks, develop robust CI/CD pipelines, and streamline data workflows to improve operational efficiency and reduce manual effort.
3+ years of experience in data engineering , with hands-on expertise in Databricks .
~ Proven ability to deliver large-scale data pipelines in consulting or enterprise environments.
~ Strong programming skills in Python and SQL , with optional experience in Scala .
~ Experience with cloud platforms such as Azure (preferred), AWS , or GCP .
~ Strong understanding of relational and non-relational databases , data lakes , and real-time data processing .
~ Experience with tools like MLflow or other AI/ML frameworks .
Familiarity with data visualization tools such as Power BI or Tableau .
Azure Data Engineer Associate ).
Experience in automation and CI/CD practices for data engineering pipelines.
Challenging Projects: Work on diverse and impactful projects for clients across various industries, contributing to real-world data solutions.
Growth Opportunities: Gain exposure to cutting-edge technologies and continue developing your technical expertise within a fast-paced, innovative environment.
We offer an attractive salary, comprehensive benefits, and flexible working options, including remote work opportunities.
If you're passionate about data engineering , enjoy tackling complex challenges, and want to be part of a forward-thinking team, we want to hear from you!