Sr Databricks Developer

Experience Level Required: 7-10 years

Mandatory Required Skills:
Azure Databricks

Apache Spark

Data Modelling

Azure Data Lake Creation

Python Programming

Preferred /Desired Skills:

predictive analytics

Experience in Client Libraries

Responsibilities:

• Develop and maintain ETL (Extract, Transform, Load) pipelines using Databricks to process and transform large datasets.
• Collaborate with data engineers and data scientists to design and implement scalable and efficient data processing workflows.
• Build and optimize Apache Spark jobs and clusters on the Databricks platform.
• Develop and maintain data ingestion processes to acquire data from various sources and systems.
• Implement data quality checks and validation procedures to ensure accuracy and integrity of data.
• Perform data analysis and exploratory data mining to derive insights from complex datasets.
• Design and implement machine learning workflows using Databricks for predictive analytics and model training.
• Troubleshoot and debug issues related to data processing, performance, and job failures.
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
• Stay updated with the latest advancements in big data technologies and contribute to the improvement of existing systems and processes.

Requirements:

• Solid experience in developing data processing workflows using Apache Spark and Databricks.
• Proficiency in programming languages like Python, Scala, or SQL for data manipulation and analytics.
• Strong understanding of distributed computing principles and experience with large-scale data processing frameworks.
• Familiarity with cloud platforms such as Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP).
• Experience with data modeling, database systems, and SQL.
• Knowledge of machine learning concepts and experience with Client libraries and frameworks.
• Excellent problem-solving skills and ability to work independently and in a team.
• Strong communication skills to collaborate with stakeholders from different technical backgrounds.

Pls share profiles for the mentioned onsite demands: Please fill the Skill matrix Skills Rating(out of 5) Number of years used Candidate write up zure Databricks pache Spark Data Modelling zure Data Lake Creation Python Programming predictive analytics Experience in Client Libraries