Pyspark Tech Lead

We are seeking a Technical Lead with 7 to 11 years of experience to join our team. The ideal candidate will have expertise in Cloudera HUE, Cloudera Data Platform, Apache lceberg, Spark, and PySpark. This role involves working in a hybrid model with day shifts. The candidate will play a crucial role in driving our research and development initiatives, leveraging their technical skills to deliver impactful solutions. Required Skills :Technical Skills : Cloudera HUE PySpark. Spark Pyspark.Cloudera Data Platform,Apache IcebergDomain Skills Responsibilities :- Lead the design and implementation of data solutions using Cloudera HUE and Cloudera Data Platform-Oversee the integration of Apache Iceberg and Spark into existing data workflows- Provide technical guidance and mentorship to junior team members on PySpark best practices-Collaborate with deliver scalable solutions- Ensure data cross-functional teams to understand data requirements and quality and integrity through rigorous testing and validation processes- Develop and maintain documentation for data architectures and workflows- Optirmize data processing pipelines for performance and efficiency- Conduct code reviews to ensure adherence to coding standards and best practices- Troubleshoot and resolve complex technical issues related to data processing-Stay updated with the latest advancements in data technologies and incorporate them into projects- Drive innovation by exploring new tools and techniques in the research and development domain- Communicate effectively with stakeholders to gather requirements and provide project updates Contribute to the overall success of the company by delivering high-quality data solutions that drive business value Qualifications-Possess strong expertise in Cloudera HUE and Cloudera Data Platform- Demonstrate proficiency in Apache Iceberg and Spark- Have extensive experience with PySpark for data processing- Show a solid understanding of data architecture and data engineering principles- Exhibit excellent problem-solving skills and attention to detail- Have a background in research and development is a plus- Display strong communication and collaboration skill - Be able to work effectively in a hybrid work model- Have a proactive approach to learning and staying current with industry trends- Be capable of mentoring and guiding junior team members-Demonstrate the ability to deliver high-quality solutions within deadlines- Show a commitment to continuous improvement and innovation- Possess a strong sense of ownership and accountability for project outcomes