Databrick Architect
Apply NowCompany: Cloud BC Labs
Location: Houston, TX 77084
Description:
Position : Databrick Architect
Location : Houston, TX (Onsite)(Local only)
Term : C2C role
Job Description :
Responsibilities:
Requirements:
Location : Houston, TX (Onsite)(Local only)
Term : C2C role
Job Description :
Responsibilities:
- Design and implement large-scale data processing systems using the Databricks platform.
- Work closely with clients to understand their requirements and design solutions that meet their needs.
- Collaborate with data engineers and data scientists to develop and implement data pipelines.
- Ensure the security and reliability of data processing systems.
- Optimize data processing systems for performance and scalability.
- Keep up-to-date with emerging technologies and industry trends related to big data.
Requirements:
- Bachelor's degree in Computer Science, Engineering, or related field.
- At least 5 years of experience in designing and implementing big data processing systems.
- Strong knowledge of Apache Spark and the Databricks platform.
- Experience with data modeling, data warehousing, and data architecture.
- Experience with cloud-based data processing systems (AWS, Azure, GCP).
- Excellent communication and collaboration skills.
- Ability to work independently and as part of a team.
- Experience with Python, Java, or Scala programming languages is a plus.
- Programming in Python, and SQL is a must with experience in Scala/Spark or pyspark.
- Design on following Databricks constructs - Workflows, Delta Lake, Delta Live Tables.
- High expertise in Optimize, Vacuum, TimeTravel, etc., to tune the Deltalake.
- Design on Datamarts in the warehouse.
- High expertise in Structured Streaming and Watermarking.