Databricks Architect

Role : Databricks Architect

Location : O'fallon ,MO (Day 1 onsite )

Num of Positions : 1

Mandatory Skills : DataBrick Arc , Lakeouse , ACID , Schema , ETL/Orchestration , API , RDD API's , Spark , SQL , AI/ML .

Job Descriprion :

Minimum 10+ years of IT experience and minimum 2+ years of hands-on experience in Databricks with validated knowledge in designing database models to store structured & unstructured data efficiently and in creating effective data tools for analytics experts along with following:

• Familiarity with Lakehouse Architecture with Delta Tables - Delta tables, Schema Evolution, ACID compatibility, History capture etc.
• Integrating Databricks with various ETL/Orchestration tools and perform required operations.
• Expertise in Spark ecosystems, Data frame API, Data set API, RDD APIs & Spark SQL.
• Expertise in workspace and cluster configuration, performance tuning and log monitoring.
• Experience with structured streaming along with streaming frameworks.
• Hands on experience with Python, Scala and SQL.
• Create and maintain optimal data pipeline architecture with knowledge in best practices from Databricks around Cluster setup/Development/Deployment
• Strong understanding of Data Modelling and defining conceptual logical and physical data models.
• Experience in building Data Engineering solutions using Azure native stack
• Experience in managing various client relationships to get consensus on solutions.
• Nice to have AI/ML skill and any architect level certification from Databricks.

Mandatory Skills

DataBrick Arc , Lakeouse , ACID , Schema , ETL/Orchestration , API , RDD API's , Spark , SQL , AI/ML . Day 1 onsite