10817 - Advanced Data Engineer Level 2

24th July, 2024

Data Engineer Level 2

For more than three decades, Strategic Data Systems (SDS) has been a software consultancy firm specializing in strategy, technology, and business transformation for Fortune 100 companies, mid-sized firms, and startups. At SDS, we empower our development teams to address our clients' critical business challenges by leveraging cutting edge technologies. If you seek a workplace where your contributions are truly appreciated, then SDS is the company for you. Join us today to work alongside fellow development specialists and become a crucial part of our dynamic and cohesive community.

What You'll Do

• Working with varied data file formats (Avro, json, csv) using PySpark for ingesting and transformation
• DevOps process and Terraform scripting
• Leadership skills: must be able to put together and deliver presentation.
• Solid communication while working with PMO. Must be able to translate what product is asking for and regurgitate to team.

What You'll Need

• 6 month C2H, must be willing to convert to full-time
• minimum of 7 years of data related experience
• Top 3 Skills: Someone who is experienced in setting up databases, creating dimensional model synapse, and tuning - this is more important than Databricks experience
• Experience in Data modeling and advanced SQL techniques
• Experience working on cloud migration methodologies and processes including tools like Databricks, Azure Data Factory, Azure Functions, and other Azure data services
• Expert in SQL, Python, Spark, Databricks
• Experience working with varied data file formats (Avro, json, csv) using PySpark for ingesting and transformation
• Experience with DevOps process and understanding of Terraform scripting
• Understanding the benefits of data warehousing, data architecture, data quality processes, data warehousing design and implementation, table structure, fact and dimension tables, logical and physical database design
• Experience designing and implementing ingestion processes for unstructured and structured data sets
• Experience designing and developing data cleansing routines utilizing standard data operations
• Knowledge of data, master data, metadata related standards, and processes
• Experience working with multi-Terabyte data sets, troubleshooting issues, performance tuning of Spark and SQL queries
• Experience using Azure DevOps/Github actions CI/CD pipelines to deploy code
• Microsoft Azure certifications are a plus
• Minimum of 7 years of hands-on experience working on design, configuration, implementation, and data migration for medium to large sized enterprise data platforms

Soft Skills Needed

• Leadership Experience -must have experience managing a team
• Collaborative
• Relationship Building

Core Technologies
Azure Functions
Python
SQL Server
Azure Data Factory
Azure Databricks
Terraform
Azure DevOps
GitHub / GitHub Actions
T-SQL and SQL stored procedures
Azure Log Analytics
Azure Data Lake Storage
Azure Synapse

Nice to Have's

• Click Stream experience would be a HUGE plus**

What You'll Get

SDS, Inc. provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, gender, sexual orientation, national origin, age, disability, genetic information, marital status, amnesty, or status as a covered veteran in accordance with applicable federal, state, and local laws.

• Competitive base salary
• Medical, dental, and vision insurance coverage
• Optional life and disability insurance provided
• 401(k) with a company match and optional profit sharing
• Paid vacation time
• Paid Bench time
• Training allowance offering
• You'll be eligible to earn referral bonuses!