Lead Data Engineer

Lead Data Engineer

Location: Redwood City, CA

Fulltime

Candidate must be open to relocate post COVID/Client opens facilities

Keywords

Amazon Redshift, ETL, Ab Initio, Informatica, Hive, Python, Airflow, Automic, Cloud Data Warehouse, SQL, Databricks, Data lake

Lead Data Engineer - Job Description

Lead - Data Engineer with Software Engineering skills for Data Warehouse Development team.

The lead will, manage and drive end-to-end solutions and data infrastructure, will work with analytics and business partners to deliver data solutions in support of insights and analysis of a multi-million customer ecommerce business with both internal and external data by applying expertise knowledge with data-warehousing technical concepts, CS fundamentals and data and system architecture to multi-terabyte, multi-source data.

Responsibilities

• Build data expertise and own data quality for the pipelines you build
• Architect, build and launch new data models and data marts that provide intuitive analytics to your customers
• Design, build and launch extremely efficient & reliable data pipelines to move data (both large and small amounts) into and out of the Data Warehouse
• Design and develop new systems and tools to enable folks to consume and understand data faster
• Use your coding skills across a number of languages including Python, Shell Scripting, PL/SQL, AI PDL.
• Have a clear understanding of the reports/analyses/insights to be driven by data and build data solutions to optimally support the analytics needs
• Integrate third party data to enrich our data environment and enable new analytic perspectives
• Work across multiple teams in high visibility roles and own solutions end-to-end
• Work with program managers, business partners and other engineers to develop and prioritize
• Lead and coordinate with offshore team

Must Have Skills -

• Experience in building, maintaining and automating reliable and efficient ETL, ELT jobs
• 2-3 years hands on experience with AWS Cloud Data Warehouses, AWS IAM Roles, Glue, EC2, S3 and Redshift
• Hands-On experience with ETL development Informatica 9/10.x Power Center, Ab Initio 3.x GDE, AI Control Center and Shell Scripting.
• Experience in automation and orchestration platforms such as Airflow, Control M & Automic
• Strong CS fundamentals and experience developing with object-oriented programming (Python, Java)
• Expertise with data warehouse data models (dynamo, star, snowflake schemas)
• Experience with multi-Terabyte MPP relational databases, such as My Sql, Oracle, Teradata
• Understanding of streaming technologies and concepts used with data warehouses, is preferred

Good to have skills- Other Cloud Datawarehouse, Hive, Hadoop, Datalake, Databricks, JIRA experience