Lead Data Engineer
Apply NowCompany: Technogen International Company
Location: Philadelphia, PA 19120
Description:
Lead Data Engineer(Azure,Databricks)
Location: Philadelphia, Pennsylvania(Initially Remote)
EST Candidates only
Job Description
As a Azure Principal Data Lead, you will be responsible for:
Strategic planning and hands-on engineering of Big Data and cloud environments that support our clients' advanced analytics and data science initiatives.
Provide support in defining the scope and sizing of work
Working closely with various enterprise architects Information security teams, Data management team, to ensure the architected solution meets all the needs of a customer, from a functionality perspective and IT solution engineering perspective.
Translate business requirements into technology solutions
Lead designing of all aspects of our data solution including artifact creation such as diagrams, playbooks and other technical documents
Mentor and guide Jr. team members to deliver the solutions on-time
Create various architecture blueprints and work with the development team to deliver the vision.
Skills:
Overall 10+ years of experience of experience with Data Management, Big Data, Data Warehousing and Analytics.
At least 3 to 4 years of experience in Architecting and Implementing Big Data Solutions.
3+ years of experience in building data lakes, Delta Lakes on Azure by using Databricks.
More than one year of multi-cloud experience in at least two of the three public Cloud Platforms (Azure, AWS and GCP)
Two years of experience in data migrations to Azure by using data box or Data migration Services.
Hands on working experience with scripting languages such as Java, Scala, Python, or Shell Scripting, etc.
Extensive hands-on experience in data warehousing design, tuning and building ETL/ELT processes by using cloud native technologies like Azure ADF and Databricks Notebooks.
In-depth understanding of various storage services offered by Azure.
Experience with implementation of data security, encryption, PII/PSI legislation, identity and access management across sources and environments.
Experience with data process Orchestration, end-to-end design and build process of Near-Real Time and Batch Data Pipelines.
Certification in solution architecture, data engineering or Databricks in Azure is preferred
Strong client-facing communication and facilitation skills.
Location: Philadelphia, Pennsylvania(Initially Remote)
EST Candidates only
Job Description
As a Azure Principal Data Lead, you will be responsible for:
Strategic planning and hands-on engineering of Big Data and cloud environments that support our clients' advanced analytics and data science initiatives.
Provide support in defining the scope and sizing of work
Working closely with various enterprise architects Information security teams, Data management team, to ensure the architected solution meets all the needs of a customer, from a functionality perspective and IT solution engineering perspective.
Translate business requirements into technology solutions
Lead designing of all aspects of our data solution including artifact creation such as diagrams, playbooks and other technical documents
Mentor and guide Jr. team members to deliver the solutions on-time
Create various architecture blueprints and work with the development team to deliver the vision.
Skills:
Overall 10+ years of experience of experience with Data Management, Big Data, Data Warehousing and Analytics.
At least 3 to 4 years of experience in Architecting and Implementing Big Data Solutions.
3+ years of experience in building data lakes, Delta Lakes on Azure by using Databricks.
More than one year of multi-cloud experience in at least two of the three public Cloud Platforms (Azure, AWS and GCP)
Two years of experience in data migrations to Azure by using data box or Data migration Services.
Hands on working experience with scripting languages such as Java, Scala, Python, or Shell Scripting, etc.
Extensive hands-on experience in data warehousing design, tuning and building ETL/ELT processes by using cloud native technologies like Azure ADF and Databricks Notebooks.
In-depth understanding of various storage services offered by Azure.
Experience with implementation of data security, encryption, PII/PSI legislation, identity and access management across sources and environments.
Experience with data process Orchestration, end-to-end design and build process of Near-Real Time and Batch Data Pipelines.
Certification in solution architecture, data engineering or Databricks in Azure is preferred
Strong client-facing communication and facilitation skills.