Pyspark Data Engineer
Apply NowCompany: SysMind Tech
Location: Columbus, OH 43230
Description:
Role Description:
1 Data transformation using strem sets, Spark knowledge2 Designing and implementing data ingestion pipelines from multiple sources using Azure Data bricks3 Developing scalable and re-usable frameworks for ingestion datasets4 Linux commands and Shell/Pearl Scripting.5 Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.6 Working experience in ETL testing including Master Data Management, Data Completeness Data Cataloging, Data Leverage, and Data Quality for various data feeds coming from the source7 Should have experience working on complex data architecture8 On-call support9 Experience working in Agile/ Scrum methodology.10 IBM Sterling B2B Integrator and IBM Sterling File Gateway.11 Strong communication and documentation skills
Competencies:
Digital : Databricks, Digital : PySpark
Experience (Years):
6-8
Essential Skills:
Hands on development experience in Spark Scripting (R, Scala) and DHF tools Should have Good Communication skills and be a Good Team player
Desirable Skills:
Expertise in designing and deploying data applications on cloud solutions, such as Azure or AWS
Country:
United States
Branch | City | Location:
TCS - Cincinnati, OH
COLUMBUS
Columbus, OH
1 Data transformation using strem sets, Spark knowledge2 Designing and implementing data ingestion pipelines from multiple sources using Azure Data bricks3 Developing scalable and re-usable frameworks for ingestion datasets4 Linux commands and Shell/Pearl Scripting.5 Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.6 Working experience in ETL testing including Master Data Management, Data Completeness Data Cataloging, Data Leverage, and Data Quality for various data feeds coming from the source7 Should have experience working on complex data architecture8 On-call support9 Experience working in Agile/ Scrum methodology.10 IBM Sterling B2B Integrator and IBM Sterling File Gateway.11 Strong communication and documentation skills
Competencies:
Digital : Databricks, Digital : PySpark
Experience (Years):
6-8
Essential Skills:
Hands on development experience in Spark Scripting (R, Scala) and DHF tools Should have Good Communication skills and be a Good Team player
Desirable Skills:
Expertise in designing and deploying data applications on cloud solutions, such as Azure or AWS
Country:
United States
Branch | City | Location:
TCS - Cincinnati, OH
COLUMBUS
Columbus, OH