Big Data with SRE

Technical/Functional:

BigData Support
5+ years of IT experience
Experience in big data technologies like Hadoop, Hive, Spark, Kafka
Understanding of data ingestion, CI/CD process
Ability to analyze logs for error and exceptions - Ability to drill down errors to cluster issues, code issues etc.
Proficiency in at least one scripting language (Shell/Python etc.)
Good knowledge of Linux and debugging skills
Solid troubleshooting abilities and able to work with a team to fix large production issues

Data Engineer
5+ years of professional experience with Big Data systems, pipelines, and data processing
Experience in Java/Scala programming and Python scripting
Hands on experience in SQL and should be able to write complex queries using joins
Hands on experience with Big Data, data ingestion, data processing using Spark, Spark Streaming, Flink, HIVE, Kafka, Hadoop, HDFS, S3
Confirmed understanding of design and development of large scale, high throughput and low latency applications is a plus
Excellent problem solving and programming skills
Experience with NoSQL databases like Cassandra and containerization technologies like Kubernetes, Docker, is desirable
Experience with CI/CD, debugging and monitoring applications and big data jobs is desirable

Platform Experience
Experience working on managing applications and/or Jobs in AWS
Experience managing datasets in S3
Experience managing and supporting applications and/or jobs in Kubernetes platforms.
Experience managing datasets in HDFS.