AIA MLEU - GCP/ BIG Query/ Hadoop Data Engineer

Apply Now

Company: Avani Technology Solutions, Inc.

Location: Dearborn, MI 48126

Description:

Job Description: -
  • MUST HAVE: Hadoop, Spark, Java
  • Nice to Have : SpringBoot, PCF, GCP
  • Bachelor's degree in Software Engineering or Computer Science.
  • 5+ years designing, developing and implementing solutions in Hadoop environments
  • Previous experience as a Hadoop Developer /Big Data Engineer.
  • Advanced knowledge of the Hadoop ecosystem and its components.
  • In-depth knowledge of SPARK, Hive, H Base.
  • In depth knowledge of Map Reduce.
  • Minimum 5 years' experience with Java
  • Must have minimum 5 years' experience with software delivery implementations and working with customers.
  • 8+ Good Experience in any of the Hadoop distributed platform like HDP/Cloudera
  • Hands on experience on in Java or Scala and Python programming language.
  • Experience in Spark to build efficient solutions and ability to write, optimize and tune Hive queries
  • Good grasp on Hive data modeling/ ACID feature/ Partitions and Clustering
  • Exposure to debug Map Reduce/Spark jobs in Yarn
  • Processing and consumption and implement solutions to those into a central Big Data Platform System Design, Development and Implementation.
  • Defining job flows using Hadoop Scheduler, e.g. Oozie, Apache Falcon, Zookeeper etc.
  • Develop MapReduce programs or Hadoop streaming.
  • Working experience in Hive is mandatory
  • Participate in design reviews, code reviews, unit testing and integration testing.
  • Manage Hadoop jobs using scheduler
  • Reports critical issues to management effectively, timely and with clarity.
  • Good exposure to Data Ingestion tools like Attunity, Talend , Sqoop
  • Design and architect end-to-end systems and processes - use analytical skills to meet reliability, scalability, security requirements
  • Converse with product owners / product managers / business customers about the realm of the possible, help with scoping and defining Minimum Viable Products
  • Be forward thinking to identify and build the architecture runway needed to keep our capabilities and infrastructure modern and able to leverage new technologies as they emerge
  • Pair regularly and lead by example, staying current with the technology stack and being integrally involved with the code base
  • Apply a test-first mentality and impress upon the team the importance of a healthy suite of tests (unit, integration, smoke tests, etc.)

Similar Jobs