PySpark /SQL Engineer
Apply NowCompany: Tata Consultancy Services
Location: New York, NY 10025
Description:
Job Title - PySpark /SQL Engineer
Experience Range - 6+ years
Technical/Functional Skills
6+ years of software development experience in a professional environment and/or comparable experience
Strong proficiency and Hands-on coding experience in Python/PySpark, SQL
Experience with Big data, ETL pipelines
Exposure to Java and ability to understand Java code logic
Experience with Distributed Computing
Experience with Hadoop ecosystem
Experience in Apache Spark
Experience with Hive, PostgresSQL, Cloud experience
Extensive hands-on experience with designing, building, and optimizing data pipelines and applications
using PySpark and SQL for large-scale data processing and analysis, often in a big data environment.
Optimize data pipeline performance for speed and scalability.
Bachelors degree in computer science, computer science engineering, or related experience required
Roles & Responsibility :
Design, develop, and maintain data pipelines and applications for large-scale data
Collaborate with cross-functional teams to gather requirements, architect solutions, and implement
features that meet business needs.
Troubleshoot and debug production issues, conduct root cause analysis, and implement solutions to
prevent recurrence.
Stay updated on emerging technologies and industry trends, new tools and frameworks to improve efficiency
and productivity.
Communicate effectively with team members, stakeholders, and management to provide project updates,
status reports, and technical recommendations.
application.
Salary Range - $90000 to $110000 a year
Experience Range - 6+ years
Technical/Functional Skills
6+ years of software development experience in a professional environment and/or comparable experience
Strong proficiency and Hands-on coding experience in Python/PySpark, SQL
Experience with Big data, ETL pipelines
Exposure to Java and ability to understand Java code logic
Experience with Distributed Computing
Experience with Hadoop ecosystem
Experience in Apache Spark
Experience with Hive, PostgresSQL, Cloud experience
Extensive hands-on experience with designing, building, and optimizing data pipelines and applications
using PySpark and SQL for large-scale data processing and analysis, often in a big data environment.
Optimize data pipeline performance for speed and scalability.
Bachelors degree in computer science, computer science engineering, or related experience required
Roles & Responsibility :
Design, develop, and maintain data pipelines and applications for large-scale data
Collaborate with cross-functional teams to gather requirements, architect solutions, and implement
features that meet business needs.
Troubleshoot and debug production issues, conduct root cause analysis, and implement solutions to
prevent recurrence.
Stay updated on emerging technologies and industry trends, new tools and frameworks to improve efficiency
and productivity.
Communicate effectively with team members, stakeholders, and management to provide project updates,
status reports, and technical recommendations.
application.
Salary Range - $90000 to $110000 a year