AWS data engineer
Apply NowCompany: SysMind Tech
Location: Austin, TX 78745
Description:
1 Data Engineer
Resource will be working across different Indeed tools, this role is a BE data engineer and should have experience with data pipeline
The role involves using various tools to provide updates on how Indeed is generating revenue by conducting A/B testing experiments. The primary responsibilities include working on the backend with Indeed tools, which involves managing data lakes, AWS Athena, Presto, Trino, iQL, S3, and orchestration frameworks like Airflow and Trestle.
The person in this role will prepare tables and ensure the scalability of the system. Additionally, they will work with other tools such as Heartbeat, alerting systems, and call metrics, which are updated on a weekly or daily basis. Maintaining data pipelines is key to this role
Trestle is an internal tool at Indeed designed to facilitate the movement of data. It is scoped for support on Python v3.11 and aims to standardize, streamline, and simplify data movement processes
Airflow is a scheduling framework that enables users to orchestrate Orc and other jobs
Revenue alerting is a system designed to quickly and accurately detect anomalies and deviations from company forecasts of revenue and other key health metrics at Indeed
Heartbeat includes monitoring connections and ensuring the proper functioning of different applications
Required Technical Skills:
Programming Languages: Proficiency in Python, SQL
Data Engineering Tools: Experience with AWS and data pipeline engineering.
A/B Testing: Experience in designing and conducting A/B tests to inform feature performance and product improvements.
Nice to have Technical Skills:
Data Visualization: Strong skills in data visualization tools like Tableau and Power BI. (nice to have but, mostly outside of scope)
Statistical Analysis: Ability to conduct statistical analysis, opportunity sizing, and causal analysis. (no need to be an expert in this area, mostly outside of scope)
Data Analysis: Proficiency in exploratory data analysis and hypothesis testing.
Some of the tools against revenue alerting have some Java code and Slack integration, experience connecting with these types of systems would be nice to have
Cross-functional Collaboration: Experience working with BI Team
Responsibilities/Deliverables:
Improve the scalability of the backend for the Proctor Impact Tool to ensure it runs without errors.
Enhance revenue alerting systems to ensure more connections with fewer errors.
Resource will be working across different Indeed tools, this role is a BE data engineer and should have experience with data pipeline
The role involves using various tools to provide updates on how Indeed is generating revenue by conducting A/B testing experiments. The primary responsibilities include working on the backend with Indeed tools, which involves managing data lakes, AWS Athena, Presto, Trino, iQL, S3, and orchestration frameworks like Airflow and Trestle.
The person in this role will prepare tables and ensure the scalability of the system. Additionally, they will work with other tools such as Heartbeat, alerting systems, and call metrics, which are updated on a weekly or daily basis. Maintaining data pipelines is key to this role
Trestle is an internal tool at Indeed designed to facilitate the movement of data. It is scoped for support on Python v3.11 and aims to standardize, streamline, and simplify data movement processes
Airflow is a scheduling framework that enables users to orchestrate Orc and other jobs
Revenue alerting is a system designed to quickly and accurately detect anomalies and deviations from company forecasts of revenue and other key health metrics at Indeed
Heartbeat includes monitoring connections and ensuring the proper functioning of different applications
Required Technical Skills:
Programming Languages: Proficiency in Python, SQL
Data Engineering Tools: Experience with AWS and data pipeline engineering.
A/B Testing: Experience in designing and conducting A/B tests to inform feature performance and product improvements.
Nice to have Technical Skills:
Data Visualization: Strong skills in data visualization tools like Tableau and Power BI. (nice to have but, mostly outside of scope)
Statistical Analysis: Ability to conduct statistical analysis, opportunity sizing, and causal analysis. (no need to be an expert in this area, mostly outside of scope)
Data Analysis: Proficiency in exploratory data analysis and hypothesis testing.
Some of the tools against revenue alerting have some Java code and Slack integration, experience connecting with these types of systems would be nice to have
Cross-functional Collaboration: Experience working with BI Team
Responsibilities/Deliverables:
Improve the scalability of the backend for the Proctor Impact Tool to ensure it runs without errors.
Enhance revenue alerting systems to ensure more connections with fewer errors.