Data Engineer

As a Data Engineer, you will be responsible for designing and building a quality cloud-based data pipeline and its peripheral tooling. You must be self-driven and comfortable working in a highly collaborative environment while interfacing with both technical and non-technical persons. In this role, you will join forces with other teams of software engineers and data scientists to support our business initiatives.

Core Attributes:

The ideal candidate will possess...

• Design and analytical capabilities needed to build cloud-based data platforms
• Collaboration and communication skills are required to support our software engineering practices including agile development, CI/CD, code review, etc.
• Strong software engineering and computer science fundamentals along with software testing experience including unit tests, integration tests, etc.
• A well-developed sense of ownership in their work, driving decisions from inception through completion
• Passion for delivering value to the business with a relentless drive to improve process
• A keen desire to understand the domain knowledge around the products and how the products are developed along with knowledge of industry trends
• A strong sense of ethical responsibility as big data touches the privacy of our clients
• The ability to develop innovative solutions in a greenfield environment

Principal Responsibilities:

• Create and maintain optimal data pipeline architecture
• Identify, design, and implement internal process improvements focused on: automating manual processes, optimizing data delivery, re-designing infrastructure for better scalability, etc.
• Build the infrastructure required for extraction, transformation, and loading of data from various data sources
• Build analytic tools to provide actionable insights into key business performance metrics
• Work with stakeholders across the business and technology to solve their data needs
• Be responsible for key aspects of project delivery at a high level of quality

Required Technical Experience:

Experience in...

• Building and optimizing "Big Data" data pipelines, architectures, and data sets
• Object-oriented or functional languages (Java8+, Python, Scala, etc.)
• CI/CD Tools (Git, Docker)
• AWS cloud services (EC2, ECS, EMR, S3, RDS, etc.)
• ETL Orchestration Tools (Airflow, Glue, Luigi, NiFi, etc.)
• SQL and NoSQL data store (PostgreSQL, MySQL, DynamoDB, Elasticsearch, etc.)

Knowledge of...

• Big data tools and vendors (Hadoop, Spark, Kafka, Databricks, etc.)
• HashiCorp Tools (Terraform, Vault, Consul, etc.)
• Big data concepts (message queuing, streaming processing, high availability, scalability, etc.)
• Data security concepts (data governance, access control, etc.)
• The workflow of data wrangling, feature engineering, model development and model deployment along with necessary architectural requirements

Soft Skills:

• Strong interpersonal skills and ability to forge relationships resulting in successful partnerships
• Ability to convey ideas effectively to help drive solutions for the business and tech partners
• Enthusiasm for learning and is results-oriented

Required Education:

Strong academic record and a BA/BS degree with high mathematical and computing content (e.g., Computer Science, Statistics, Mathematics, Engineering, or Physics)