Senior Data Engineer
Apply NowCompany: EarthLink
Location: Atlanta, GA 30349
Description:
The Senior Data Engineer is responsible for designing, building, and maintaining the infrastructure and tools needed to handle large datasets.
ESSENTIAL DUTIES
Data Ingestion and Collection:
-Develop and maintain data pipelines to collect data from various sources, including databases, APIs, and streaming platforms.
-Implement data ingestion processes to ensure the availability, reliability, and scalability of data.
Data Transformation and ETL:
-Design and build ETL (Extract, Transform, Load) processes to transform raw data into structured and usable formats.
-Perform data cleansing, enrichment, and normalization to ensure data quality and consistency.
Data Warehousing:
-Create and manage data warehouses, data lakes, or storage solutions to store and organize data for analysis.
-Optimize data storage and retrieval mechanisms for performance and cost-efficiency.
Data Modelling and Schema Design:
-Collaborate with data analysts and data scientists to design data models and schema for analytical purposes.
-Develop and maintain data dictionaries and documentation for data assets.
Data Quality and Governance:
-Implement data quality controls and ensure data integrity throughout the data lifecycle.
-Define and enforce data governance policies, including access controls and data retention policies.
Performance Optimization:
-Monitor and fine-tune data pipelines and processes for optimal performance.
-Identify and resolve data-related performance bottlenecks and issues.
Collaboration and Communication:
-Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand -data requirements and deliver data solutions.
-Communicate technical concepts and findings to non-technical stakeholders in a clear and understandable manner.
MINIMUM REQUIREMENTS
These specifications are general guidelines based on the minimum experience normally considered essential to the satisfactory performance of this position. The requirements listed below are representative of the knowledge, skill and/or ability required to perform the position in a satisfactory manner. Individual abilities may result in some deviation from these guidelines.
REMUNERATION
We offer an attractive remuneration and compensation package, including competitive salary, benefits, and opportunities for professional growth.
At One Communications, we are committed to creating an inclusive and diverse workplace. We are an equal opportunity employer and welcome applicants from all backgrounds, regardless of race, colour, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status.
ESSENTIAL DUTIES
Data Ingestion and Collection:
-Develop and maintain data pipelines to collect data from various sources, including databases, APIs, and streaming platforms.
-Implement data ingestion processes to ensure the availability, reliability, and scalability of data.
Data Transformation and ETL:
-Design and build ETL (Extract, Transform, Load) processes to transform raw data into structured and usable formats.
-Perform data cleansing, enrichment, and normalization to ensure data quality and consistency.
Data Warehousing:
-Create and manage data warehouses, data lakes, or storage solutions to store and organize data for analysis.
-Optimize data storage and retrieval mechanisms for performance and cost-efficiency.
Data Modelling and Schema Design:
-Collaborate with data analysts and data scientists to design data models and schema for analytical purposes.
-Develop and maintain data dictionaries and documentation for data assets.
Data Quality and Governance:
-Implement data quality controls and ensure data integrity throughout the data lifecycle.
-Define and enforce data governance policies, including access controls and data retention policies.
Performance Optimization:
-Monitor and fine-tune data pipelines and processes for optimal performance.
-Identify and resolve data-related performance bottlenecks and issues.
Collaboration and Communication:
-Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand -data requirements and deliver data solutions.
-Communicate technical concepts and findings to non-technical stakeholders in a clear and understandable manner.
MINIMUM REQUIREMENTS
These specifications are general guidelines based on the minimum experience normally considered essential to the satisfactory performance of this position. The requirements listed below are representative of the knowledge, skill and/or ability required to perform the position in a satisfactory manner. Individual abilities may result in some deviation from these guidelines.
- Bachelor's or master's degree in computer science, Data Science, Information Technology, or a related field.
- Proven experience as a Data Engineer, with a strong background in data warehousing, ETL, and data integration.
- Proficiency in programming languages like Python, Java, or Scala.
- Strong SQL skills for data manipulation and querying.
- Experience with big data technologies such as Hadoop, Spark, or related frameworks.
- Knowledge of database management systems (e.g., ETL, SQL, NoSQL) and data modelling.
- Good knowledge of Cloud DWH like Snowflake or redshift.
- Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their data services.
- Excellent problem-solving skills and attention to detail.
- Good communication and teamwork skills.
- Ability to work in a fast-paced, dynamic environment and adapt to evolving technologies.
- Preferred Skills (not required but beneficial):
- Understanding of machine learning and data science concepts.
REMUNERATION
We offer an attractive remuneration and compensation package, including competitive salary, benefits, and opportunities for professional growth.
At One Communications, we are committed to creating an inclusive and diverse workplace. We are an equal opportunity employer and welcome applicants from all backgrounds, regardless of race, colour, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status.