GCP Data Engineer - DBA

GCP Data Engineer with DBA

Roles and Responsibilities:
Data Engineering:

• Design, develop, and maintain a Data Lake architecture using GCP services, particularly BigQuery and AlloyDB.
• Build robust data pipelines for ETL/ELT processes using tools like Dataflow, Cloud Composer, or other GCP services.
• Integrate data from various sources (structured and unstructured) into the Data Lake, ensuring consistency and reliability.
• Optimize BigQuery data models and queries for high performance and scalability.

Database Administration:

• Administer and maintain AlloyDB instances, ensuring high availability, security, and performance.
• Perform backup, recovery, and disaster recovery planning for AlloyDB and BigQuery.
• Monitor and optimize database performance, query execution, and resource utilization.
• Manage schema design, indexing, partitioning, and clustering to enhance database efficiency.
• Apply database governance and compliance best practices to ensure data security and regulatory adherence.

Data Governance and Management:

• Implement data governance policies, including role-based access controls, data masking, and encryption.
• Manage metadata, data cataloging, and data lineage tracking to support audit and compliance requirements.
• Conduct regular health checks on the Data Lake to ensure data quality and integrity.

Collaboration and Reporting:

• Collaborate with data scientists, analysts, and application developers to define data requirements and deliver solutions.
• Design and deploy real-time and batch data analytics solutions using GCP's BigQuery and Looker.
• Provide documentation and training to teams on using Data Lake features effectively.

Automation and Monitoring:

• Automate database and data pipeline tasks.
• Set up monitoring and alerting for BigQuery and AlloyDB using Cloud Monitoring and Cloud Logging.
• Resolve incidents and troubleshoot performance issues in real time.