Senior Systems Administrator - Databricks and Azure

Job Summary:

We are seeking a talented and passionate Systems Engineer to join the Data Services (DS) Systems team. This role involves working in a multi-technology environment, where the Systems Engineer will be responsible for managing and maintaining our Enterprise Data Warehouse (EDW) and all associated components. This includes, but is not limited to, Azure Databricks (DBX), on-premise and cloud-based servers, third-party software installations, cloud-based services interfaces, and integrations, databases, and DevOps pipelines. The successful candidate will ensure well-managed and secure platforms for optimal performance and availability.

Key Responsibilities:

• Platform Management: Manage and maintain Data Services platforms, including on-premise and cloud-based Windows and Linux servers.
• Azure Components: Gain knowledge and manage Azure IaaS, PaaS, and SaaS components such as Function and Logic Apps, Key Vaults, ADLS, DBX, Elastic Search, and Azure Data Factory (ADF).
• DevOps Functions: Perform DevOps functions, deploy applications, and seek opportunities for automation and efficiency in the DevOps pipelines.
• Data Pipeline Development: Design, develop, and maintain scalable data pipelines (classic and YAML-based) and analytics solutions.
• On-Call Support: Participate in standard Run The Engine (RTE) and on-call support for EDW, Data Lake, and Azure Data Backbone platforms, ensuring platform availability.
• Month-End Processing: Support Client Month-End Financial Close (MEFC) processing to meet published SLAs.
• Incident Management: Manage incidents in a timely and accurate manner, ensuring actions, communications, and resolutions are documented in the Known Errors Database (KEDB).
• Software Upgrades & License Management: Perform necessary application software upgrades, license management, and Key/Certificate rotations for platforms.
• User Access Support: Work with end-users to address access requests and issues with the Data Services platforms.
• Documentation: Maintain accurate documentation in Confluence for new support processes, platforms, and software applications.
• Automation: Create automation processes to eliminate manual tasks and improve platform management.
• Platform Maintenance: Ensure security, performance, and maintenance of the EDW to ensure high availability and consistent end-user experience.
• Monitoring & Metrics: Implement monitoring, alerting, and escalation processes that conform to published NFRs and ensure platform availability and performance.
• Incident Response: Respond promptly to service outages, access issues, and performance degradation events to maintain EDW platform health in Azure.

Required Qualifications:

• Education: Bachelor's degree in Computer Science, Information Systems, or a related field, or equivalent work experience/certifications.
• Server & Database Administration: 3+ years of experience in Windows and Linux server administration, and SQL Server and Oracle database systems, including incident, software, security, and access control management.
• Azure Experience: Hands-on experience with Azure Cloud (IaaS, PaaS, and SaaS), Azure Databricks, Azure CLI, Azure Synapse, Azure Automation, and Azure PowerShell.
• Hybrid Environment: Experience working in a hybrid environment (on-premise and cloud) with server and software administration.
• ETL & BI Tools: Experience with ETL tools and Business Intelligence (BI) applications like QlikSense, Power BI, and Informatica.
• ITIL Framework: Experience operating within an ITIL Framework; ServiceNow experience is preferred.
• Security & Compliance: Strong understanding of platform and application security, compliance concepts (e.g., SOX), and cost-management considerations.
• Performance Monitoring & Automation: Experience with performance monitoring and developing new alerting and monitoring solutions.
• Scripting Skills: Proficiency in SQL, YAML, Terraform, and PowerShell scripting.
• SQL Development: Strong experience in SQL development and tuning.

Preferred Qualifications:

• Certifications: Azure-related certifications (e.g., Azure Solutions Architect, Azure DevOps Engineer).
• Cloud Infrastructure: Additional experience with Azure Key Vaults or Unity Catalogs.
• Business Intelligence Tools: Experience with advanced Business Intelligence tools such as Qlik Replicate.

Certifications:

Azure Certification (e.g., Azure Solutions Architect, Azure DevOps).

ITIL Certification (preferred).

ServiceNow (preferred).

Terraform Certification (preferred).

Primary Skills:

Azure Databricks: Proficiency in managing and optimizing Azure Databricks environments.

Automation & DevOps: Experience in automation and DevOps pipelines, including YAML and Terraform.

Platform Administration: Strong experience in managing Windows and SQL Server platforms, with expertise in Azure and hybrid environments.

Performance Monitoring: Ability to implement and manage monitoring and alerting systems to ensure platform availability and performance.

Education: Bachelors Degree