Senior Data Engineer
Apply NowCompany: Nexus Health Systems
Location: Houston, TX 77084
Description:
POSITION SUMMARY:
The Senior Data Engineer will be responsible for designing, implementing, and managing large-scale data pipelines and cloud-based data solutions on Microsoft Azure, with a strong emphasis on SQL Server technologies. The role involves collaboration with cross-functional teams and business stakeholders, to develop efficient, scalable, and secure data platforms. The ideal candidate will have extensive experience with Azure services, deep expertise in SQL Server development, optimization, and management, and a passion for delivering data solutions that drive business insights and decision-making. Experience with Microsoft Fabric services for integrated analytics and data experiences will be a key advantage. The role will also involve integrating third-party data sources, developing robust data warehousing solutions, and leveraging Power BI for reporting and visualization.
JOB SPECIFIC RESPONSIBILITIES:
Design and implement scalable and robust data architectures on Azure, including data ingestion, storage, transformation, and analytics.
Lead the development of Azure data solutions using services like Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, and Azure SQL Database.
Architect ETL/ELT pipelines and workflows to efficiently move and transform large volumes of structured and unstructured data.
Design and implement enterprise-level data warehousing solutions leveraging Azure Synapse Analytics, Azure SQL Database, and Microsoft Fabric for analytics.
Design and optimize SQL Server data models, stored procedures, indexes, and partitioning strategies for performance, scalability, and reliability.
Develop, deploy, and monitor data pipelines using Azure Data Factory (ADF) and other Azure tools.
Integrate third-party data sources such as APIs, flat files, and external databases into Azure SQL, Microsoft Fabric, and reporting platforms.
Create and optimize complex SQL queries, stored procedures, and views to support data transformation and reporting requirements.
Automate data ingestion, processing, and monitoring workflows to minimize manual intervention.
Implement data governance best practices, including data security, quality, and compliance with organizational and regulatory standards.
Develop dimensional models (star and snowflake schemas) and optimize data structures for reporting and analytics.
Collaborate with BI analysts and business stakeholders to design and build Power BI reports and dashboards that deliver actionable insights.
Develop Power BI datasets, measures (DAX), and paginated reports for various business scenarios.
Work closely with business analysts, engineers, and key stakeholders to gather requirements and translate them into technical solutions.
Collaborate with DevOps teams to streamline CI/CD processes for data pipelines.
Troubleshoot and resolve data pipeline failures, bottlenecks, and performance issues.
Provide detailed documentation of data pipelines, solutions, and processes.
POSITION QUALIFICATIONS:
EDUCATION:
Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field.
EXPERIENCE:
Minimum of 7+ years of experience in data engineering with a strong focus on SQL Server technologies.
Proven track record of delivering enterprise-scale data solutions using SQL Server, Azure, Microsoft Fabric, and data warehousing platforms.
Extensive experience in SQL Server performance tuning, optimization, and high availability configurations.
Experience integrating third-party data sources and creating Power BI reports for analytics and visualization.
Background in Agile development methodologies.
Experience working with real-time and batch data solutions is a plus.
SQL Server Expertise: Extensive experience with Microsoft SQL Server, including:
o T-SQL programming for stored procedures, functions, and triggers
o Performance tuning, indexing strategies, and query optimization
o SQL Server Integration Services (SSIS) for ETL workflows
o SQL Server Reporting Services (SSRS) and SQL Server Analysis Services (SSAS)
o High Availability and Disaster Recovery (HA/DR) implementation
Azure Expertise: Strong experience with Azure data services, including:
o Azure SQL Database and Managed Instances
o Azure Data Factory (ADF)
o Azure Synapse Analytics (SQL Data Warehouse)
o Microsoft Fabric for unified data integration and analytics
o Azure Data Lake Storage
o Azure Functions
Power BI: Proven experience developing interactive reports, dashboards, and datasets using Power BI.
o Advanced knowledge of DAX, Power Query, and performance optimization.
o Experience integrating third-party data sources with Power BI reports.
Data Warehousing: Strong experience in designing, implementing, and optimizing data warehousing solutions using Azure Synapse Analytics and SQL-based architectures.
Programming Languages: Strong proficiency in T-SQL, Python, DAX or Scala for data processing and scripting.
ETL/ELT Pipelines: Proven expertise in building SQL Server-based ETL/ELT pipelines and integrating with Microsoft Fabric and Azure tools.
Data Governance & Quality: Hands-on experience with SQL Server data quality frameworks, Microsoft Fabric's data governance capabilities, and security compliance.
DevOps & CI/CD: Familiarity with DevOps tools such as Azure DevOps, Git, and CI/CD pipelines for SQL Server deployment automation.
Infrastructure as Code (IaC): Experience using tools like Terraform, ARM templates, or Bicep for SQL Server resource and Microsoft Fabric provisioning.
Soft Skills:
Strong problem-solving and critical-thinking abilities.
Excellent communication skills with technical and non-technical stakeholders.
Proven leadership and mentoring experience.
Ability to work independently and in team settings in a fast-paced environment.
Highly organized with attention to detail.
Preferred Qualifications
Knowledge of machine learning pipelines and data science workflows.
Familiarity with containerization technologies (Kubernetes, Docker).
Exposure to data security frameworks. (GHIPAA, SOC 2, etc.)
Experience with C#, JavaScript, HTML/CSS languages a plus.
LICENSURE/CERTIFICATION:
Microsoft SQL Server and Azure certifications (e.g., MCSA: SQL Server, Azure Data Engineer Associate, Fabric Analytics Engineer) are highly desirable.
The Senior Data Engineer will be responsible for designing, implementing, and managing large-scale data pipelines and cloud-based data solutions on Microsoft Azure, with a strong emphasis on SQL Server technologies. The role involves collaboration with cross-functional teams and business stakeholders, to develop efficient, scalable, and secure data platforms. The ideal candidate will have extensive experience with Azure services, deep expertise in SQL Server development, optimization, and management, and a passion for delivering data solutions that drive business insights and decision-making. Experience with Microsoft Fabric services for integrated analytics and data experiences will be a key advantage. The role will also involve integrating third-party data sources, developing robust data warehousing solutions, and leveraging Power BI for reporting and visualization.
JOB SPECIFIC RESPONSIBILITIES:
Design and implement scalable and robust data architectures on Azure, including data ingestion, storage, transformation, and analytics.
Lead the development of Azure data solutions using services like Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, and Azure SQL Database.
Architect ETL/ELT pipelines and workflows to efficiently move and transform large volumes of structured and unstructured data.
Design and implement enterprise-level data warehousing solutions leveraging Azure Synapse Analytics, Azure SQL Database, and Microsoft Fabric for analytics.
Design and optimize SQL Server data models, stored procedures, indexes, and partitioning strategies for performance, scalability, and reliability.
Develop, deploy, and monitor data pipelines using Azure Data Factory (ADF) and other Azure tools.
Integrate third-party data sources such as APIs, flat files, and external databases into Azure SQL, Microsoft Fabric, and reporting platforms.
Create and optimize complex SQL queries, stored procedures, and views to support data transformation and reporting requirements.
Automate data ingestion, processing, and monitoring workflows to minimize manual intervention.
Implement data governance best practices, including data security, quality, and compliance with organizational and regulatory standards.
Develop dimensional models (star and snowflake schemas) and optimize data structures for reporting and analytics.
Collaborate with BI analysts and business stakeholders to design and build Power BI reports and dashboards that deliver actionable insights.
Develop Power BI datasets, measures (DAX), and paginated reports for various business scenarios.
Work closely with business analysts, engineers, and key stakeholders to gather requirements and translate them into technical solutions.
Collaborate with DevOps teams to streamline CI/CD processes for data pipelines.
Troubleshoot and resolve data pipeline failures, bottlenecks, and performance issues.
Provide detailed documentation of data pipelines, solutions, and processes.
POSITION QUALIFICATIONS:
EDUCATION:
Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field.
EXPERIENCE:
Minimum of 7+ years of experience in data engineering with a strong focus on SQL Server technologies.
Proven track record of delivering enterprise-scale data solutions using SQL Server, Azure, Microsoft Fabric, and data warehousing platforms.
Extensive experience in SQL Server performance tuning, optimization, and high availability configurations.
Experience integrating third-party data sources and creating Power BI reports for analytics and visualization.
Background in Agile development methodologies.
Experience working with real-time and batch data solutions is a plus.
SQL Server Expertise: Extensive experience with Microsoft SQL Server, including:
o T-SQL programming for stored procedures, functions, and triggers
o Performance tuning, indexing strategies, and query optimization
o SQL Server Integration Services (SSIS) for ETL workflows
o SQL Server Reporting Services (SSRS) and SQL Server Analysis Services (SSAS)
o High Availability and Disaster Recovery (HA/DR) implementation
Azure Expertise: Strong experience with Azure data services, including:
o Azure SQL Database and Managed Instances
o Azure Data Factory (ADF)
o Azure Synapse Analytics (SQL Data Warehouse)
o Microsoft Fabric for unified data integration and analytics
o Azure Data Lake Storage
o Azure Functions
Power BI: Proven experience developing interactive reports, dashboards, and datasets using Power BI.
o Advanced knowledge of DAX, Power Query, and performance optimization.
o Experience integrating third-party data sources with Power BI reports.
Data Warehousing: Strong experience in designing, implementing, and optimizing data warehousing solutions using Azure Synapse Analytics and SQL-based architectures.
Programming Languages: Strong proficiency in T-SQL, Python, DAX or Scala for data processing and scripting.
ETL/ELT Pipelines: Proven expertise in building SQL Server-based ETL/ELT pipelines and integrating with Microsoft Fabric and Azure tools.
Data Governance & Quality: Hands-on experience with SQL Server data quality frameworks, Microsoft Fabric's data governance capabilities, and security compliance.
DevOps & CI/CD: Familiarity with DevOps tools such as Azure DevOps, Git, and CI/CD pipelines for SQL Server deployment automation.
Infrastructure as Code (IaC): Experience using tools like Terraform, ARM templates, or Bicep for SQL Server resource and Microsoft Fabric provisioning.
Soft Skills:
Strong problem-solving and critical-thinking abilities.
Excellent communication skills with technical and non-technical stakeholders.
Proven leadership and mentoring experience.
Ability to work independently and in team settings in a fast-paced environment.
Highly organized with attention to detail.
Preferred Qualifications
Knowledge of machine learning pipelines and data science workflows.
Familiarity with containerization technologies (Kubernetes, Docker).
Exposure to data security frameworks. (GHIPAA, SOC 2, etc.)
Experience with C#, JavaScript, HTML/CSS languages a plus.
LICENSURE/CERTIFICATION:
Microsoft SQL Server and Azure certifications (e.g., MCSA: SQL Server, Azure Data Engineer Associate, Fabric Analytics Engineer) are highly desirable.