Azure Data Integration Engineer

About Us:
Building Services 32BJ Benefit Funds ("the Funds") is the umbrella organization responsible for administering Health, Pension, Retirement Savings, Training, and Legal Services benefits to over 100,000 SEIU 32BJ members. Our mission is to make significant contributions to the lives of our members by providing high quality benefits and services. Through our commitment, we embody five core values: Flexibility, Initiative, Respect, Sustainability, and Teamwork (FIRST). By following our core values, employees are open to different and new ways of doing things, take active steps to improve the organization, create an environment of trust and respect, approach their work with the intent of a positive outcome, and work collaboratively with colleagues.

For 2023 and beyond, 32BJ Benefit Funds will continue to drive innovation, equity, and technology insights to further help the lives of our hard-working members and their families. We use cutting edge technology such as: M365, Dynamics 365 CRM, Dynamics 365 F&O, Azure, AWS, SQL, Snowflake, QlikView, and more. Through this technology investment, we have gathered and analyzed thousands of data insights to influence health insurance legislation and propose new health policy. Our efforts have galvanized many leaders and the consensus is there is plenty more work to be done.

Please take a moment to watch our video to learn more about our culture and contributions to our members: youtu.be/hYNdMGLn19A

Summary: Under the supervision of the Team Lead, Data Integration, the Azure Data Integration Engineer will be for designing, developing, implementing, and maintaining data integration solutions using Azure Data Factory, Azure Databricks, and SQL Server Integration Services. The Azure Data Integration Engineer will collaborate with cross-functional teams to understand data requirements, extract, transform data from various sources, and load it into the data warehouse for analysis, application integration, and reporting purposes. TheAzure Data Integration Engineer should have a strong background in data integration, data engineering, data analysis, and various Azure technologies with best practices.

Essential Duties and Responsibilities:

• Design, develop, and implement data integration solutions using Azure Data Factory to meet business requirements and data ingestion needs.
• Collaborate with data architects, business analysts and developers to understand data requirements and ensure data pipelines are optimized for performance, scalability, and reliability.
• Extract, transform, and load (ETL) data from various sources, including on-premises and cloud-based systems, APIs, databases, and files.
• Monitor and troubleshoot data pipelines to identify and resolve issues in a timely manner to minimize disruptions in data processing for on premise and cloud environments.
• Implement data quality checks and validation processes to ensure accuracy and completeness of data.
• Work with DevOps teams to manage CI/CD pipelines for deploying and managing data integration solutions.
• Optimize data integration processes for performance and cost-effectiveness, making use of Azure Data Factory features and capabilities.
• Optimize performance for SQL query, stored procedures, and table index.
• Maintain documentation for data pipelines, configurations, and data flow diagrams.
• Collaborate with data scientists, data analysts, and other stakeholders to understand data usage patterns and provide recommendations for data optimization and improvements.
• Stay updated with the latest trends and best practices in data engineering, cloud computing, and Azure services to suggest innovative solutions.
• Develop and design database schema, SQL queries and tuning, and stored procedures.
• Write well designed, testable, efficient code, that adheres to security standards.
• Ensure designs follow specifications.
• Production and End User Support Activities including developing documentation and assistance tools.
• Perform other relevant tasks as required by management/supervisory staff.
• Provide support after hours, on weekends for emergency issues and production releases as required.

Qualifications: To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required.

• Proven experience in designing, developing, and deploying data integration solutions using Azure Data Factory and Azure Databricks.
• Strong understanding of data engineering concepts, ETL principles, and data modeling.
• Strong experience in SQL server includes T-SQL, stored procedures, CTE, functions, SSIS and SQL agent jobs.
• Proficiency in Azure cloud services, including Azure Blob Storage, Azure SQL, Azure Data Lake, Azure Service Bus, and Azure Data Factory.
• Experience with data integration techniques for both structured and unstructured data.
• Solid programming skills in languages such as Python, PowerShell, and C#
• Experience with Rest API for data integration.
• Familiarity withdata governance, data catalog & profiling, data security, data encryption (RSA, PGP), and compliance in the context of cloud-based data solutions.
• Knowledge of Azure DevOps, Git Repos and CI/CD pipelines for code deployment is a plus.
• Excellent problem-solving skills and attention to detail.
• Strong communication and interpersonal skills to work effectively in a collaborative team environment.
• Knowledge inPython,PySpark/Ray framework, Prefect Workflows, and Healthcare data domain experience preferred.
• Familiarity with Dynamics CRM and Dynamics F&O is a plus.

Interpersonal Skills:

• Detail oriented with excellent organization and analytical skills.
• Ability to plan and take initiatives to accomplish objectives in a timely fashion.
• Ability to prioritize work and meet deadlines.
• Ability to establish and maintain effective working relationships with project team members, supervisors, and employees from other departments.

Education and/or Experience: Bachelor's degree in Computer Science, or a related discipline.

Language Skills: Speak, read, write and understand English.

Reasoning Ability: High

Certificates, Licenses, Registrations: None

Physical Demands: The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodation may be made to enable individuals to perform the essential functions.

• Under 1/3 of the time: Standing, Walking, Climbing or Balancing, Stooping, Kneeling, Crouching, or Crawling
• 1/2 to 2/3 of the time: Sitting, Reaching with Hands & Arms
• Over 2/3 of the time: Talking or Hearing
• 100% of the time: Using Hands

Work Environment: The work environment characteristics described here are representative of those an employee encounters while performing the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.

• 1/3 to 2/3 of the time: Work near moving or mechanical parts, exposure to radiation, moderate noise.