Kafka Development and Support/ Senior Developer

Title: Kafka Development and Support/ Senior Developer

Work Location: VANCOUVER, BC (Hybrid)

Job duties/accountabilities

Position Overview: The Kafka Development and Support Senior Developer is responsible for designing, developing, implementing, and supporting Apache Kafka-based solutions within the organization. This role involves working closely with cross-functional teams to deliver high-performance, scalable, and reliable streaming data solutions. The senior developer will also take ownership of Kafka-related troubleshooting, performance tuning, and issue resolution to ensure the smooth operation of Kafka-based services.

Key Responsibilities:

Kafka Development:

• Design, develop, and maintain Kafka-based streaming data solutions, including producers, consumers, and stream processing applications.
• Work with Kafka topics, partitions, replication strategies, and message formats to build scalable, high-performance systems.
• Implement Kafka Connectors to integrate Kafka with various external systems (e.g., databases, file systems, and other messaging systems).
• Develop and implement Kafka Streams applications to process real-time data streams and transformations.

Kafka System Support:

• Provide second and third-level support for Kafka-related issues, including diagnosing and troubleshooting problems with message delivery, cluster health, and system performance.
• Monitor Kafka clusters and ensure that they are running smoothly with high availability and minimal downtime.
• Respond to and resolve incidents, service requests, and performance bottlenecks affecting Kafka-based services.
• Perform regular cluster health checks and implement necessary patches or upgrades.

Performance Tuning and Optimization:

• Monitor Kafka system performance and identify opportunities for optimization.
• Tuning Kafka configurations (e.g., retention policies, partitioning, replication) to improve throughput, latency, and availability.
• Troubleshoot Kafka-related issues, such as latency spikes, message loss, or consumer lag.
• Implement best practices for Kafka consumer group management and scaling.

Collaboration and Knowledge Sharing:

• Collaborate with other developers, data engineers, and system administrators to design Kafka-based solutions that integrate with other components in the ecosystem.
• Provide technical guidance and mentoring to junior developers and peers on Kafka best practices.
• Participate in code reviews, providing constructive feedback to enhance the quality and performance of the codebase.
• Maintain proper documentation for Kafka-based solutions, including development guidelines, operational procedures, and troubleshooting steps.

Support Automation:

• Automate routine Kafka maintenance tasks such as configuration management, monitoring, and alerting.
• Build and improve CI/CD pipelines to automate the deployment of Kafka applications and related components.
• Use infrastructure-as-code (IaC) tools like Terraform or Ansible for Kafka-related deployments and management.

Security and Compliance:

• Implement security measures for Kafka clusters, including SSL/TLS encryption, SASL authentication, and proper access control.
• Ensure compliance with security policies and best practices, including monitoring access logs and auditing Kafka systems.

Upgrades and Patch Management:

• Plan and execute Kafka upgrades and patches in coordination with system administrators to minimize downtime.
• Test and validate new versions of Kafka and associated components before rolling them out into production.

Troubleshooting and Issue Resolution:

• Investigate and resolve complex issues with Kafka clusters, consumers, producers, or stream processing applications.
• Debug and analyze log files, metrics, and monitoring systems to pinpoint issues related to performance, reliability, or data integrity.
• Provide root cause analysis and detailed reports for any Kafka-related incidents.

Incident Management:

• Participate in the resolution of critical incidents and provide post-incident reports to management.
• Implement preventive measures to avoid the recurrence of similar issues in the future.

Required Skills and Qualifications:

• Education: Bachelor's degree in Computer Science, Information Technology, Engineering, or related field.
• Experience:
  • 5+ years of hands-on experience with Apache Kafka, including designing, developing, and managing Kafka-based solutions.
  • Proven experience in supporting and troubleshooting Kafka clusters in production environments.
  • Familiarity with Kafka administration tasks, including managing topics, brokers, producers, and consumers.
  • Experience with stream processing frameworks like Kafka Streams, KSQL, or other real-time processing solutions.
• Technical Skills:
  • Proficiency in Java, Scala, or Python for Kafka producer/consumer development.
  • Strong understanding of Kafka concepts, such as topics, partitions, replication, consumer groups, and brokers.
  • Experience with Kafka Connect and integrating Kafka with external systems (e.g., databases, file systems).
  • Experience with monitoring and alerting tools (e.g., Prometheus, Grafana, Elasticsearch) for Kafka clusters.
  • Strong understanding of performance tuning and troubleshooting in distributed systems.
  • Familiarity with containerization and orchestration tools like Docker and Kubernetes.

Knowledge of cloud platforms (AWS, GCP, Azure) and experience deploying Kafka on these platforms is a plus.