Confluent KAFKA Administrator
Apply NowCompany: Virtusa Corporation
Location: Tampa, FL 33647
Description:
Description
Big Data Confluent KAFKA Administrator Tech Lead Architect
Creation of key performance metrics, measuring the utilization, performance, and overall health of the cluster.
Capacity planning and implementation of new upgraded hardware and software releases as well as for storage infrastructure.
Research and recommend innovative ways to maintain the environment and where possible, automate key administration tasks.
Document and share design, build, upgrade and standard operating procedures. Conduct knowledge transfer sessions and workshops for other members in the team. Provide technical expertise and guidance to new and junior members in the team
Minimum 6lus years of experience in BigData administration using KAKFA and related tools like Spark etc.
Minimum 4 years of experience in setting up the environment for solutions that collects and process data using Kafka Spark, Scala
Expert on Kafka Big Data solutions along with sound understanding of Kafka architecture
Manage single and multi-node Kafka cluster deployed on VM, Docker and Kubernetes Container platform. Experience with Confluent Platform running on prem
Perform Kafka Cluster build, MRC, including Design, Infrastructure planning, High Availability and Disaster Recovery
Implementing wire encryption using SSL, authentication using SASL LDAP authorization using Kafka ACLs in Zookeeper, Broker Client, Connect cluster connectors, Schema Registry, REST API, Producers Consumers, Ksql
Upgrades for the Kafka Cluster landscape comprising of Development, Test, Staging and Production/DR systems
Create topics, setup Apache Kafka MirrorMaker 2, Confluent Replicator to replicate the topics, create connect clusters, Schemas for the topics using Confluent Schema Registry
Configure various Opensource and licensed Kafka Source/Sink Connectors such as Kafka Connect for SAP HANA, Debezium Oracle and MySQL Connectors, Confluent JDBC source sink, Confluent ADLS2 Sink connector and Confluent Oracle CDC source connector
Develop and maintain Unix scripts to perform day to day Kafka Admin and Security related functions using Confluent REST Proxy server
Setting up monitoring tools such as Prometheus, Grafana to scrape metrics from various Kafka cluster components Broker, Zookeeper, Connect, REST proxy, Mirror Maker, Schema Registry and other endpoints such as webservers, databases, logs etc. and configure alerts for Kafka Cluster and supporting infrastructure to measure availability and performance SLAs
Linux SLES or RHEL system administration basic or advanced, creating shell scripts
Working experience on docker and Kubernetes clusters opensource, Rancher, RedHat OCP, Client Tanzu involving administration of containers Operator level skills, deployments, updates, integration with products running outside of the cluster
Working knowledge with container registry such as Harbor, Quay, Nexus etc. Exposure to
Database administration skills in Oracle, MSSQL, SAP HANA, DB2, Aerospike, Postgres
Exposure to SaaS based observability platform like New Relic
Deployment of container images and pods using CI CD pipelines using Jenkins or comparable tools.
Experience in building Kafka deployment pipelines using Terraform, Ansible, Cloud
Worked in Public cloud environment such as Azure or AWS or GCP, preferably in Azure
Effective written and verbal communication skills
Effective analytic diagnostic skills
Good presentation skill
Big Data Confluent KAFKA Administrator Tech Lead Architect
Creation of key performance metrics, measuring the utilization, performance, and overall health of the cluster.
Capacity planning and implementation of new upgraded hardware and software releases as well as for storage infrastructure.
Research and recommend innovative ways to maintain the environment and where possible, automate key administration tasks.
Document and share design, build, upgrade and standard operating procedures. Conduct knowledge transfer sessions and workshops for other members in the team. Provide technical expertise and guidance to new and junior members in the team
Minimum 6lus years of experience in BigData administration using KAKFA and related tools like Spark etc.
Minimum 4 years of experience in setting up the environment for solutions that collects and process data using Kafka Spark, Scala
Expert on Kafka Big Data solutions along with sound understanding of Kafka architecture
Manage single and multi-node Kafka cluster deployed on VM, Docker and Kubernetes Container platform. Experience with Confluent Platform running on prem
Perform Kafka Cluster build, MRC, including Design, Infrastructure planning, High Availability and Disaster Recovery
Implementing wire encryption using SSL, authentication using SASL LDAP authorization using Kafka ACLs in Zookeeper, Broker Client, Connect cluster connectors, Schema Registry, REST API, Producers Consumers, Ksql
Upgrades for the Kafka Cluster landscape comprising of Development, Test, Staging and Production/DR systems
Create topics, setup Apache Kafka MirrorMaker 2, Confluent Replicator to replicate the topics, create connect clusters, Schemas for the topics using Confluent Schema Registry
Configure various Opensource and licensed Kafka Source/Sink Connectors such as Kafka Connect for SAP HANA, Debezium Oracle and MySQL Connectors, Confluent JDBC source sink, Confluent ADLS2 Sink connector and Confluent Oracle CDC source connector
Develop and maintain Unix scripts to perform day to day Kafka Admin and Security related functions using Confluent REST Proxy server
Setting up monitoring tools such as Prometheus, Grafana to scrape metrics from various Kafka cluster components Broker, Zookeeper, Connect, REST proxy, Mirror Maker, Schema Registry and other endpoints such as webservers, databases, logs etc. and configure alerts for Kafka Cluster and supporting infrastructure to measure availability and performance SLAs
Linux SLES or RHEL system administration basic or advanced, creating shell scripts
Working experience on docker and Kubernetes clusters opensource, Rancher, RedHat OCP, Client Tanzu involving administration of containers Operator level skills, deployments, updates, integration with products running outside of the cluster
Working knowledge with container registry such as Harbor, Quay, Nexus etc. Exposure to
Database administration skills in Oracle, MSSQL, SAP HANA, DB2, Aerospike, Postgres
Exposure to SaaS based observability platform like New Relic
Deployment of container images and pods using CI CD pipelines using Jenkins or comparable tools.
Experience in building Kafka deployment pipelines using Terraform, Ansible, Cloud
Worked in Public cloud environment such as Azure or AWS or GCP, preferably in Azure
Effective written and verbal communication skills
Effective analytic diagnostic skills
Good presentation skill