Senior Software Engineer - Event Capture & Processing (Kafka)
Atlan
Administration, Software Engineering
Washington, DC, USA
Posted on Tuesday, October 10, 2023
What will you do?
- Ensure Reliability, Availability & Scalability of Kafka Clusters
- Collaborate with the team to ensure Kafka clusters meet reliability, availability, and scalability requirements.
- Optimize Kafka configurations and settings to enhance performance and stability.
- Design and implement effective clustering, partitioning, and topical strategies for efficient Kafka message processing.
- Develop and implement monitoring and observability solutions to gain insights into Kafka cluster performance and consumer behaviour.
- Develop robust consumers over Atlan's event processing engine (Numaflow) to ensure fault tolerance and reliability.
What makes you a match?
- Demonstrated expertise in deploying and managing applications using Kubernetes.
- Experience working with Custom Resource Definitions (CRD) to define and manage custom resources.
- Proven track record of working with Kafka and utilizing its features to build efficient and scalable solutions.
- Familiarity with Atlan's event processing engine (Numaflow) is a significant advantage.
- Demonstrated experience in scaling Kafka clusters and consumers to handle increased load and demand.
- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration abilities.
- Proven ability to work effectively in a team-oriented, collaborative environment.