Staff Engineer - Data Processing / Messaging

The Data Processing and Workflow Foundations (DPWF) team is looking to hire a Staff Software Engineer that thrives at the intersection of infrastructure and software development. This team’s challenges break into 2 domains, which we consider engineering kafka as a platform and promoting event driven architectures.

Engineering kafka as a platform: Higher-level orchestration of Kafka to support our multi-cloud, multi-region, deployments. A subset of current focuses include:

• Software automation that creates, manages, and destroys kafka clusters in our environments.

• APIs and controllers that support operation of kafka clusters at scale atop kubernetes

• Automation of replication and other cluster management concerns.

Promoting event driven architectures: Reddit is increasingly adopting geographically distributed, eventually consistent architectures. As part of this shift, many systems are shifting to event driven architectures, and the Messaging team is working to make this adoption simpler and faster. Current projects are:

• Schema Registries and enforcement

• Capacity management and ACLs through client enforced authentication

• GitOps and other developer experience tooling for Kafka interaction

As a member of the Messaging team, your work will span these 2 domains, which are rich with challenging infrastructure and software engineering problems. You will work with a team of engineers to develop a vision for asynchronous communication tooling. Your work will directly impact hundreds of millions of users around the world. Join us and help build the future of Reddit!

In your day-to-day, you can expect to:

• Work collaboratively with a team of software engineers to create and maintain the foundational platform for running Reddit’s infrastructure.

• Deliver software to improve the availability, scalability, latency, and efficiency of Kafka and other messaging components.

• Contribute feedback to the technical and strategic direction of eventing at Reddit.

• Automate critical aspects of the event driven development process

• Share on-call responsibilities. 

You have:

• 7+ years of experience developing internet-scale software, preferably in the context of infrastructure.

• Experience with asynchronous communication patterns and solutions, bonus if familiar with Kafka specifically

• Experience developing on top of Kubernetes or similar distributed systems.

  • Kubernetes controller or operator development experience is a huge plus.

• Strong troubleshooting capabilities surrounding both systems and software.

• Experience engineering large systems, tracking work, and being a self-starter on projects.

• Excellent communication skills to collaborate with a service-oriented team and company.

• Experience navigating cross-functional migrations

• Experience mentoring other engineers.

Benefits:

• Comprehensive Healthcare Benefits

• 401k Matching

• Workspace benefits for your home office

• Personal & Professional development funds

• Family Planning Support

• Flexible Vacation (please use them!) & Reddit Global Wellness Days

• 4+ months paid Parental Leave

• Paid Volunteer time off

#LI-remote, #LI-JS5