Staff Data Platform Engineer

About us

Character’s mission is to empower everyone with AGI. Our vision is to enable people with our technology so that they can use Character.AI any moment of any day.

Achieving our mission will require solving ambitious technical challenges, including engineering, research, and design, and we are assembling a world-class team to do so. Our founding team includes AI pioneers from Google Brain and Meta Research whose research has led to major breakthroughs in natural language understanding and dialog applications such as Transformers and Google LaMDA.

Check out our beta to get a glimpse into the future.

About the Role

The data platform team at Character seeks to accomplish the following: 

1. Provide high quality data for analytics and model training

2. Remove the accidental complexity of Data Engineering tasks to empower ML Researchers, SWEs, and Data Scientists to move as quickly and confidently as possible.

3. Make data go vroom while gpus go brrr!
   

Responsibilities: 

• Make our Data Warehouse, Data Lake, and ML Research pipelines a joy to use with your infrastructure expertise, knowledge of big data systems, and keen eye for organization.

• Build internal systems for our Product and ML Research teams to manage datasets, run large-scale inference jobs, manage crowdworker data, and anything else under the DataOps umbrella.

• Make the most of vast compute resources to enable rapid development of AI products.

• Help manage and scale our Kubernetes deployments for training, data engineering, and data acquisition.

• Ensure our data platform can scale reliably to the next several orders of magnitude

Qualifications:

• Expertise with Data Platform architecture and big data tools (Spark, Apache Beam, Ray, PubSub/Kafka, etc). 

• Solid understanding of Spark and ability to write, debug and optimize Spark code. Apache Spark on K8s a plus.

• Experience building production systems and data pipelines in languages like Python, SQL, Go, Java, Scala 

• 5+ years experience with Kubernetes. Experience as a Cluster Administrator a plus.

• Experience with production cloud networking, service meshes (istio), CNIs (cilium, eBPF), and managing contention in shared clusters a plus.

Character is an equal opportunity employer and does not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status. We value diversity and encourage applicants from a range of backgrounds to apply.