Benefits:

• Competitive salary and equity packages

• Health, dental, and vision insurance plans

• 401k plan

• Unlimited PTO

• Fun global offsites!

Education Requirements:

• BS/MS in Computer Science, Mathematics, Statistics, or other quantitative field

Experience Requirements:

• Strong engineering background, preferably in machine learning or data science

• Experience with programming languages like Python, Java, C++, or similar

• Experience with cloud environments like AWS

• Excellent communication and analytical problem-solving skills

• Willingness to travel to customers if needed

Other Requirements:

• Have good character, integrity, and respect for others!

Responsibilities:

• Apply creativity, analytical ability, and technical skills to solve key AI problems for customers

• Set up and walk through custom product demos to showcase platform capabilities to prospective customers

• Create example scripts, develop custom evaluation datasets, build custom web apps, and write technical docs to educate customers on how to use the platform

• Guide customers on best practices for LLM evaluation and advise them on AI strategy

• Write technical blog posts and evangelize the product and company on social media and at community events

Benefits:

• Competitive salary and equity packages

• Health, dental, and vision insurance plans

• 401k plan

• Unlimited PTO

• Fun global offsites!

Education Requirements:

• PhD in Computer Science, Mathematics, Statistics, Linguistics or other quantitative field.

Experience Requirements:

• Publications at leading AI conferences, journals or workshops, such as NeurIPS, ICML, EMNLP, ACL, AAAI.

• Experience conducting empirical NLP research in an academic or industry research lab.

• Knowledge and understanding of state-of-the-art machine learning concepts, with a focus on NLP and search architectures.

• Experience training language models in applied or research settings.

• Experience working and communicating cross functionally in a team environment.

Other Requirements:

• Have good character, integrity and respect for others.

Responsibilities:

• Lead a team of Research and ML engineers to conduct research experiments and translate findings to applied AI features in the product.

• Solve challenging, open ended problems in AI evaluation research.

• Work closely with the CTO to set and drive research vision for the company.

• Scope and drive research projects, including experiment design, timelines for research deliverables, results analysis.

• Synthesize literature on AI evaluation and LLM development.

Benefits:

• Competitive salary and equity packages

• Health, dental, and vision insurance plans

• 401k plan

• Unlimited PTO

• Fun global offsites!

Education Requirements:

• BS/MS in Computer Science, Mathematics, Statistics, or other quantitative field; PhD preferred

Experience Requirements:

• Knowledge and understanding of state-of-the-art machine learning concepts, with a focus on NLP.

• Familiarity with transformer-based architectures, attention mechanisms, evaluation metrics and benchmarks

• Experience training language models in applied or research settings.

• Deep familiarity with pytorch and ML tooling such as job schedulers, the Hugging Face transformers library, and Weights & Biases.

• Experience conducting AI research in an academic or industry research lab.

Other Requirements:

• Have good character, integrity and respect for others!

Responsibilities:

• Develop state-of-the-art systems for AI evaluation.

• Train language models for novel use cases, such as evaluating whether content is engaging, hallucinatory, age appropriate, or contains PII.

• Evaluate whether language models are aligned with human preferences.

• Collect high quality, novel datasets for classification and generative tasks, through synthetic data augmentation techniques and publicly available datasets.

• Conduct novel research on red teaming language models.