Performance Lead, AI Agent
- linkCopy link
- emailEmail a friend
Minimum qualifications:
- Bachelor’s degree or equivalent practical experience.
- 5 years of experience in an analytical role in customer operations or for an AI/ML product.
- 5 years of experience in manipulating datasets and conducting analyses using SQL and Python.
- Experience creating executive communications (e.g., memos, slide decks, etc.) and presenting to non-technical/analytics audiences.
Preferred qualifications:
- Master’s degree in Business, Statistics, Mathematics, Economics, Engineering or Applied Science, or a related field.
- Experience in data analysis and modeling, specifically for AI/ML products, including forecasting and predictive modeling.
- Experience developing Key Performance Indicators (KPIs) and measurement frameworks for Large Language Models (LLMs) or other generative AI products.
- Familiarity with experimental design (A/B testing) in the context of conversational AI or user-facing ML products.
- Ability to navigate ambiguity, manage projects, and drive issues to resolution.
- Excellent communication and presentation skills, with the ability to translate complex data into actionable insights.
About the job
Our goal is to ensure millions of creators and partners on YouTube have easy access to comprehensive support when and where they need it—a mission we are increasingly scaling through our AI agent—and to allow our creators’ voices to help shape product solutions.As a Performance Lead focused on our AI Agent, you will drive data-driven decision-making to enhance the creator experience on YouTube. You will translate complex conversational and performance data into actionable insights, using strong problem solving skills to identify key drivers of AI performance and solve defined business problems in automated support. You will also develop new measurement frameworks to improve AI agent operations and address future needs.
As a thought leader, you will contribute to strategic planning, defining objectives, and developing novel KPIs specifically for LLM-based support, providing data-backed recommendations. You will conduct discrete modeling on new AI capabilities and prompt changes to forecast their impact on containment, user satisfaction, and escalations, using historical data to improve accuracy. You will influence teams and build cross-functional relationships to align resources and drive projects forward. You will communicate complex findings to both executive leadership and stakeholders, driving a deeper understanding of AI performance and opportunities.
Finally, you will apply your expertise in business and data analytics, including statistical methods and research fluency, to design and evaluate A/B experiments on prompts and conversational flows, and conduct advanced analyses to optimize the end-to-end AI journey for creators across YouTube.At YouTube, we believe that everyone deserves to have a voice, and that the world is a better place when we listen, share, and build community through our stories. We work together to give everyone the power to share their story, explore what they love, and connect with one another in the process. Working at the intersection of cutting-edge technology and boundless creativity, we move at the speed of culture with a shared goal to show people the world. We explore new ideas, solve real problems, and have fun — and we do it all together.
Responsibilities
- Establish and manage the core KPIs for the AI Agent, including novel LLM-based metrics for quality and satisfaction.
- Analyze performance and conversational data to identify optimization opportunities and provide data-driven recommendations that shape the product roadmap.
- Structure and interpret A/B tests on prompts and conversational flows, delivering clear impact analyses to guide AI development and improve performance.
- Build forecasting models to project the impact of new AI capabilities and analyze pre- and post-launch data to measure their actual effect.
- Translate analytical findings into clear, actionable insights for PSM, Engineering, and Leadership to drive the overall AI support strategy.
Information collected and processed as part of your Google Careers profile, and any job applications you choose to submit is subject to Google's Applicant and Candidate Privacy Policy.
Google is proud to be an equal opportunity and affirmative action employer. We are committed to building a workforce that is representative of the users we serve, creating a culture of belonging, and providing an equal employment opportunity regardless of race, creed, color, religion, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, veteran status, marital status, pregnancy or related condition (including breastfeeding), expecting or parents-to-be, criminal histories consistent with legal requirements, or any other basis protected by law. See also Google's EEO Policy, Know your rights: workplace discrimination is illegal, Belonging at Google, and How we hire.
If you have a need that requires accommodation, please let us know by completing our Accommodations for Applicants form.
Google is a global company and, in order to facilitate efficient collaboration and communication globally, English proficiency is a requirement for all roles unless stated otherwise in the job posting.
To all recruitment agencies: Google does not accept agency resumes. Please do not forward resumes to our jobs alias, Google employees, or any other organization location. Google is not responsible for any fees related to unsolicited resumes.