🚨 Enrollment for the Final 2025 Cohort Closes Soon The September Cohort is SOLD OUT. This is your last chance to join in 2025. After the course, a frequent comment we hear is, "My only regret is not enrolling sooner." To help you turn your new skills into immediate career results, we're including our Momentum Mastermind Coaching program with your enrollment. Use code 'MOMENTUM' at checkout to unlock:* Momentum Mastermind Coaching ($1,000 Value): A private coaching program with our Executive-in-Residence to accelerate your career goals.* A Surprise $500 Off Your Enrollment: our biggest discount of the year.* This is the final offering until 2026 and the MOMENTUM discount expires Sunday, Sept 22nd, 2025. Team Enrollment (3+): Contact us for special pricing and arrangements at support@productfaculty.com Why it matters: AI Product Manager is the new Product Manager In 2025, every product decision will be an AI decision. This course will allow you to master the skills you need to lead in the AI-driven future. 👨‍💻 𝗔𝗕𝗢𝗨𝗧 𝗧𝗛𝗘 𝗜𝗡𝗦𝗧𝗥𝗨𝗖𝗧𝗢𝗥 Miqdad Jaffer, a Product Leader at OpenAI, has 18 years of experience in building products at some of the world's most successful companies. Currently, he leads enterprise products at OpenAI and was previously the Head of AI Products at Shopify. Miqdad is one of the world's top AI executives and an award-winning AI Product educator. His work on AI has been featured in TechCrunch, MIT’s AI Podcast, FastCompany, The Globe and Mail, and Spotify. He has been listed as one of the top 100 people to follow in Artificial Intelligence by Business Insider and has received many other awards and accolades. If you have any questions about the training, please reach out to me support@productfaculty.com. 💡 𝗔𝗕𝗢𝗨𝗧 𝗧𝗛𝗘 𝗖𝗢𝗨𝗥𝗦𝗘This course is not just about learning; it's about doing. It’s an exercise-heavy training that goes beyond gaining "AI speak" to enhance your credibility and confidence to build AI Products and future proof your career.&nbsp;The live online sessions spread across three classes that will equip you with a comprehensive AI Product playbook. This playbook covers everything from AI use-case validation, product design, development, and delivery, to launch strategies, metrics, and growth.&nbsp;Learn from successful AI product features at leading companies like Shopify, Google, Klarna, and Amazon, and apply these insights in a capstone project where you'll build your own AI-powered feature. Topics Covered (Amongst Others):&nbsp;• Taxonomy of Prompting Techniques• Basic AI Concepts and Processes• Training and Model Improvement• Quality, Bias, and Interpretability• Computational Concepts (Deterministic vs. Non-Deterministic)• Qualitative vs programmatic evaluation&nbsp;• Model Customization Framework• Types of AI Models and Applications• Chain of Thought Prompting• Self-Reflection &amp; Self-Consistency• ReAcT Prompting Framework• Retrieval Augmented Generation (RAG)• Fine-Tuning &amp; RLHF• Function Calling &amp; Tool Usage• LLM-Powered Agents• LLM Evaluation &amp; Judge LLMs• AI Safety &amp; Moderation Tools• Adversarial Prompting (Jailbreaking and Prompt Injections)• Common Real-World Use Cases of LLMs• Understand Advanced Prompting Techniques&nbsp;• Zero-shot prompting• Few-shot prompting/in-context learning• Chain-of-Thought (CoT) 💡 COURSE PREREQUISITES﻿This course is NOT for you if:• You work exclusively on non-software products (hardware, pharmaceuticals, deep climate tech, defense tech, etc.)• You're looking for a purely theoretical overview with no hands-on practice• You're looking for a low-effort introduction to AIWhat to expect:• A structured learning path that starts with the basics – no prior coding required• Beginner-friendly guidance through setting up the OpenAI Playground environment• Practical exercises that build your confidence step-by-step• With dedicated effort, you'll achieve results that directly apply to your work 💡 𝗔𝗕𝗢𝗨𝗧 PRODUCT FACULTY&nbsp;Product Faculty is one of the most recognized names in the PM training industry and has been around for over 8 years. At Product Faculty, we’ve trained and mentored over 10K product managers working at some of the leading companies in&nbsp;the world. Product Faculty’s training has some of the highest NPS (Net promoter scores in the industry with most classes having an NPS of 70+ (considered exceptional and reflect outstanding customer satisfaction). Product Faculty has been recognized as the top rated Product Management Bootcamp from the following organizations:&nbsp;&gt; 2023 Course Reports - Best PM Bootcamp&gt; 2022 Course Reports - Best PM Bootcamp&gt; Career Carma - #1 Ranked PM Bootcamp&gt; CourseCompare - Best PM Bootcamp&gt; Switchup - Best Bootcamp Reach out to support@productfaculty.com for any questions, corporate trainings, and group discounts.

Acquire the skills & confidently build AI products and future proof your career

Course syllabus

Access hours of expert-led content on your schedule. Build foundational knowledge in AI Product Management with in-depth modules designed for flexibility. Perfect for busy professionals looking to upskill at their own pace and apply learnings immediately.

OnDemand Content

Join live sessions to work through real-world AI case studies. Engage with exercises and get your questions answered by expert Miqdad. Gain practical insights that turn theory into action, with direct support tailored to your unique challenges.

Live Case Study and Q&A

Complete a hands-on AI capstone that simulates launching a real AI feature. Tackle ideation to deployment with weekly feedback from experts. Finish with a portfolio-ready project that showcases your skills in real-world AI Product Management.

Capstone Project

Timings for each session have been chosen based on what works best for most product people in practice. Don't worry about missing classes, recordings for all modules will be made available for you to watch in your Maven account.

Optimized for very busy schedules of Senior PM, GPMs and VPs

Course schedule

For PMs &amp; Leaders: The #1 rated certificate to master the age of AI. Learn the A-to-Z playbook to ship successful AI products.

For Engineers &amp; Tech Leads: Don't just build. Lead. Get the product strategy to make better tech decisions &amp; fast-track your career.

For Founders, Designers &amp; UX: The playbook to build a winning AI venture or design human-first AI experiences users will love.

Who is this course for

What you’ll get out of this course

Clear, actionable processes for analyzing AI use cases and developing strategies towards tangible outcomes.

Solve Real Problems with AI

I’ll act as your personal innovation guide as you create functional AI applications, no prior coding experience needed.

Build Real AI Prototypes

Grasp foundational models and large language models (LLMs), and learn to code and train models independently.

Dive Deep Into AI's Technical Side

Transform theory into practice by developing your AI product idea or tackling your company's AI challenge, guided by me.

Hands-on Capstone Project

Discover essential strategies for enhancing decision-making and using data to steer results in your professional and personal life.

Make Better, Data-Driven Decisions

Master the art of managing risks by blending automation with human oversight to maximize value from AI applications.

Balance Automation with Oversight

Utilize in-context learning for efficient prototyping, RAG applications, and more, without getting engineers involved.

Learn Prompt Engineering

Deepen your understanding of AI memory/context and Retrieval Augmented Generation (RAG) to enrich AI interactions.

Master Retrieval Augmented Generation

Gain a comprehensive understanding of crucial AI algorithms, concepts, and techniques to foster organization-wide knowledge.

Boost Data & AI Literacy

Master the art of pitching your AI product ideas so clearly and excitingly that everyone's eager to jump on board.

Make your ideas irresistible: 

This intensive weekend course is tailored for professionals with weekday commitments, complete with recorded sessions for revisiting.

Optimized for Busy Senior PMs

Secure lifelong entry to our expansive alumni network for unparalleled opportunities, job connections, and continuous learning.

Lifetime Access to Product Faculty Community

Equip yourself with cutting-edge AI skills and insights to stay ahead in the evolving tech landscape, ensuring your career remains resilient and growth-oriented.

Future-Proof Your Career

#1 Rated AI Certificate. Acquire the skills needed to confidently build AI products, positioning yourself for success in the AI world

Miqdad Jaffer

Product Leader at OpenAI, TechCrunch & MIT featured, Top 100 by Business Insider

Cohort 6

Cohort 7

#1 AI PM Course - Build & launch real AI products with a no-code tool, team support & engineers—earn 3 certifications + pitch at Demo Day.

September 2025

Dr. Marily Nika

Gen AI PM Lead @ Google | ex-Meta, Fellow @ Harvard | TED AI Speaker | 40 u 40

Amazon

𝗚𝗲𝘁 𝗰𝗲𝗿𝘁𝗶𝗳𝗶𝗲𝗱 &amp; 𝗹𝗮𝘂𝗻𝗰𝗵 𝗮𝗻 𝗮𝗰𝘁𝘂𝗮𝗹 𝗔𝗜 𝗽𝗿𝗼𝗱𝘂𝗰𝘁 (you get an 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 𝗮𝘀𝘀𝗶𝗴𝗻𝗲𝗱 𝘁𝗼 𝘆𝗼𝘂𝗿 𝘁𝗲𝗮𝗺)👩‍💻 ABOUT THE INSTRUCTOR Dr. Marily Nika, Gen AI Product Lead at Google (ex-Meta), most followed AI educator, Harvard Business School fellow, best selling author, fortune 40 under 40💡 ABOUT THE COURSE The world’s first &amp; top-rated AI Product Management Certification, taken by 20k+ professionals. Learn from Marily &amp; experts from OpenAI, Anthropic, Amazon &amp; Meta through 65 lessons, 17 hours live content, hands-on exercises, and group coaching.🤖 ABOUT THE CAPSTONE Unlike any other course: you’ll join a team, get exclusive access to a no-code prototyping tool that generates full front-ends &amp; actual products (UI included), and have a dedicated engineer to ensure your idea becomes a fully functional product. You’ll launch it, pitch at Demo Day, and present in front of an EIR scout looking for standout founders.📜 Earn 3 industry-recognized certifications + $1,395 in free credits for top PM tools.✨ Topics: AI PM 101, AI Strategy, AI Evals, Metrics, LLMs, AI Agents, System Design, Deployment &amp; more.

“All Product Managers will be AI Product Managers in the future” - Marily Nika. 

Launch a fully functional AI product & earn 3 certifications.

Accelerate your AI product career

Master the AI product lifecycle

Pitch products at Demo Day with expert feedback

Join a thriving AI product community.

Seasoned Product Leaders ready to pivot into AI Product Management and drive AI strategy inside their companies.

Career Accelerators &amp; Job Seekers of all levels aiming to build a portfolio project, and stand out in the job market.

Software Engineers &amp; Data Scientists who want to transition into PM roles, expand beyond technical skills, and lead AI initiatives.

Have you seen other PMs leveraging AI to ship prototypes at lightning speed , but when you try it yourself:- Basic tutorials leave you stuck on simple examples ?- Technical obstacles slow your momentum ?- Debugging feels impossible without coding experience ?- You're unsure which AI tools actually deliver value ? You know mastering AI prototyping could transform your workflow. But the path from curiosity to confidence isn't clear. With AI, you can build functional prototypes in minutes. The challenge with AI prototyping isn't getting started — it's understanding what these tools can and can't do. That’s why this cohort is designed to expose you to essential AI prototyping tools and their unique capabilities. You'll learn to debug issues (even without coding experience) to stay productive and deliver real value. And our community will be here to give you the support, guidance, and accountability you’ll need to finally stick with it. In just 4 weeks, you'll go from zero experience to shipping multiple prototypes -- from interactive prototypes to accompany your PRD, to personalized tools that boost your productivity. Now imagine if you could: ✅ Ship prototypes in hours without getting stuck in an infinite loop ✅ Learn how software applications work by building them ✅ Build personalized software to fit your exact needs ✅ Master advanced concepts like integrations to supercharge your prototypes Getting started is easy. 1️⃣ Begin with my 5-day email course to build your first AI prototype this week (sign up below!) 2️⃣ Join the 4-week cohort to master AI prototyping tools, debug common issues, and integrate with other apps in minutes&nbsp; 🔥 All alumni can stay up to date with our private community, 1:1 support, and more! 📣 Live sessions are at 1PM EST! 🎉&nbsp;Bonus:&nbsp;When you enroll in this course, you’ll receive $1,395 in free credits to level up your workflow with Superhuman, Linear, Sprig, Dovetail &amp; Gamma PLUS one month sponsored with Replit Core &amp; Bolt Pro (see FAQ for details) Interested in an enterprise license so your whole team or company can take the course? Please reach out directly to colin@techforproduct.com

Convert your PRD to a prototype in minutes

Get 1:1 help with your prototypes and questions

Office Hours

Live, interactive sessions where we dive deeper into AI prototyping and go through scenarios

Live Sessions

Complete a weekly project to apply what you've learned

Weekly projects

Senior Product Managers who want to leverage AI to iterate faster

Product Leaders who want to get first-hand experience with how AI is transforming product development

UX Designers who want to translate their designs into functional prototypes the same day

Master cutting-edge AI tools like Bolt, V0, Replit, and Claude to accelerate your development process. You'll learn practical workflows that go beyond basic prompts to create functional prototypes.

Learn to use AI prototyping tools

Get hands-on experience building real-world applications, from simple MVPs to complex features. Through guided projects, you'll develop a portfolio of working prototypes that demonstrate tangible AI development skills.

Build 5+ prototypes in 4 weeks

Develop a mental model of how modern applications are structured and how different components work together. This foundation helps you make better architectural decisions and communicate more effectively with developers.

Build a fundamental understanding of how software works

Learn systematic approaches to troubleshoot common AI coding issues and improve your prototype quality. You'll master techniques for error handling, prompt optimization, and code refinement that help you build production-ready features.

Discover 6 strategies for debugging to get high-value apps

Connect your prototypes to popular platforms and APIs without complex setup. You'll learn streamlined integration patterns that work with tools like Slack, Google Sheets, and your existing product stack.

Integrate with your favorite tools in minutes

Get direct access to an experienced PM and builder (me!) who can help you resolve technical and product challenges.

Weekly office hours and Q&A

Build your first AI prototype next week -- even if you have no coding skills

Colin Matthews

Sold my first SaaS in 2021. Product Manager, 8500+ students, featured by Lenny.

Cohort 8

12+ total hours of content: 6+ hours of core lecture material video recordings for self-learning followed by 6+ hours of live sessions (Q&amp;A, implementation advice, and peer networking). All live sessions are video recorded so you can catch up and revisit as needed. Learn the Standards of Executive Performance- How to know where you stand- How to break out of being stuck- How to develop and demonstrate executive presence- How to gain executive visibility- How to engage executive sponsors- How the leaders above you are evaluated and thus how you can help them- How to get scope, including during a flat or negative growth environment- How to overcome obstacles and poor managers- How to become a scaled deep leader, change agent, and trusted decision maker- How to manage your team to execute with little oversight, take calculated risks, and deliver results- How to think strategically and be seen as a influential leader who enhances the whole company- How to lead AI adoption Learn How to Manage the Executive Promotion Process- How to position yourself for promotion- How executives assess promotion readiness- Overcoming blocks to promotion- How to deal with a manager who gives little feedback- How to manage peers and sponsors- How to win skip level support- How to run the document creation process Scope, team size, and consistent high-impact results are important...But are not enough. I will teach you specific actionable steps on how to prioritize the things that matter to executives. Bonus material:- Executive Presence handout by Mariposa Leadership (one of our recommended coaching options)- Recommended coach list- Class slides as a PDF- 20% off all career development on-demand courses Advice is based on Ethan's personal promotion experience from Amazon Senior Manager to Director to Vice President and diverse insights from his network of executives, coaches, venture capitalists, army officers, and startup founders. Proven ROI for promotions and internal/external hires: Ethan promoted 8 reports from Senior Manager (Amazon L7) to Director (Amazon L8), contributed to 25+ Director promotions, hired 10+ Directors internally and externally, drove the promotion of 3 engineers to Principal, and of his former reports 2 are current Amazon VPs (L10) and 5 are C-Suite outside of Amazon. More than 1,000 senior managers and leaders worldwide have taken this course (including engineering managers, senior product managers, general managers, head of partnerships and business development, design leaders, and directors/VPs who have also found the course applicable to their careers). Many leaders buy this course for their team: Citing it is a big net positive for an organization by up-leveling team skills with practical and actionable insights. Multiple executives and leaders have bought 10+ seats for their direct reports / team. Alumni from: Amazon, Meta, Microsoft, Google, Apple, Netflix, Airbnb, Uber, Stripe, Shopify, Flexport, Oracle, Salesforce, Red Hat, Affirm, eBay, Expedia, Zillow, Groundspeed, Snapchat, TikTok, Unity, LinkedIn, Disney, McKinsey &amp; Company, Vimeo, JPMorgan Chase, HSBC, Kohl's, Capital One, Warner Bros, Discord, IBM, Vimeo, Waymo, and more startups &amp; large companies all over the world. Titles of course members range from Manager/Senior/Group/Principal to Director/VP/SVP/CPO. Alumni success stories:- "Hope all is well with you and your family Jason! Happy to share that due to executing some of the tactics shared in the program (taking action for one), I have been successful in getting myself promoted." — Promoted to Director - “What was key was that I actively participated in the live course AND followed through with some participants afterwards to talk shop and build relationships. One of those relationships just happened to be able to help when I needed it.&nbsp;Simply adding connections wouldn’t have been successful as they would have had no context about me, and not enough trust to put me in touch with important people in THEIR network.” — Landed FAANG Role - “When I took the ‘stuck at sr. manager’ course, Ethan mentioned that he knows that everyone wants to get promoted and wants it now but in the grand scheme of things, it’s not a big deal (paraphrasing, of course) since he knows us high performers are going to get there. Oddly, that comment didn’t make me lazier (“I guess i’ll get there anyway, so why try so hard”) but it grounded me a bit to take a breather and realize that it is not as urgent as I think it is and take the challenge (wanting a next level title) in stride!” — Promoted to Director - “I give so much credit to the Level Up: Breaking through to the Executive course and getting a front seat to Ethan's coaching and wisdom. Thank you for working to build this community that has given me to confidence to reach even in the current market environment! It is my dream role and I'm so grateful for the opportunity.” — Promoted to Director - “I am excited to share that I got promoted to Director and I sincerely thank and appreciate your help on this. I was not able to break through the barrier to Director level, partly due to constant changes happening in the company at leadership and strategies. I continued to apply my learnings from the community to influence the cross functional teams. Leadership was seeing the adaptability that I exhibited in continuing to add business value in new role and I was starting to get positive feedback from the VP on my flexibility. If I have to just pick one thing across all of Ethan’s teachings that helped me the most, it is being PROACTIVE. I am very grateful for your help and I really appreciate the learnings I have got as part of your community and your course on Stuck at Senior Manager. It really helped me to achieve this big milestone in my career. I thank you and Jason deeply as this promotion was very close to my heart." — Promoted to Director - "I’m excited to share some exciting news—I’ve landed a new job! I’m deeply grateful for your support and the scholarship. I can’t wait to apply everything I’ve learned in this new chapter of my career. Thank you again for your wisdom and generosity." — New Job TOP MAVEN COURSE- Top Course in Leadership &amp; Career- Top Instructor in Leadership- Featured in Maven's Executive Track- Featured in Maven's Fast Track- Featured in Maven 100- Featured in Lenny's List WHO THIS IS NOT FOR:- This is not for you if you want 1:1 coaching from Ethan.- This is not for you if you believe you already know what it takes to demonstrate executive performance and how to manage the promotion process.- This is not for you if you want a slow-paced experience. This course provides in-depth lectures in a 2-day weekend intensive followed by 4 weeks of implementation follow-up. It is designed for busy leaders who want to get the information and then quickly take action. BONUS:(From Maven): When you enroll in this course, you’ll receive $1,395 in free credits to level up your workflow with Superhuman, Linear, Sprig, Dovetail &amp; Gamma (see FAQ for details).

Level Up from Senior Manager to Executive

See the Course Syllabus section for exact live session dates and times

Ambitious Senior Managers from all disciplines at startups and large companies who want to become true executives.

Those managing large groups, managers of managers, including Amazon and Google L7, Meta M2, and Microsoft 66/67.

Also applicable to Senior Individual Contributors (ICs - Principal at Amazon) and Managers of 2-5 reports.

Many leaders buy this course for their team to help uplevel their skills with practical and actionable insights.

This is not for you if you want 1:1 coaching from Ethan.

What you will get out of this course

We will dive deep into specific tactics on how to get promoted, what great looks like, the executive bar, how to build a reputation that signals you are an executive, how senior executives assess promotion readiness, and how to drive team and company-wide impact.

How to get promoted to Executive

Learn how to stand out from your peers with specific tactics on how to better position and advocate for yourself, how to gain senior executive sponsorship, and how to influence the promotion approval meeting.

How to manage your promotion process

Course content is developed by Ethan's personal promotion experience from Senior Manager to Director to Vice President at Amazon along with input from Ethan's wide network of executives, coaches, VCs, army officers, and startup founders. You will get specific actionable advice.

Optimized actionable advice for Senior Managers who want to get promoted

Hear Ethan's break through experience first hand and learn from the specific actions Ethan took with his direct reports to help them break through to executive.

Learn from Ethan's personal stories (not theory)

<ul><li>How to identify the 10x AI opportunity</li><li>Your role as a leader and a change agent</li><li>How to lead from multiple personal positions</li><li>How to support your team</li><li>How to lead by example</li><li>Leadership actions to take now</li><li>The state of AI (April 2025)</li><li>Resources</li></ul>

Leading AI Adoption (new bonus module) 

Alumni from: Amazon, Meta, Microsoft, Google, Apple, Netflix, Airbnb, Uber, Stripe, Shopify, Flexport, Oracle, Salesforce, Red Hat, Affirm, eBay, Expedia, Zillow, Groundspeed, Snapchat, TikTok, Unity, LinkedIn, Disney, McKinsey &amp; Company, Vimeo, JPMorgan Chase, HSBC, and more.

You will be in great company

Learn actionable specific standards by which executives are selected and how to manage your promotion process to show you meet them. 

Ethan Evans

Retired Amazon Vice President; now Coaching Leaders to become True Executives

Cohort 9

You're excited about the opportunity to land a PM role at a major tech company, but the interview process feels like a maze. Despite all the advice online, you're still second-guessing yourself. What exactly are they looking for? How do you stand out? You've tried prepping on your own: reviewing countless resources, practicing frameworks, and memorizing examples. But you're not sure if you're actually getting better. You're stuck wondering, "Am I ready?" I've been there. Tech interviews are more competitive than ever. With every passing year, the bar seems to get higher. Plus, many companies are leaning even harder on Product Sense and Analytical Thinking to evaluate PMs. If you're not confident in these areas, you're at a serious disadvantage going into interviews. 📚 What you'll learn in this course---✅ How to approach Product Sense questions with clarity — from identifying the right user segment to selecting a high-leverage problem and designing a focused v1 solution✅ How to structure Analytical Thinking interviews — from defining the right success metrics to setting guardrails, tradeoffs, and team goals✅ How to compare your take-home work to bar-raising examples through live review sessions where we walk through model answers and invite Q&amp;A✅ How to self-assess using interview rubrics and sharpen your instincts by watching top-tier examples broken down in real time✅ How to learn from a Meta-style interview coach and alumni who’ve landed offers — without over-engineering or second-guessing every answer 🔧 Course Format---🧠 2 live case sessions (Product Sense + Analytical Thinking)📝 2 take-home projects to apply the frameworks on your own🔍 2 live homework review sessions where we walk through model answers and invite Q&amp;A💬 2 live office hours with a seasoned coach (ex-Meta, ex-Google)🙋 1 AMA session with a course alum who landed L6 at Meta🎥 All sessions are recorded and available for rewatching Time commitment: ~14 hours over 3 weeks. Designed to fit into a busy schedule while maximizing interview readiness. 🤖 Copilot Included---Enrollment includes access to the AI Copilot — a structured practice companion where you can:• Upload your answers to get rubric-aligned feedback• Simulate an interviewer or candidate to sharpen your skills• Get unstuck while working through the frameworks

Does this sound familiar?

↳ You're seeking generic interview prep - this course is specifically designed for Product Sense & Analytical Thinking interviews

Structured PS & AT Interviews

We'll have two 2-hour sessions (one in 1st week, one in 2nd week). There will also be three optional 1-hour office hours for Q&amp;A. Students will also have two take-home projects (one for Product Sense, one for Analytical), each expected to require ~1.5 hours to complete).

Interactive sessions + office hours + homework + AMA

PM Candidates preparing for Meta-style Product Sense and Analytical Thinking interviews (L5-L9)

📧 Email me anytime with 24-hour response guarantee💡 Schedule 1:1 huddles (15 minutes) when concepts need clarification🤝 Get invited to office hours in all future cohorts

Direct Access & Support

📋 Receive detailed templates optimized for interview success🎯 Learn evaluation criteria used by interviewers✨ Access pre-filled templates showing passing answers to PS and AT questions

Interview Frameworks & Templates

🎭 Get unlimited practice with your personal AI interview copilot🔄 Receive instant feedback based on expert frameworks💻 Step-by-step guide to set up your copilot in Claude Projects (ChatGPT coming soon)

AI-Powered Practice

🗣️ Participate in live case-style discussions with expert facilitation📝 Work through two Product Sense &amp; two Analytical Thinking questions🔍 Live homework review sessions walking through model answers

Hands-On Practice

🤝 Connect with fellow students who share the same foundation💪 Coordinate with peers for mock interviews📈 Share actionable feedback with each other using my evaluation rubric

Community & Collaboration

Master Product Sense and Analytical interviews with live case discussions, an AI practice copilot, and expert coaching from a former Meta PM

Ben Erez

Former Product Manager at Meta


July 2025

October 2025

🚨 Oct 6 is the last cohort we are teaching in 2025. Seats are limited. 🚨 All students in this course get:---- 🗄️ Lifetime acccess to all materials and recordings!- 🤖 10 months of unlimited access to our new AI Eval Assistant (more info towards bottom of this page).- 🧑‍🏫 9+ hours of office hours to maximize the value of live interaction. - 🏫 Lifetime Access to a Discord community with 1k+ students and instructors. This is a flipped classroom setting. All lectures are professionally edited and recorded, with an emphasis on live office hours and student interaction. Do you catch yourself asking any of the following questions while building AI applications? 1. How do I test applications when the outputs are stochastic and require subjective judgements? 2. If I change the prompt, how do I know I'm not breaking something else? 3. Where should I focus my engineering efforts? Do I need to test everything? 4. What if I have no data or customers, where do I start? 5. What metrics should I track? What tools should I use? Which models are best? 6. Can I automate testing and evaluation? If so, how do I trust it? If you aren't sure about the answers to these questions, this course is for you. This is a hands-on course for engineers and technical PMs. Ideal for those who are comfortable coding or vibe coding. ---WHAT TO EXPECT This course will provide you with hands-on experience. Get ready to sweat through exercises, code and data! We will meet two times a week for four weeks, with generous office hours (read below for course schedule). We will also hold office hours and host Discord community where you can communicate with us and each other. In return, you will be rewarded with skills that will set you apart from the competition by a wide margin. (see testimonials below). All sessions will be recorded and available to students asynchronously.--- COURSE CONTENT Lesson 1: Fundamentals &amp; Lifecycle LLM Application Evaluation- Why evaluation matters for LLM applications - business impact and risk mitigation- Challenges unique to evaluating LLM outputs - common failure modes and context-dependence- The lifecycle approach from development to production- Basic instrumentation and observability for tracking system behavior- Introduction to error analysis and methods for categorizing failures Lesson 2: Systematic Error Analysis- Bootstrap data through effective synthetic data generation- Annotation strategies and quantitative analysis of qualitative data- Translating error findings into actionable improvements- Avoiding common pitfalls in the analysis process- Practical exercise: Building and iterating on an error tracking system Lesson 3: Implementing Effective Evaluations- Defining metrics using code-based and LLM-judge approaches- Techniques for evaluating individual outputs and overall system performance- Organizing datasets with proper structure for inputs and reference data- Practical exercise: Building an automated evaluation pipeline Lesson 4: Collaborative Evaluation Practices- Designing efficient team-based evaluation workflows- Statistical methods for measuring inter-annotator agreement- Techniques for building consensus on evaluation criteria- Practical exercise: Collaborative alignment in breakout groups Lesson 5: Architecture-Specific Evaluation Strategies- Evaluating RAG systems for retrieval relevance and factual accuracy- Testing multi-step pipelines to identify error propagation- Assessing appropriate tool use and multi-turn conversation quality- Multi-modal evaluation for text, image, and audio interactions- Practical exercise: Creating targeted test suites for different architectures Lesson 6: Production Monitoring &amp; Continuous Evaluation- Implementing traces, spans, and session tracking for observability- Setting up automated evaluation gates in CI/CD pipelines- Methods for consistent comparison across experiments- Implementing safety and quality control guardrails- Practical exercise: Designing an effective monitoring dashboard Lesson 7: Efficient Continuous Human Review Systems- Strategic sampling approaches for maximizing review impact- Optimizing interface design for reviewer productivity- Practical exercise: Implementing a continuous feedback collection system Lesson 8: Cost Optimization- Quantifying value versus expenditure in LLM applications- Intelligent model routing based on query complexity- Practical exercise: Optimizing a real-world application for cost efficiency

Eliminate the guesswork of building AI applications with data-driven approaches.

If your events are recurring and at the same time, it might be easiest to use a single line item to communicate your course schedule to students

Monday & Wednesday & Friday

Feel free to type out dates as your title as a way to communicate information about specific live sessions or other events.

May 7, 2022

Schedule items can also be used to convey commitments outside of specific time slots (like weekly projects or daily office hours).

Engineers &amp; Technical PMs building AI products who are interested in moving beyond proof-of-concepts.

Those interested in moving beyond vibe-checks to data driven measurements you can trust, even when outputs are stochastic or subjective.

Founders and leaders who are unsure of the failure modes of their AI applications and where to allocate resources.

Who this course is for

We've tried all of them so you don't have to.

Acquire the best tools for finding, diagnosing, and prioritizing AI errors.

And how to best leverage data when you do have users.

Learn how to bootstrap with synthetic data for testing before you have users

Data flywheels ensure you have examples to draw from for prompts, tests, and fine-tuning.

Create a data flywheel for your applications that guarantees your AI will improve over time.

How can you really trust LLM-as-a-judge? How should you design them? We will show you how. We will also show you how to refine prompts, generate metadata, and other tasks with the assistance of AI.

Automate parts of your AI evaluation with approaches that allow you to actually trust and rely on them.

We will show you approaches to discover all the ways AI is not performing

Ensure your AI is aligned to your preferences, tastes and judgement.

There are an infinite number of things you can try, tests you can write, and data you can look at. We will show you a data-driven process that helps you prioritize the most important problems so you can avoid wasting time and money.

Avoid common mistakes we've seen across 35+ AI implementations.

We will provide end-to-end exercises, examples and code to make sure you come away with the skills you need. We will NOT just throw a bunch of slides at you!

Hands-On Exercises, Examples and Code

Generous office hours ensures students can ask questions about their specific issues and interests.

Personalized Instruction

Learn proven approaches for quickly improving AI applications.  Build AI that works better than the competition, regardless of the use-case.

Hamel Husain

ML Engineer with 20 years of experience.

Shreya Shankar

ML Systems Researcher Making AI Evaluation Work in Practice

Cohort 2

All lectures are recorded, 8x live office hours.

Cohort 3

Yes, we teach RAG, evals, agents, MCP, multi-agents, context engineering, and all that jazz. But always as tools to solve a business problem. If you want a checklist of hype items without knowing when or why to use them, please don’t join our course. :) 🚨🚨Notes:- For current offers/promotions check out the FAQs section below-We follow a flipped-classroom format. All lectures are pre-recorded so folks can go at their own pace, but we’ll still meet 5 times a week for office hours and live sessions. The course includes 35+ hours of live time. Check schedule below- For questions or bulk requests, reach out to: problemfirst.ai@gmail.com- October will be our final cohort. We don’t plan to run this course in 2026, it’s simply gotten too hectic. As much as we love teaching AI, we love building enterprise AI systems a bit more. So if you’ve been considering it, this is your last chance to join.- This course is an independent offering and is not affiliated with, endorsed by, or related to the instructors' current or past employers. ⛳&nbsp;The only prerequisite: you should have coded at least once in your life. The course includes low-code assignments, and even folks who hadn’t touched code in over 15 years have found it approachable and rewarding. That said, a basic understanding of coding really helps you get the most out of it — and of course, there’s AI to assist you along the way. The course is built for everyone, whether you’re a Product Manager, Architect, Director, C-suite leader, or someone seriously exploring agentic AI. Agentic AI or AI systems capable of operating with some degree of autonomy, is transforming how we interact with technology. In the coming years, most software systems will integrate AI agents to enhance their capabilities. This shift will drive a growing demand for professionals who can move beyond surface-level understanding and apply AI effectively to solve real business challenges while navigating practical constraints. This course focuses on practical AI agent development, covering key agentic design and usage paradigms. Instead of just explaining what these techniques are, we focus on when and how to use them, so you're equipped to make informed, business-driven AI decisions. What You'll Learn All core content is pre-recorded so students can focus on two-way interaction. Lectures are watched asynchronously, and we host four office hours each week for questions and brainstorming Week 1 (Let’s get you to understand what problem-first means)Decode why agentic AI breaks traditional software assumptionsFrame hallucinations, latency, and prompt brittleness through the determinism spectrumOpen vs. closed models: tradeoffs across compliance, latency, and costProblem-first, evaluation-driven design using early datasets and proxy metricsDeconstruct a production-grade use case and redesign it across progressive system versions Week 2 (Prompt engineering is still the core part of agents, do it smarter with right evals)Break down the evolution from zero-shot prompts to self-optimizing modelsMaster context engineering: Decomposition, meta-prompts, algorithmic optimizationAnalyze when to use prompting-only systems based on task, cost, and latencyCompare model-level strategies: reasoning vs. regular, and when each makes senseAdd guardrails and evaluation layers using LLM judges, semantic scoring, and offline tests Week 3 (RAG is not dead, it’s in fact the basis of self-improving agents)Address statelessness via dynamic retrieval and memory-backed context injectionBuild robust RAG pipelines with advanced chunking, embedding selection, and retrieval methodsExplore GraphRAG, Agentic RAG and multimodal RAG and other advanced methods and learn tradeoffsArchitect episodic, semantic, procedural, and working memory layers for self-reflective agent behavior Week 4 (MCP from an enterprise lens and multi-agents + Fine-Tuning)Understand planning autonomy in agents and how dynamic tool use and multi-turn reasoning go beyond static workflowsCompare agent levels and their control dimensions: action, planning, evolution, and physical autonomyExplore MCP (Model Context Protocol) and A2A as emerging agent-tool communication standardsInvestigate critical security challenges in MCP and A2A. Understand how guardrails, tool signing, audit trails improve reliabilityAnalyze coordination patterns in multi-agent systems, including shared memory governance, state sync, AI collusion risks, evaluation, logging, and observabilityExplore fine-tuning levers (SFT, RLHF, PEFT etc.), compare with RAG, and determine when to shift from context injection to model adaptation Week 5–6 (Put it all together in a capstone)Work in groups of 6Take a business problem and design/implement a solutionDemo to 4000+ public attendees including leaders, VCs, and hiring managers Homeworks: You'll supplement your learning by building an agentic search system (Perplexity like) in 3 iterations with the final iteration using agentic RAG, MCP and multi-agents. You can choose between low-code/code routes to complete assignments. ---- ❌ Who This Course Is Not For For Those Who Have Already Deployed Gen AI in Enterprise: This course is designed as an applied foundations course for enterprise AI with only basic Python as a prerequisite and no ML background required. If you’re already familiar with deploying AI systems, you won’t gain much from the core content. However, if you're looking to network and refine best practices, you're welcome to join. Those Seeking Heavy Theoretical Knowledge: This course emphasizes applied learning and practical problem-solving, not deep dives into theoretical topics like transformer architecture, pre/post-training optimization, inference techniques, or alignment. Those Who Have Never Coded Before: While we provide low-code options, this course assumes you have some coding experience. It’s not suitable for those who have never written or worked with even basic code. Individuals Expecting Deep AI Research Focus: While we’ll cover cutting-edge techniques, this course is centered on applying AI to business problems, not research-heavy exploration. Scaling and Ops Enthusiasts: This course does not focus heavily on scaling or operational aspects (i.e., LLMOps). Deployment will be covered at a high level, but not in-depth.

Design and build impactful agentic AI systems to solve business problems.

You should have coded at least once in your life. The course includes low-code assignments, and even folks who hadn

The only prerequisite

We offer generous office hours to support you throughout the course, along with 24-hour help on Slack. Typical weekly schedule:<ul><li>Wednesdays at 10 am PT and Saturdays at 9 am PT: Assignment Support</li><li>Thursdays at 5 pm PT and Saturdays at 8 am PT: Content/Use-case discussions</li></ul>

Live Office Hours

Our weekly community discussion channel covers the latest trends in AI, industry updates, and all the unfiltered tea, open, casual, and freeform.

Chai & AI Sessions

We bring in industry experts to share proven insights and discuss real enterprise challenges

Guest Lectures

Software/AI Engineers, Strategists, Data Professionals, Solution Architects and Consultants who want to master AI system design

Business Leaders and Product Managers seeking to gain the technical understanding needed to make informed decisions &amp; lead AI initiatives

Entrepreneurs looking to understand common generative AI use cases and learn how to develop and implement AI-powered solutions

While most courses stop at teaching tools and frameworks, this course goes further by focusing on solving real-world business problems. You’ll tackle practical constraints like cost, scalability, latency, and performance, learning to design AI solutions tailored to real use cases

Solving Real Enterprise Challenges, Not Just Concepts 

While learning applied AI concepts, we’ll put them into action by building a Perplexity-like AI-powered search system through detailed, hands-on tutorials that demonstrate their practical application (Low code options will be provided)

Apply Concepts to Build an Agentic Search System

Learn how to connect cutting-edge research with real-world applications. For the capstone, you’ll use our curated list of the latest research papers to design and implement solutions for practical business use cases. Some of our capstones have received VC funding too. <a href="https://areganti.notion.site/Capstone-Projects-Problem-First-AI-1e0c91fa32368011b935e00a35d9bceb" rel="noopener noreferrer" target="_blank">Examples</a>

Gain a deep understanding of key challenges in building AI systems, including handling hallucinations, adversarial attacks, security, privacy issues etc., and learn best practices to evaluate AI solutions comprehensively

Understand Challenges and Effective Evaluation
 

The course includes guest lectures from industry experts, AMA sessions, and our Chai &amp; AI discussions, culminating in a final in-person meetup in the Bay Area. You'll have plenty of opportunities to network and become part of our community.

Access to the Problem-First AI Community

Learn to make decisions tailored to business constraints, understand when & how to apply AI effectively & build a multi-agent application

Learn directly with top operators I admire

Curated by me

Curated by me

Grow with peers

Grow with peers

Take action

Take action

Learn directly with top operators I admire

Curated by me

Curated by me

Grow with peers

Grow with peers

Take action

Take action

My top AI PM courses

AI Product Management Certification

AI Product Management Bootcamp & Certification by AI Product Academy

AI Prototyping for Product Managers

My top leadership and career courses

Stuck at Senior Manager - How to Break Through to Executive

PM Interview Bootcamp with AI Copilot: Product Sense & Analytical Thinking

AI engineering courses curated by Claire Vo

AI Evals For Engineers & PMs

Building Agentic AI Applications with a Problem-First Approach

AI Red Teaming and AI Security Masterclass

🎁 $1,395 in free credits for Lenny’s favorite product tools