Introductions

Course: Product School

From assessing all of the Product Manager-related programs currently available, I believe Product School's instructors and program is the best out there. It's also the most legitimate credential from my perspective. After the first day of the week-long intensive, I'd have to agree.

The following are my notes from the course. If you're reading these and haven't done the course, I recommend doing it for the following reasons:

• The instructors are product leaders at the most respected technology companies. Hearing them share advice live is higher bandwidth for learning then just reading notes. You'll know what's more important to focus on from tonality, etc. Building a relationship with your teacher is valuable and can only be done through the course.
• The peer group is high quality. Ours has 28 people around the world that range from ML engineers, to senior PMs at FAANG companies. These are great people to get connected with so afterwards when you're a PM, you have peers to bounce questions off of, and share tips with.
• The live Q&A is valuable in filling in holes in your knowledge.
• There's supplementary content to learn even more about each unit.
• There are live breakout groups to get reps in performing common PM workflows.

Instructor: Oleg Pashinin

Oleg has spent 8 years at Google. Oleg working in ads, moved to cloud, then ended in research. He's writing about product management regularly on LinkedIn and open for advice.

As an aside, Google is renound for having the best product managers. They invented the associate product manager program to train PMs. So you can trust what Oleg teaches to be the best in the industry.

Table of contents

• Definition of a product manager
• Traits of successful and unsuccessful PMs
• Great content for product managers
• Doing a professional reflection
• On the product development lifecycle
• On user personas
• Product roadmaps
• Metrics (keeping score)
• On assessing the competition & market climate
• On public speaking
• Creating an opportunity hypothesis
• Qualitative validation methods
• Quantitative validation methods
• A/B testing
• Random notes

The definition of a product manager

A product manager sits in between engineers and users and does everything to make the product succeed. You have to fill in any gap, be it: product marketing, vision and guidance, etc. Simultaneously everyone and no one. You manage the team, present the product, and guide everyone towards what they should do.

• Some core components you'll focus on are strategy, goal setting, prioritization, execution. Focus on the user, and everything else will follow.
• A PM focuses on these two things: 1) What game are we playing? What advantages do we have over competitors, what is the value we're driving? 2. How do we keep score? Which metrics show us whether we're winning or not?
• No job is beneath product manager. If something goes wrong, it's your fault. If it goes right, shine the light on the members of the team that made it succeed.
• A PM is like a conductor in orchestra. You move everyone towards goal but you don't produce sound.
• A PM is the voice of the customer.
• As a PM, it's critical you like writing and are good at it. You need to write great ideas, inspiration, etc. Make sure that the team keeps track of these.
• It's important to be a great presenter. Watch Ted Talks. Don't make your product launches bland, even internally. Be like Napoleon and make them entertaining and inspiring.
• You're herding cats. Engineering typically wants to solve the hardest problem. Sales will try to close deals. Founder has a strong opinion around one specific things.
• Key product manager skills: technical background, industry domain, communication ability.

Traits of successful and unsuccessful PMs

Successful PMs

• Noticing products you love and hate, and why.
• Building rapport with engineers and the rest of the team and continuously adding value so they want you in conversations.
• Bias towards simple products, versus shipping a ton of features. These are the hardest to build.
• Respected by engineering team. Being able to code helps with this. Reading code, understanding system design tradeoffs is useful as well.
• Focus on matters most right now for team.
• Understanding your industry inside and out.

Unsuccessful PMs

• Biggest problem of a PM is not focusing on helping users and their engineering team.
• Not capturing ideas in writing.
• Giving in to top stakeholders and losing product focus.
• Getting obsessed with technical problems that don't solve user problems.
• Not understanding enough of the tech, which will result in engineering teams losing confidence in you as a leader
• Not collaborating early with designers/developers.

Great content for product managers

• Lean Startup. This shapes your thinking on how to succeed in product management.
• How great leaders inspire, by Simon Sinek. Shows that great leaders from Steve Jobs to MLK all communicate the same way, which is the opposite of everyone else. People don't buy what you do they buy why you do it. People want to do business with people who believe what they believe. Communication style: Why → How → What. This mirrors our biology which is Limbic brain → Neocortex. The limbic brain is responsible for feelings, trust, and loyalty. The Neocortex, thought and language. We make decisions from the inside out. It has to feel right.
• Blue Ocean Strategy
• Good PM / Bad PM by a16z
• AARRR Framework by Dave McClure
• The Hard Things About Hard Things by Ben Horowitz
• The Startup Owner's Manual by Steve Blank and Bob Dorf. Great for customer development.
• The Mom Test by Rob Fitzpatrick. Also great for customer development.
• Showspace. For building your product management portfolio.

Doing a professional reflection

Ask yourself these questions to gain more clarity into how yourself as a professional.

• What have I done until now (education, skill-gathering, experience, jobs) to prepare me for a PM role?
• What do I need to do (further training, skill-acquisition, time spent in employment) to get where I want to be in PM?
• Why do you want to be a PM?
• What does success look like?
• Why is the success important to you?
• Describe yourself as a superhero and what is your one superpower that's related to product management?
• Every feature you work on as a PM either improves customer satisfaction (make key use cases easier for customers), or influences growth (more use cases fo new personas to expand market).

On the product development lifecycle

Broadly, you'll come up with hypothesis on how to solve the problem, figure out how to build a business out of it, validate it, build it as scrappy as possible, delegate. There are two schools of thought on this process: Waterfall and Lean. In reality, the approach is hybrid. Start with a high level plan, then break it down into small initial steps, and iterate towards goal.

Product development lifecycle steps

1. Identify

Pick the right problem to solve. If you're a founder, don't try to solve someone else's problems that you don't have a relation to. Try to solve a problem that you have yourself.

2. Plan

• SMART goals: Specific, measurable, achievable, results oriented, timely
• Customer Need x Company Skills x Solid Execution = Product/Market Fit
• Do customer market research, talk with executive team, engineering team, marketing team, users. You should hear feedback from all directions and make sure there are no red flags.
• What is the game we're playing? What are success metrics?
• Plan MVP. What are the features to achieve a minimum lovable product?

Minimum viable product (MVP) process:

1. What value are you delivering?
2. List each feature and why it matters.
3. Cross off features that don't aid core value prop.

An even better MVP is a minimum lovable product.

3. Design

1. Write Product Requirements Document - written by you from user perspective. Could include UI/UX mockups. Write high level with vision to start to share context with team.
2. Write Technical Design Document - should be done by engineering team.

4. Develop

Build prototype as scrappily as possible to validate it.

5. Launch

• Figure out product marketing and positioning.
• As a startup, if you're not embarrassed by the first version of the product, you probably launched too late. For a larger company, you should launch with a very small close circle of users, then refine and expand.

6. Assess

Has the launch been successful? You shouldn't celebrate the launch itself very much. Celebrate when it's successful.

7. Repeat

User personas

User personas represent groups of customers. When capturing them, you should focus on how they behave, think, what they want to accomplish, why they want to accomplish it.

Only include details that are relevant to all users of the persona. For example, if age doesn't matter, don't include it.

It's fine to start with an educated guess of what your user personas look like, then iterate on them as you gather more info from your team and users.

It's important to distinguish who makes the buying decision and who is the user. Especially at larger companies.

Metrics (keeping score)

• Metrics can work for you and against you. If you only focus on metrics, that can lead you astray. If you don't look at them at all, that's also not good.
• You should start with defining which metric you want to move.
• Success metrics change as the product changes. AARRR metrics are a good model - Acquisition, Activation, Retention, Revenue, Referral
• NPS score is important in determining user love. How this works is you ask users the question "On a scale of 1-10, how likely is it that you would recommend [brand/product] to a friend or colleagues?" Formula: NPS = % promoters - % detractors. Score ratings: 1-6 = detractor, 7-8 = neutral, 9-10 = promotors.
• When picking metrics to track, chose actionable ones versus vanity.

Process of using metrics:

1. Establish a baseline
2. Determine course of action
3. Build a business case
4. Evaluate success
5. Move forward using data

Product roadmaps

• Usually these will be more detailed if they are sooner, and more vague further out.
• They can be used internally for team, as well as showing select customers that certain features are in development.
• This helps your team and other teams understand where you are.

Tools

• Specific: Aha!, Roadmunk, ProductPlan
• Presentation: Keynote, Excel
• Task Management: Pivotal Tracker, Asana

On assessing the competition & market climate

Content sources to follow trends are the a16z podcast and Stratechery.

Use the 5Cs to assess competition:

• Competitors - Direct and potential future ones
• Company - Products, culture, strenghts/weaknesses, etc.
• Climate - Regulation, technical environment, cultural trends
• Collaborator - Suppliers, distributors, partnerships
• Customer - Demographics, purchase behavior, market size, personas

Public speaking

Along with writing, public speaking is one of your other primary skills as a PM.

Here are some general tips:

• You'll be speaking from stages, to customers, recording training content for users, presenting in front of stakeholders and teams on a daily basis. To prepare, be confident in what you're saying, practice, and structure your speaking.
• Try to emulate the best public speakers, like Steve Jobs and Tony Robbins.
• Know your audience. Who are they? What is the value you're bringing them? Keep it simple, avoid jargon.
• With slides have one idea per slide for verbal presentations. Try to simplify slides as much as possible.
• Communicate confidently, humbly, clearly, and consisely.

Verbal speaking tips:

• Project and modulate your voice.
• Speak slowly and clearly.
• Pauses are fine.
• Never apologize or make excuses.
• Never read slides directly - too monotone and not engaging audience.
• Remember: you are talking with people, not at them.

Non-verbal speaking tips:

• On Zoom, put on gallery view and constantly be watching faces to see how engaged everyone is
• Eye contact, body language, mannerisms, hands, avoid insincere gestures
• React to the audience

On speaking structure:

• Always have an outline so you can make sure you hit your points.
• Start with conclusion, then work backwards from there.
• Be sure to have three topical messages at the beginning that you want to convey. That way if your audience gets distracted or fatigued, you know they caught your message.
• Good presentations follow a similar format to good emails: Main point at the top, then use a few more sentences to give background. you should use the same logic in your presentations. Start with why the audience should care.
• Personal stories are a great way to start presentations for engagement.

Q&A section tips:

• Repeat question out loud so audience can follow.
• If you don't know, say so.
• You don't always have to answer directly. Like if someone asks what revenue is but you can't share that, you can share something closely related like "revenue is doing very well, we're increasing user count by X% MoM"
• Take it offline if need be.
• Remember the core message you're communicating.

‍

Creating an opportunity hypothesis

How do you know that product is the right one? This is where you apply the scientific method to product.

The world is not short of ideas, it's short of quality execution. 95% of new products fail.

Roughly 50% of the time, your work as a PM will improve the metric it's supposed to. So you need to make sure you do your work to validate your hypothesis before spending the time to build the product.

Your job as a PM is figuring out the critical user journeys that your users are coming to your product for, and then improving them.

No matter the idea, you need to validate and test it if you want the company to be successful. Think of yourself like a scientist in a laboratory who meticulously validates ideas to see if they're correct.

Find Blue Ocean opportunities — areas where your competitors aren't yet. Then you're not limited by your local maxima. This involves risk though.

Setting and approaching goals

1. What noticeable change do you want to see in the world?
2. What type of product/feature will get us there? Is it an iterative change in an existing product? Or something entirely new. Come in at a humble place where you don't know if the solution is accurate. Gather data from team, stakeholders, and users, on whether the idea is correct.

Quantitative validation methods

These are less risky and more proven, but less likely to make a breakthrough.

The reason metrics are important is because people often tell you one thing that's not necessarily the case. Different departments can spin their success with a narrative, but including metrics shows more of the truth. (How could blockchains affect this since it's reliant on trust?)

Keep asking "why" a metric is what it is. This gets you to the root cause. Call the "5 whys" and originally invented by Toyota.

• Why is the clickthrough rate on our app low? Why is it so low on this browser versus others?
• Why is user adoption of feature X so low? Is it not being marketed? Are there bugs?

Leveraging data effectively

The sequence for leveraging data effectively is:

1. Collecting
2. Processing
3. Analyzing
4. Publishing

You'll be collaborating with data scientists, data engineers, and data analysts here. So it's important you're friends with these people!

Looking at data

Segmentation - Grouping users by a common trait

Cohort analysis - Adding time to segmentation

• Why is one cohort different from another?
• What changed in the world or in our product to cause the different?

Funnels - Looking at a sequence of actions

• What steps can be improved in your product?

Be sure you sanity check your analytics. Make sure you're tracking the right metrics, using consistent labels, they're properly reported, and the tool is properly collecting data.

Figuring out highest leverage opportunities

You can employ one of the following tactics:

1. Create a grid of scenarios and how each would affect metrics

2. Do a feature audit

Side note: if your product only has one core feature and there's another competitor doing it better than you, then you're in trouble.

3. Use the RICE method

Measure the following against each other:

1. Reach - How many users it would affect
2. Impact - How much a metric will move
3. Confidence - Confidence it will have the level of impact stated
4. Effort - How much human resource it will take

More details on RICE

4. Use the Kano model

Figure out how much time you would have to spend versus resources needing to be spent.

Definitions:

• Basic expectations - What customers expect, but if it's not there then they will be frustrated. Like toilet paper in a hotel room.
• Satisfiers - Things customers ask about.
• Delighters - Over-delivery. Features customers don't ask for but are big selling points that people love. Like Snapchat filters. Innovations.

Over time, as technology evolves, delighters become satisfiers, then satisfiers become basic expectations. Think about the phone touch screen? That never used to exist but now everyone expects it.

Qualitative validation methods

While Quantitative Validation can help you make incremental progress, qualitative validation metrics are where the breakthroughs are made.

Areas that you'll prioritize due to qualitative reasoning are:

• Known issues: bugs and feature requests.
• Tech debt: useful to pay off after big release. Also important if you haven't worked on it for awhile.
• Company vision: balance vision versus improvement.
• Intuition: A lot of time you'll just build something that feels right. Be sure to validate this.
• Team ideas: listen and make part of ideation
• R&D: productize inventions
• Competition: coping does not equal innovation

Methods of qualitative validation

Business Model Canvas: High level thinking towards business and customers

Value Proposition Canvas:

Internal validation

Doing a SWOT analysis is a popular way to validate hypotheses:

• Strengths - Strengths of your team and/or product.
• Weaknesses - Weaknesses of your team and/or product.
• Opportunities - Changes in the world that would be useful.
• Threats - Challenges the startup may face, like a pandemic.

It's important to be able to say "No" as a product manager so you're focused. Steve Jobs said that he's most proud of the things that Apple chose not to work on. This is especially a problem with Junior PMs. You need soft influence to do this.

Ways to say "no":

• If idea isn't aligned with values of org
• Figuring out that opportunity isn't there based on data
• Defer to management
• Say there are commitments that you made that you need to focus on
• Important thing is to have an argument for why the "no" exists. Goal is for no one to feel like they are personally blamed but it's the circumstances.

Customer development

It's recommended that you have a vision for your product before you do customer development.

Customer development is not:

• A focus group to get ideas/product vision
• A change for customers to give you a wish list
• User testing for designs

To do customer development, prepare for interviews.

Interview questions fall into the following categories.

• Specific vs. General questions
• Ideal vs. Real questions
• Avoid leading questions - Don't say: "Don't you think our interface is good?"
• Stories are fantastic - "What tools have you used over the last three months?" Try to get the customer to tell you a story. Then they'll implicitly tell you what they value based on what they emphasize.

As your last question, be sure to always ask: "Is there anything else about <this topic> that I should have asked about?"

This can be a golden question since users will mention things you never thought of.

Finding customers to speak with

While you can ask users to jump on a call and many will for free, you can also offer a $25-50 Amazon gift card. If the person is higher paid, offer a more expensive gift card.

Drawing conclusions

After 10 or so interviews, you'll notice the same trends. You're kind of being a psychologist here — understanding why people want what they want.

While interviews let you talk directly to a small number of customers, surveys are great for being at the quantitative level.

Survey tools: Google forms, Survey Monkey, Typeform, Ask your target market (aytm.com), targeted ads.

A/B testing

A/B testing is a method of validating whether the opportunity hypothesis is working. For this to be effective, you need a lot of data. Optimizely is a great tool to use here.

At larger companies, you'll start your A/B test with a very small number of users, then increase the number.

Example of a small change that has a big impact:

MVPs

Building the leanest MVP (or MLP) possible to validate the hypothesis is the goal. A few ways to do this:

Pre-order MVP - This helps you evaluate if people are interested enough in your idea to open their wallet to it.

Concierge MVP - You work with your customers like a concierge to accomplish whatever your opportunity hypothesis is around. So you're doing things manually instead of building a product.

Google Ads MVP - You A/B test ad copy with different product pitches on them. Then drive the traffic to a landing page. But the data you want to look at is the ad clickthroughs.

Wizard of Oz MVP - Appears automated to the user but it's actually note. You're manually doing everything.

Fake Door MVP - Add a UX trigger (like a button) in your product for whatever feature you're proposing. It won't do anything but throw up an error (or similar). Record how often people take the action. Then if enough people want it, build it.

Random notes

• Project managers are continuously in more demand. Especially since the pandemic.
• The best course to understand technology behind computers at the highest level is Harvard's free CS50 course.
• B2B products are purchased more through rational drivers. B2C products are purchased more through emotional drivers. This has slightly changed with companies like Slack and Intercom though. So there's more emotion entering in B2B.
• If you want to become a leader in the organization, it's especially important to build credibility with the existing leadership team. Every time you interact with them you should make sure everything you're doing gives them more confidence in you.
• Iterations can be big. This could be trashing everything you've done if it doesn't work. This is why startups succeed over larger companies. Startups don't have as much at stake with their public image and profile, so it's easier for startups to quickly iterate and ship more simple MVPs.

A list of small things to gain respect and rapport with teams, and specifically engineering teams:

• Make friends with them. If there is a casual after-work meetup, be sure to attend. If there isn't, consider organizing one.
• Try to understand topics that engineers are talking about so you can have educated conversations with them.
• Bring some structure to the product development process. If you can do things like create wiki, organize documents, etc, that's useful. Most engineers would like to have more structure, but they don't have the time.
• Try to understand the decisions engineers have made in the codebase, with architecture, etc.
• Be humble in your communication style, dare to ask questions when you do not understand something, and show that you want to understand their reasoning and take their points into consideration. The worst thing to do is to pretend you understand something that you don't. You will lose respect fast if you do this. Especially with engineers.
• Don't commit to dates or deadlines without consulting other teams first. That's a good way to destroy a relationship between teams.
• Listening to team ideas and rational behind them.