Hypothesis — An assumption or guess. In the context of product development, it is a reasonable assumption about the increase in product performance that is put forward by the team for evaluation and verification through experimentation.
Thus, the hypothesis generates the task of conducting the experiment, but is the task itself.
‣
| Hypothesis | Task |
|---|---|
Not intended to confirm or deny | |
It has only the goal of realization |
Requirements for hypotheses
A working hypothesis is a hypothesis that the team is ready to use. For a hypothesis to become a working hypothesis, it must:
- Be evaluated by the team as real,
- Provide significant growth or new valuable knowledge,
- Meet the requirements of the production
- In case of failure, it should suggest the direction of developing a stronger hypothesis
Staging requirements
- The hypothesis must have a formal author and "owner"
- The author must formulate the hypothesis before proposing it.
- The hypothesis must be fundamentally verifiable.
- The hypothesis must refute the "null hypothesis" - the default assumption that there is no relationship between two observed events.
- The hypothesis should lead to a closer approach to the goal, not a further one.
- A hypothesis must contain a condition and a consequence. The consequence of the hypothesis should be related to the metric and the amount of its change ("will increase conversion by 10%"). This will allow you to run the hypothesis through the economic model and correlate it to the current strategy.
- The hypothesis should contain a premise or other rationale for success* (e.g., "analytics show that people on large monitors scroll better than people on small monitors, so they probably don't see the content below").
- It should not contradict the previously established facts it is intended to explain.
- Should be applicable to the widest possible class of phenomena. For example,* "the poor contrast of the phone significantly impaired calls, so it is likely that poor contrast in other places could also cause problems. "*
Examples of hypothesis generation
‣
‣
‣
Ways to test hypotheses
An experiment is a practical test of a hypothesis in an accessible way. The value of the experiment is determined by the value of the knowledge gained, so the maximum amount of knowledge should be obtained from the experiment.
Prioritised ways to test hypotheses from the most economical to the most expensive:
- ROI modeling
- Data analysis
- Talk with users (User Research. Customer Development)
- MVP
- Usability research
- A/B test
Levels of testing and verification
- Specific implementation of the feature (UI version)
- The relationship between the mechanism and key metrics (push → retention)
- The direction of product development (for example, a test of the impact of speed on retention)
Unconfirmed hypotheses.
Any hypothesis should carry knowledge. Even if the hypothesis is not confirmed, it is also new knowledge, for example: "segment X is not sensitive to price changes within 20%" or "all segments can handle the card entry interface".
Usable information.
- Useful techniques for working with hypotheses
- To get tangible results, you should increase the number of working hypotheses to 5 per week. According to various estimates, even in strong teams, only 20% of hypotheses are successful. Given that there are about 40 productive work weeks in a year, this means that only 40 hypotheses per year will be successful. If 1 successful hypothesis generates a 5% increase in revenue, then you can expect a 200% increase in revenue per year.
- How to distinguish a hypothesis from a task? If an idea implies failure and a rollback to the previous state, it is a hypothesis. If it's "done and forgotten," it's a task.
- Occam's Razor:** If a phenomenon can be explained in two ways: for example, the first way is by using entities (terms, factors, facts, etc.) A, B, and C, or the second way is by using entities A, B, C, and D, and both ways give the same result, the first explanation should be preferred. Entity D is redundant in this example, and its involvement is excessive.
- Hanlon's Razor:** the presumption that human error should be assumed first and foremost when looking for causes of unpleasant events, and only secondarily for deliberate malicious actions. It is usually expressed by the phrase: "Never attribute to malice what can be explained by stupidity."
- Hitchens' Razor:** The burden of proof for a statement is on the person who claims it to be true. If the claimant fails to provide evidence, the statement is considered unfounded and opponents should not argue further.
- What contributes to the implementation of growth methods in the company
- Google OKR and goal synchronization - otherwise, there is a conflict of interest
- Flat organizational structure - so as not to break through the hierarchy
- Ambitious goals - otherwise there is no motive to change the approach
- Steady growth stages - suddenly they are already growing faster
- Agile methodology - to quickly change the planned
