Harry's Log


How to Evaluate Startup Ideas

OK, so, here’s the question: “How to evaluate a product idea for a startup?”

There are very good resources on the Web, some specifically written for this particular topic, and this note is a mere summary/interpretation of these articles (in my own words). All references which I consulted will be included at the bottom of this note, but I think I’m mostly influenced by Evan Williams’ (very well written) article from 2007 and the article/video by Tony Wright. I highly recommend that you check out all the references included here as well as others available on the Web.

So, how do you evaluate product ideas for your startup? There can be multiple factors, and they are all subjective considerations to a large degree. Some might be more important to you than others depending on your particular situation, and more importantly, depending on your personal value system. It should be noted that the following factors are not mutually exclusive. In some cases, one consideration might even be conflicting with another.

[1] Passion:

How passionate are you about the product you are creating? Do you really want it to exist in the world? Are you compelled to do this for the next several years, if not the rest of your life? Do you think it is your mission to create this product? Do you think you should be the one working on this product? This is a very personal question. The idea does not have to be about world peace or curing cancers, but something you are really passionate about.

[2] Tractability:

How easy will it be for you to create a version 1.0 product? How difficult will it be to launch a worthwhile version 1.0? Often we use the term MVP (coined by Eric Ries). This criterion is about the threshold to reach that “minimum viability” in terms of the required resources and the time it takes to reach that point, etc. Often people tend to emphasize the “minimum” part of MVP, but the more crucial question is, how hard will it be to reach the “viability”. This is an important factor to consider if you are a small startup.

[3] Novelty:

Is the idea new? Is it an invention or re-invention? Is there a competition? Is there an exiting market? This is a complex issue. In general, creating a new market is virtually impossible for a startup with limited marketing resources. Therefore, completely new ideas are discouraged. On the other hand, new ideas face generally less competition. Ideally, the products should belong to an established category with an exiting market, but have a clear differentiation (or, other competitive advantages). Note that a new idea is generally more tractable (e.g., due to less competition, etc.).

[4] Obviousness:

Is the idea obvious? Is it obvious why people should use your product or service? Is it clear how to use it? Is it easy to explain? Does it have an obvious business model? The same caveat as in novelty applies to obviousness as well. Non obvious ideas will face less competition. On the other hand, non obvious ideas will generally be much harder to implement and execute, and communicate. Often, the best ideas are the obvious ideas which nobody have thought about.

[5] Value:

How valuable would your product be to your target customer? How deeply could you imagine impacting the life of your customer? How much value can you ultimately deliver to your customer? A “value” is a subjective measure. It could mean anything, including monetary reward. Can your product help your customer make or save money? Different products deliver different types of values. E.g., is it a vitamin or a pain killer? Also, your products may cater to more than one category of customers. E.g., an e-commerce platform serving both merchants and shoppers.

[6] Market:

This is the second part of the value proposition, in terms of reach. How many people’s lives can your product affect in the long run? How big is, or could be, your market? How many people may ultimately use your product? How will people learn about your product? Big market means more potential revenue but more competition. However, small market is not necessarily better for a small startup. “Marketability”, “discoverability”, or “virality” of your product, etc. are also important part of market analysis. Niches with clearly defined, and inexpensive, marketing channels are ideal for startups. (Note: You can divide this item into two: One for the market size and the other for the accessibility of the market or the distribution channels, etc.)

[7] Cause:

Is your product (or could it be) worth talking about? This question is closely related to the value (depth) and market (breadth) criteria, but not always. If your product has clear benefits to consumers, then people often recommend it to other people. More so, if your product has mass appeal (e.g., not a niche product). In addition, some ideas are simply worth talking about, which I call it having a “cause”. Does your product support a cause? For example, “green” is a good cause, and “helping others” is a good cause (e.g., Charity Water). Also, you will be more passionate about ideas which people find worth talking about.

[8] Timing:

Many products fail or succeed mainly due to its timing. “Timing” is related to market opportunities as well as technological risks. Is your product too early? Is the market going to be ready when the product is released? Is your product too late? Are there any big incumbents in the market? Is the industry consolidated or going through consolidation? Can your idea be implemented using the current, or near future, technologies? Is the infrastructure available to deliver your products/services? Is this a good time to release your product?

[9] Expertise:

Your domain knowledge can be a make-or-break factor in many cases. The more you are familiar with your product industry, the more likely you will succeed with that product idea. The less time you will end up spending learning/wasting, and you will have clear advantage over other competitors who have less domain knowledge than you do. Often, you may have to ask yourself, “Am I the right person for this idea?” An idea might be a good one, just not for you.

[10] Defensibility:

Are you in a defensible position against any current competitions with respect to your ideas? Or, against any future competitions, or copycats and knockoff artists? It could be your expertise in the domain (technical or non-technical). It could be your technology. It could be IP such as patents on your ideas or trademarks. Consumer web products are not very defensible, in general, unless you are a first mover in the field and your product is “sticky” or it has inherent network effect, etc. Otherwise, you’ll need large capital.

[11] Monetizability:

How hard will it be to extract money from your product or service? Is there a good ratio of revenue per customer (LTV) and cost of customer acquisition (CAC)? Does your product create recurring revenue? Not all ideas are, or don’t have to be, for profit, and so I am listing this near the end. But, your ideas are merely ideas until you have clear business model. Especially, for small bootstrapping startups, this is one of the more important criteria when you are evaluating ideas. You have to be concerned with not just how to make money, but how to make money as early as possible. Ideas with clear business models are better ones. 

[12] Fundability:

I am not sure if “fundability” is a word. :) I just wanted to use a single word for each criterion (for some odd reason). Is your idea fundable? What’s the capital requirement for your business? If you are planning to build a product, which takes a long time to monetize or to reach a profitability, or which requires big capital, you may need to seek outside funding at some point (angel or venture capital). For such ideas, the fundability is an important criterion. There are a lot of good resources on the Web regarding what kind of products/businesses VCs typically fund.

Add more, if you wish…

I’ve created a table with 12 criterion columns, and each idea on a row. This is a “live” document. In the next few days or weeks, until you make a commitment on one idea (or, more), you will constantly evaluate these ideas. Over time, you will learn more and your ideas will change. This table will change as well.

How to use the evaluation table:

Come up with a simple scale. A three value system, with 1 being positive, 0 neutral, and -1 negative, might be good enough in most cases. Or, you can use 1 through 5 scale with 3 being neutral, etc. You don’t probably want to use a system with too fine scales since these are really arbitrary and subjective measures and you cannot be “precise”.

Evaluate each of your ideas for each criterion. Just to be clear, the 12 criteria above do not have one dimensional up and down scales. For example, being novel might be a positive characteristic for some ideas, but it might be negative for some other ideas. You are evaluating your “advantage”, not as to whether an idea is novel or not (although being novel, or having a clear differentiation, is generally a positive trait). This general principle applies to all other criteria as well.

Once you rate all ideas across all factors, then sum their values or ratings over all factors. You can add an extra column at the end with these rating sums.

Do you think, or agree, or are you convinced, that the ideas with higher ratings are better ideas for you? OK, now it’s time for you to go ahead with your best idea(s) and start implementing.

(Note: You can give a different weight to each evaluation criterion. For instance, you might consider “passion” more important than “monetizability”, etc. You can use 0~1 weights and do weighted average across different factors. Or, you can try using only a few select criteria which you think are important to you and ignore the rest. For instance, I ignore “expertise” (domain knowledge) since most of my ideas are really consumer products (Web or mobile), which I am familiar with, and not verticals. Also, “fundability” is not a factor for me since I’m bootstrapping my company, and I use “capital requirement” instead. Ideas which require less capital are better ideas for me.)



You can find here my exemplary Idea Evaluation Matrix.