how we research: a framework for making decisions that dont suck
most "research" is just confirmation bias with better formatting. you have a hunch, you find three sources that agree with you, and you call it due diligence.
thats not research. thats a cosplay of rigor.
the problem with how most people research
the default approach goes like this:
- have an opinion
- find evidence that supports it
- ignore or explain away evidence that contradicts it
- call it "research" and move forward with confidence
this feels productive but its actually dangerous. it confirms what you already believe while giving you a false sense of certainty.
a better framework
step 1: state your assumptions clearly
before you look at any data, write down:
- what you believe to be true
- why you believe it
- what would need to happen for you to change your mind
if you cant articulate the last one, you're not researching. youre defending a position.
step 2: actively seek disconfirmation
this is the hardest step because it feels unnatural. we're wired to seek agreement. but the most valuable research comes from finding what youre wrong about.
- find the smartest person who disagrees with you
- read arguments against your position before arguments for it
- ask "what would make this a bad idea?" before asking "why is this a good idea?"
step 3: audit your sources
not all evidence is equal. a quick hierarchy:
| type | weight | why |
|---|---|---|
| your own data | high | its from your actual context |
| peer reviewed | medium | rigorous but may not apply to you |
| industry reports | medium | useful for trends, biased by agenda |
| anecdotes | low | useful for hypotheses, not conclusions |
| twitter threads | very low | please stop |
step 4: decide on a threshold
dont research until you're 100% certain. that day never comes. instead, decide upfront:
- what does "enough to act" look like?
- what's the cost of being wrong?
- what's the cost of waiting for more data?
once you have enough to act with acceptable downside risk, stop researching and start doing.
a practical example
lets say you're deciding between two pricing models for your saas.
bad research: asking your twitter followers what they'd pay (they'll say free), then picking the lower price because "thats what people want."
good research:
- state assumption: "i believe a $49/month subscription will convert better than a $499 one-time fee"
- seek disconfirmation: interview 10 people who charge premium prices. read about perceived value and price anchoring.
- audit: your own trial data (high), pricing studies from similar markets (medium), founder anecdotes (low)
- threshold: you need enough signal to run a 2-week a/b test. not enough to get it right forever.
the real lesson
good research doesnt give you certainty. it gives you direction. it reduces your surface area of ignorance from "massive" to "manageable."
and then you act.
because action beats perfect information every time.
~ shivam