Let's talk a bit about anecdotal evidence. People ALWAYS think that they are being "clever" when they poo-poo anecdotal evidence... But that is always a good thing to do.

We'll start with a definition...

Anecdotal evidence is evidence from anecdotes: evidence collected in a casual or informal manner and relying heavily or entirely on personal testimony. Anecdotal evidence becomes unscientific is through fallacious reasoning such as the post hoc ergo propter hoc fallacy.

However... Do you realize that a SIGNIFICANT portion of social-science data is "technically" anecdotal? Much of social-science data is collected by SELF-REPORT. The researcher relies on the subject relating, reporting giving data/details.

The researcher cannot crack open a person's head to view their thinking. A blood test cannot identify or measure attitudes. Yes, we can and do construct objective self-report measures. If we are lucky we can collect observational data... But we rely heavily on self-reports.

Furthermore anecdotal evidence is useful when used in addition to objectively collected data. Anecdotal evidence can be a powerful tool to derive unique insights from your data if used properly.

An anecdote is a data point that provides additional information. With a disciplined approach, that info won’t mislead you, it can generate a better understanding of what you’re researching. Let anecdotal evidence drive your questions, and data analytics support the answers.

A good researcher will not ignore anecdotes, she will utilize them skillfully.

How to approach anecdotes:

Keep emotions aside.

Listen for variables you have not yet considered.

Develop novel, testable hypotheses from your new variables.

Systematically and objectively test those hypotheses.

Maintain or change your strategy based on the results.