1/ The end of this year marks the 3rd decade I have spent working with data, financial modeling, actuarial science and forecasting.

My first formal project at the intersection of these fields as a CS senior. Using Neural Networks to forecast stock market prices. The year 1991.

2/ I was a bumble bee.

I didn't know then that given what we knew at that time, what I was trying to attempt couldn't really be done. So I spent the next 18 months chasing a dream.

I didn't know enough about the problem or the tools I was using to solve it.

3/ Years later when I became a trader myself I finally understood the futility of my exercise.

Tools have limitations. Model are an imperfect caricature of markets. Mob dynamics doesn't fit conventional distributions. Markets allow you to trade only on right side of the screen.

4/ I have seen a resurgence of the same dreams for last decade or so. Data science.

As tools improve, processing power becomes cheaper, data more accessible, data science star is rising

The same thesis I found to be invalid 3 decades ago. Fools gold and snake oil, most of it.

5/ To successfully model a trend or phenomenon you need exposure to underlying drivers. An apprenticeship of sorts. An example:

Health tech projects assume you will get access to confidential patient data. As an outsider you won't make it pass the ethics review committee.

6/ The same holds true for:

a) Building rule based systems for technical analysis (how many years have you run trading dollars?),
b) Making trading (buy or sell) recommendations, or
c) Even self driving cars (how many hours or miles of driving data are you working with?)

7/ Models don't generally work. Process and discipline does.

When we see a working model. We don't see the hundreds of iterations and tweaks it took to get to that point.

Rather than an expensive shiny tool, you first need drivers that control the process you are modeling.

8/ Ironic that someone who earns a living building and selling models would say what I have said.

I am not denying or dissing models. I am simply questioning the unbound optimism I see. It takes time to get models right. Order of magnitude more than what teams budget for.

9/ Before you commit to a path that leads to data science, for an idea for your startup or final year project for your graduate or undergraduate degree, please read the fine print.

@anzarkhaliq @HabibUniversity @jehan_ara @MaroofAliSyed @MahaRehman1 https://financetrainingcourse.com/education/2017/02/debunking-data-science-myths/