Principles of rigorous backtesting. I'm at the point where I think backtesting has no positive edge - but failing to do it well can lose you a lot of money. See a lot of examples of absolute amateur hour backtesting on this site. A thread:

1/ Survivorship bias. How did signal perform on Lehman Brother's? DoubleClick? Delisted Natural Gas ETPs? What are assumptions around bankruptcy recovery or hold time post M&A announcement.

2/ Liquidity. What is rolling 21 day median volume of asset traded? What are assumptions of tcosts at different % of that volume in capital amounts? What are tcosts when providing liquidity vs taking liquidity? When taking liquidity simulate on historical order book

3/ Linear Causality. Does a stronger signal create a stronger PNL expectancy? Does bucket 1 of signal outperform bucket 5 consistently statistically over time. Good example of non linear causality: Google trends for Sea World during controversy.

4/ Temporal Causality. Does a more recent signal create a stronger PNL expectancy? Does signal t+1 day outperform t+5 days consistently?

5/ Geographic Universality. Is there a reason the Signal should not work on Japanese equities? Does it, in fact work on Japanese equities? Are there traits of a market (i.e. options penetration as % of total) that should drive PNL? Do they, in fact, drive PNL?

6/ Market Neutrality. Is the signal constructed in such a way that is indifferent to the underlying drift of the assets traded? I.e. if you're betting inflation makes bonds go down, do you also test if deflation makes bonds go up (you should)

7/ Preference for Stationary Expression. If you have an oil indicator related to the specific carry indicators of WTI -- can you trade WTI against CAD-USD with which the signal is correlated profitably? Can you make versions of signal which are asset specific

8/ Derivative Preference. All things being equal, a signal that works well should have a derivative that also works well that is causally related. Like Free Cash Flow Yield? You should like it more if expanding free cash flow yield or free cash flow margin expansion works

9/ Preference for Auto-correlation. Can the strategy be shut down historically, and turned back on without compromising its profitability? Can the rules be set on an ongoing basis without fitting them?

10/ Lack of Emergent Beta. Does the strategy have beta to a list of your other quant strategies, that is expanding inexplicably or becoming consistent? If so, that is bad, as it likely means the strategy is crowded. Can you measure the "capacity" of your strategy? How?

11/ Mapping & Quantifying Causal Drivers. So you think LQD and HYG drive the S&P 500 because Printer Go Brr? What about in markets where QE is less important (Russia) - do IG and HY predict equity premiums less effectively? Does the strategy have mappable drivers? (good if yes)

12/ Statistically Significant Out of Sample PNL. Did you write the strategy down in advance, parameter specified with a rigorous backtest and a causal thesis? Did you commit to to Github, and run it, out of sample without Tweaks? Did it work? Great. I bet you won't tweet it :)