It is common knowledge in machine learning that models deteriorate over time. First off, how do you check to see if your model is deteriorating?

A common rule of thumb:
90% < X < 100% Too Good
80% < X < 90% Very Good
70% < X < 80% Good
60% < X < 70% Poor
X < 60% Rubbish

X being the assessment metric for your model.

So, why do models deteriorate?

If your answer was a myriad of reasons, you're absolutely right. Some of them are:

1. Additional Factors

Your model includes some variables, and then another variable appears that wasn't included in your model.

2. Changes in behaviour

Mostly when you're modelling the behaviour of human beings. For example in the banking sector, since the advent of smartphones people started banking with their phones and they don't like the colour of your bank's app or it has no app

3. Changes in process

Maybe you have excluded an additional step in the process of manufacturing which was not taken into account in your model previously and now it's not performing as well.

4. Competitor

Maybe a competitor(s) has entered the market and they have been changing things or taking a share of your customers or more customers have been leaving disproportionately more to what the model is expecting.

5. Changes in regulation

A change in regulation mostly affects the way you do things and thus restructuring your model. For example in a model to predict how likely people are to give you their phone number, the regulation previously required you to send

them direct mail physically but with the new regulation you can send them electronic direct mail and they can reply by email and give you their phone numbers. Your model won't be performing as well since it was not used to this drastic change.

Feel free to add other reasons for model deterioration and how to address them @svpino @haltakov
#MachineLearning