Predictive Modeling Practices

Industrial model development is quite different than theoretical / course work model building. A number of times, industrial solution often requires

1. A model design, which can replicate the business situation without ambiguity and ensuring that it is not using effect variable (which is bound to give very good model on development data but not been able to perform after deployment)
2. A careful selection of variables, which is having trend of response, which is consistent with business sense
3. Combination of variables, which is having good separation power of responder vs non responder
4. Ensuring that there is no multi collinearity among variables
5. Calculate model’s efficiency in terms of KS, GINI, rank ordering etc.
6. Check it’s performance in out of time data to have good confidence on model performance over time
7. Track model performance over time to detect, if there is any anomaly in the model performance and decide either one needs to fine tune model or redevelop model

Unsupervised machine learning

1. Unsupervised machine algorithm tends to develop clusters based on all the variables that has been given
2. At times, one needs to gather variables which should be considered for unsupervised learning
3. If business is not so clear – then data science folks has to apply methods of variable selection, which is applicable in case of unsupervised situation (please note supervised machine learning based variable selection is of no use here)
4. Then one needs to develop cluster solution and recommend the number of clusters that should be used
5. Also one needs to visualize the cluster centres and make sense of the same

 Time series analysis

1. Time series analysis is all about using time series data to predict future
2. At times, there are other input variable for future like sales plan, festival details etc. which can be used as input and which can supplement time series data for better prediction
3. Time series analysis based prediction requires one to discover the model details and then use the same for prediction and
4. Then visualize the prediction along with actual

Association rules mining

1. Association rules mining is all about find what sells together based on support and confidence
2. And then visualize them

Collaborative filtering

1. This is a very useful method to predict, what will be liked by end user. This is used for movie recommendations by OTT platform.
2. This method is all about using the data to find, who are the similar users.
3. And then finding items which has not been used (or movie not seen) by end user but have been liked by similar user

Text Mining

1. Text mining situation is all about finding words which are most frequent
2. When there is class or sentiment available along with comment, this is called sentiment analysis / classification analysis
3. In case, where there is no class or sentiment available, this is called unsupervised text mining.
4. Text mining requires certain data cleaning steps like case conversion (all words in lower case / proper case kind of ) stemming, punctuation removal etc.
5. In case of sentiment analysis – one also needs to find out odds associated with the terms for different classes
6. At times, user needs to see actual comment along with modified comment to see why a particular word has come for a particular class

we have many decades of experience in delivering propensity models, time series analysis, cluster analysis solutions etc. on top of that we also know how to model data and store data in such a way, which can be quickly combined across months to get final data for model building. we also standardize data treatment process so that every modeler (new or old) can get data for model building, easily, effortlessly and with accurate data treatment. we also apply years of experience while designing data combinations to derive many meaningful variables – like times defaulted in last six month, maximum delinquency in last six months