Category: Predictive Analytics

Have you ever given a thought how e-commerce websites show you products with “Customer who bought this also bought this” or how Netflix recommends movies based on your interest or how facebook discovers “Person you may know” list?

Let’s look at the below pictures:

Amazon: “When you buy a book”

Netflix: “Other Movies You Might Enjoy”

You can see in these pictures that different products are getting recommended on the basis of your behavior and content. How?

You might have heard the term “Recommendation Engine”.

Recommendation engines have changed the way websites interact with visitors. Rather than providing a static experience in which users search for and potentially buy products, recommender engines personalize user experience by recommendation of products or making suggestions on the basis of past purchases, search and other behavioral traits.

Recommendation engines are algorithms for filtering and sorting items and information. These use opinions of the user community to help individuals in that community to discover interesting and relevant content from a potentially overwhelming set of choices.

One can build recommendation engines using different techniques or ensemble of techniques. Some popular techniques are:

CONTENT BASED FILTERING

Content-based recommendation engine works with existing profiles of users . A profile has information about a user and their taste. Taste is based on user rating for different items. Generally, whenever a user creates his profile, Recommendation engine does a user survey to get initial information about the user in order to avoid new user problem. In the recommendation process, the engine compares the items that are already positively rated by the user with the items he didn’t rate and looks for similarities. Items similar to the positively rated ones will be recommended to the user. Here, based on user’s taste and behavior a content-based model can be built by recommending articles relevant to user’s taste. This model is efficient and personalized yet it lacks something. Let us understand this with an example. Assume there are four categories of news A) Politics B) Sports C) Entertainment D) Technology and there is a user A  who has read  articles related to Technology and Politics. Content-based recommendation engine will only recommend articles related to these categories and may never recommend anything in other categories as user never viewed those articles before. This problem can be solved using another variant of recommendation algorithm known as Collaborative Filtering.

Example of Content-Based Recommendation

 

COLLABORATIVE FILTERING

The idea of collaborative filtering is finding users in a community that share appreciations. If two users have same or almost same rated items in common, then they have similar taste. Such users build a group or a so called neighborhood. User gets recommendations for those items that user hasn’t rated before but were positively rated by users in his/her neighborhood.

Example of collaborative recommendation

 

Collaborative filtering has basically two approaches:

1. User Based Approach 
   In this approach, Items that are recommended to a user are based on evaluation of items by users of  same neighborhood, with whom he/she shares common preferences. If the article was positively rated by the community, it will be recommended to the user. In the user-based approach  articles which  already rated by user  play an important role in searching for a group that shares appreciations with him/her.
2. Item Based Approach
   Referring to the fact that the taste of users remains constant or change very slightly, similar articles build neighborhoods based on appreciations of users. Afterwards the system generates recommendations with articles in the neighborhood that a user might prefer.

Example of User-based CF & Item-based CF

Let’s try to understand above picture. Let’s say there are three users, A,B & C. In user-based CF, user A and C are similar because both of them like Strawberry and Watermelon. Now user A likes Grapes and Orange too. So user-based CF will recommend Grapes and Orange to user C.

In item-based CF, Grapes and Watermelon will form the similar items neighborhood which means irrespective of users, different items which are similar will form a neighborhood. So when user C likes Watermelon, the other item from the same neighborhood  i.e Grapes will be recommended by item-based CF.

HYBRID APPROACH

For better results, we can combine collaborative and content-based recommendation algorithms. Netflix is a good example of a hybrid recommendation engine. It makes recommendations by comparing the browsing and search habits of similar users (i.e. collaborative filtering) as well as by offering movies that have similar characteristics to a movie which a user has rated highly (content-based filtering).Using hybrid approaches we can avoid some limitations of pure recommender systems, like the cold-start problem. Hybrid approach can be implemented in different ways:

a. Separate implementation of algorithms and joining the results.
b. Utilize some rules of content-based filtering in collaborative approach.
c. Utilize some rules of collaborative filtering in content based approach.
d. Create a unified recommender system that brings together both approaches.

One such hybrid approach is Context-aware Approach. Context is the information about the environment of a user and the details of situation user is in. These details can play more significant role in recommendations than ratings or popularity of articles. Some recommendations can be more suited to a user in evening and may not  match user preference in  morning at all. User may  like to do one thing when it’s cold and completely different when it’s hot outside. Recommender engines that pay attention and utilize such information in generating  recommendations are called context-aware recommender systems.

Addressing Cold-Start Problem

It’s difficult to generate recommendations for new users when their profile is almost empty and taste and preferences are unknown. This is called the cold start problem. We can do following things to overcome this challenge:

1. Recommend subset of most popular articles from various categories to the user.
2. A better approach would be a hybrid one like context-aware approach, we can initially collect some data about the user’s environment, situation etc. (maybe by using cookies data). And then recommend the articles after having some information of the user.

CONCLUSION

We have thrown light on some popular techniques in building recommendation engines. There are some well- known challenges in building these systems. i.e. users can exploit recommendation system  to favor one product over another- based on positive feedback on a product and negative feedback on competitive products. A good recommender system must address these issues.
Recommender Engine  use different algorithms like Pearson correlation , Adaptive Resonance Theory (ART) family, Fuzzy C-means, and Expectation-Maximization (probabilistic clustering) etc.

I hope you like this post In next post I will cover recommendation engines in depth.

People often get stuck when they are asked to improve the performance of predictive models. What usually they do is try different algorithms and check their results. But often they end up not improving the model. Today I will walk you through what we can do to improve our models.

You can build a predictive model in many ways. There is no ‘must-follow’ rule. But, if you follow these ways (shared below), you’d surely achieve high accuracy in your models (given that the data provided is sufficient to make predictions).

1. Add more data: More data is always useful. It helps us to capture all the variance that the data has.
   I understand, we don’t get an option to add more data. For example, we do not get a choice to increase the size of training data in data science competitions. But while working on a company project, I suggest you to ask for more data, if possible. This will reduce your pain of working on limited data sets.

 

2. More Features: Adding new features decreases bias on the expense of variance of the model. New features might help algorithms to capture the effect of that feature. i.e. While predicting daily withdrawal from ATMs, People may follow different pattern in the start of month by drawing higher amounts from ATMs. So it’s better to create a new feature that is responsive to the start of the month.
3. Feature selection– This is also one of the most important aspects of predictive models. If we keep all the features in the data it might overfit the model and it will behave poorly on the unseen data. So it’s always advisable to choose important features in the model and built the model again only with important and significant features.

4. Missing value and Outlier Treatment: Outliers can deflect your model so badly that sometimes it becomes essential to treat these outliers. There might be some data which is wrong or illogical. i.e. Once I was working on airline industry data, in the data there were some passengers whose age is 100+ and some of them were 2000 years. So it is illogical to use this data. This is harder to explain but it is likely that some users intentionally entered their age incorrectly for privacy reasons. Another reason might be that they might have placed their birth year in the age column. Either way, these values would appear to be errors that will need to be addressed. In the same way, missing value issue should also be addressed.

 

5. Ensemble Models: Ensemble models can produce better results most of the times. Bagging (Bootstrap Aggregating) and Boosting are some of the ways which can be used. These methods are generally more complex and black box type approaches.

We can also ensemble several weak models and produce better results by taking the simple average or weighted average of all those models. The idea behind is that one model might be only capturing variance of the data and another model might be better at capturing the only trend. In these types of cases, ensemble method works great.

 

6. Using the suitable Machine learning algorithm: Choosing the right algorithm is a crucial step in building a better model. Once I was working with holtzwinter model for prediction but It performed badly for real-time forecasting so I had to move on neural network models. Some algorithms are just better suited to some data sets than others. Identifying the right type of models could be really tricky, though!
7. Auto- feature generation: There is a lot of buzz around the term “deep learning”. The quality of features is critical to the accuracy of the resulting machine learned algorithm; no machine learning method will work well with poorly chosen features. However, due to the size and complexity of programs, theoretically there are an infinite number of potential features to choose from. If you are doing image classification or hand writing classification then deep learning is for you. Deep learning does not require you to provide the best possible features, it learns by its own. Image processing tasks have seen amazing results using deep learning.

8. Miscellaneous: It is always better to explore the data efficiently. The data distribution might be suggesting for transformation. The data might be following the gaussian function or some other family of function, in that case, we can apply algorithm with a little transformation to have better predictions. Once we get the right data distribution, the algorithm can work efficiently. Another thing we can do is fine tuning of parameters of algorithms.