https://virgool.io/feed/@m.khosravi CEO - CPO fa 2026-06-10 13:03:48 https://files.virgool.io/upload/users/38636/avatar/Lq4Rmi.png?height=120&width=120 https://virgool.io/@m.khosravi https://virgool.io/@m.khosravi/stop-using-net-promoter-score-nps-aqimzulzwesl Introduction:I am recently tasked with calculating the Net Promoter Score (NPS) in the largest e-commerce company in my country, aiming to assess its brand popularity and user loyalty. After calculating the NPS and analyzing the results, I came to realize that not only NPS is not reliable, it is also based on false assumptions, and has fundamental flaws.A good measure of customer loyalty should be valid, reliable, and sensitive to changes in user attitudes; However, NPS has none of these qualities.Many websites are currently advertising NPS, and its use is on the rise in many countries. Unfortunately, some companies judge their products and employees based on NPS results, which is a huge mistake.It is dangerous to use this system in companies (whether by managers or product managers), since the results do not provide a correct representation of the company’s condition, and also cannot accurately show the company’s progress and its position in the competition.In this article, I intend to show you the dark side of NPS, so that you can have a more clear view of it.Of course, I am not going to propose an alternative, however, averaging out the scores can help resolve some of the problems outlined below. Moreover, note that after selecting any types of surveys:“Without a plan in place to act on the results, the survey won’t help your business”What is NPS?First, here is a summary of NPS, for those who may be unfamiliar with it:In this survey, the participants are asked to answer the following question.How likely is it that you would recommend our company/product/service to a friend or colleague?(After this question, of course, other questions can be asked. However, NPS is calculated based on responses to this single question.)The Score, on a scale of 0 to 10, falls into three groups:If a customer scores you as a 9 or a 10, they are promoters.If they score you as a 7 or an 8, they are passive.And, a score of 0–6 means you have a detractor.Subtracting the percentage of Detractors from the percentage of Promoters yields the Net Promoter Score, which can range from a low of -100 to a high of 100.Why is NPS unreliable?Note: some of the examples may not happen in reality the way they are provided below, but the goal is to show the inherent shortcomings of NPS, and these shortcomings will (more or less) influence the NPS results in all companies.1- Prejudice about the Users:One of the assumptions underlying NPS is that it regards individuals with responses of 9 or 10 as “Promoter”, those with responses of 7 or 8 as “Passive”, and those with responses of 0 to 6 as “Detractor”. Individuals are in fact subjected to prejudice based on their response.This assumption is not reliable, since in some countries you’ll get a 10 for being “good” (and not excellent), whereas in others “good” means 7!In some countries where the users are generally rational buyers, and have a higher quality of life and subjective standards, a score of 8 means excellent and shows complete satisfaction of the customers, who can be regarded as promoters. They would rarely give a score of 10 to anything, and the score is reserved for products that they believe are ideal.A score of 6 means that the user’s satisfaction with the product or brand is above average, and the user should not be regarded as a detractor.This principle does not apply to some other countries where people would normally give a score of 10 to a good (and not necessarily excellent) product. In these countries, 8 does not mean “good” or “excellent”, and they can be regarded as passive customers.Therefore, the same number can be interpreted differently for two countries (or even two different cities of the same country).This idea can be applied to all numbers, and 6 and 8 were used as examples. Thus, the aforementioned hypothesis is flawed.If the provided service is international, the results obtained from, let’s say, Japan and Germany may lower your NPS, without the service has gotten worse.Also, some customers may don’t respond for some reason. That means you’ll lack data from some of your customers.In some countries, unsatisfied users may be reluctant to respond to the survey, whereas the opposite may be true in other countries, which can influence the NPS results. Even passive users (included in the calculation as the denominator of both promoter and detractor groups in the fraction) who don’t respond can influence the results!2- Invariability of User Interest and Loyalty in All Three (Promoter, Passive, and Detractor) Groups:The second hypothesis underlying NPS is that the feeling and extent of loyalty is constant in all three groups.This is also a false assumption, since, in the Detractor group for instance, the feeling of a user who gives a score of 0 is different from one with a response of 6. In the calculation of NPS, however, 0 and 6 are worth the same.Zero can be regarded as complete dissatisfaction and even detestation. Whereas, the score of 6, which is above the mathematical midpoint, can show the average feeling of the user with a little optimism. Loyalty and feeling of these two users cannot be the same, but for NPS they are!3- Non-factuality of the Results:One of the fundamental shortcomings of NPS is that It calculates how many users are “likely to refer your brand”, not how many users “actually referred your brand”.In other words, respondents giving a 9 do not necessarily promote the brand/service in real life. People may respond based on their feeling at the time of the survey, which might not match their action at a later time.4- Incompleteness of the Question:Customer satisfaction and loyalty are not one-dimensional, and appear differently in different places.asking just a single question is one-dimensional, so it gives only a narrow perspective of customer satisfactionSince the question is not comprehensive, the NPS results may be a different form of real life. It was also seen that NPS responses were sensitive to the survey location. That is, the appearance of the survey on certain pages of the website can significantly influence the results.5- Influence of the Design:Since it is a survey of feelings, results are highly dependent on the design. We altered the NPS design three times. In the final design where icons were used (and the users understood that 7 and 8 are not regarded as high scores), the NPS results were increased by 10 units!6- Inaccurate Overview of Customer Loyalty and Willingness toward Brand:What would you think if you were told that your company’s NPS is 0? It is logical to assume that your average customers neither feel good or bad about your service, and your brand is not appealing. Where, if you take a look at the chart below where NPS is 0, you can see that most users have a positive feeling toward your brand and service, and that NPS is showing a misleading overview!Now assume these scores were obtained in a country where people are not generous in giving a high score. That would aggravate the shortcoming of NPS. (Chart 1).7- An Inefficient Tool for Monitoring Progress or Regression:The hypothetical chart 3 that was obtained after chart 2 shows that people have grown dissatisfied with the service or brand, and lost loyalty. However, there are no changes in NPS, and no warnings are given.Now, imagine if the people hated the brand, and most of them gave a 0 (Chart 4).Meanwhile, managers and the staff have tried hard to improve user experience, provide a better service, and reduce the dissatisfaction in order to upgrade some of these zeroes to higher scores (Chart 5).Surprisingly, the chart shows that NPS (which is supposed to show user loyalty and willingness) remains unchanged, and cannot show the company’s progress or regression.The reason is that only the inter-group spillovers are calculated, and intra-group changes do not influence the final result. It can therefore be concluded that NPS is not sensitive enough to monitor progress or regression, and in certain cases (such as the above examples) it cannot show the changes at all.8- An Inefficient Tool Compared to Rivals:Imagine that NPS of your company and that of your rival is -50. The managers may assume their companies are under the same conditions. Now, take a look at the two charts below, both of which show the same NPS score of -50. It is obvious that the companies are not under the same condition, and NPS is not a suitable tool for comparing them, and can even be misleading. Chart 6 indicates that most people have a bad view of the company’s brand, whereas, according to chart 7, people have an average high view of the brand.Then, why is the use of NPS so widespread?Please read the following news from 2005:“450 sheep leaped to their deaths in the Turkish village of Gevas. The chain reaction started when one sheep went over the cliff, enticing nearly fifteen hundred others to follow… ”I hope I have made myself clear :) MohammadReza Khosravi MohammadReza Khosravi Tue, 03 Sep 2019 13:51:20 +0430 https://virgool.io/@m.khosravi/intelligent-sort-is-a-new-method-for-product-sorting-in-e-commerce-vyumdrqywsug 1. Introduction:Assume that a customer wishes to buy a pair of sneakers (a highly popular product on online stores). They visit Zappos.com and see 8887 products!Will the user browse through every single product? No.Most users are not seasoned buyers. They become so engrossed in the images on the product listing page (PLP) that they completely forget to sort and filter (even if they did, all sorting and filtering functions would vanish with a little scroll down the page!).More experienced users first choose their size (say, 10) to narrow down their choices. But, much to their surprise, there will be still 7367 products for the user to choose from.And, that goes to show the importance of products on top of the list.The better the top products, the greater the site’s sales. At the same time, if the products on top of the list do not appeal to users, they will not be willing to see the rest of the products.Product sorting determines what products should be put on top or at the end of the list.Therefore, a default sorting method is necessary for putting the “best” products on top of the list to maximize sales and create a better user experience.A fundamental question arises here: How do you define a “better” product? What features does it have?- Is being “better” necessarily the same as being more attractive or more viewed?The answer is no. For instance, Lamborghini has very attractive products, but most people cannot afford them. So they shouldn’t be placed on top of the list because they may hamper the sale of other potentially saleable products (Unlike magazines which strive towards attractiveness, profit generation is the top priority in e-commerce).More experienced users first choose their size (say, 10) to narrow down their choices. But, much to their surprise, there will be still 7367 products for the user to choose fromNegative. A cheap piece of electronics may sell well, but will create a bad shopping experience for the user and may even damage the reputation of the online store.- Are “better” products those with higher average customer reviews?The answer is again an unqualified no. In fact, there are many products which have been rated and reviewed by few users or none at all, meaning that their average customer reviews are irrelevant.It is obvious that the newest, the least expensive, or the most expensive product is not necessarily the best product. So Price: High to Low, Price: Low to High, and New Arrivals should not be used as the default sorting filters.The concept of a “better” product is a complicated and multidimensional concept defined by different parameters including reviews, sales, user comments, novelty, profit margin, etc.This paper proposes a sorting method that avoids the various complexities and uses a “simple” but “profound” concept to identify the best products.2. The Main Problems of Current Advanced Sorting Methods:2.1) Descending / Ascending Loop:The ratio of sales volume to the time a product is on sale is used as one of the most important parameters in the formulae used in recent innovative and advanced sorting methods.These sorting methods suffer from a serious problem: Descending / Ascending Loop.Meaning that products will be viewed less often and sell poorly if they are deep down the list. Poor sales will in return plunge them further down the list perpetuating the vicious cycle.The opposite scenario may take place on top of the list. That is, the products placed high on the list are not necessarily good, just as the products at the bottom are not necessarily bad.2.2) Validating Newly-Added Products:If new products are good, they should be put at the top. Placing them anywhere else will result in decreased sales. If new products are poorly-made, placing them on top of the list will damage the reputation of the business.The problem gets more complicated when one realizes that product quality cannot be considered in black and white, and that a specific position should be determined for products of medium quality.3. Intelligent Sort (I.S.):The intelligent sort is a new method that can be used by e-commerce companies to increase revenues and improve the user experience.This sorting method can be used as different filters such as the most popular and the most favorite applied to the lists of products.This formula benefits from a simple but deep concept at its core:SPV = SALE PER VIEW = SALE / VIEWSALE: the number of times a product has been soldVIEW: the number of times the product has been viewed by users on the product listing page (not on the product detail page)3.1) The Simplified I.S. Formula:I.S. = SPVIn this method, it is no longer important to put the item at the top or bottom of the list. For instance, consider Product A and Product B:Product A is at the top of the list, has been viewed 100 times, and has been purchased 20 times.Product B is at the bottom of the list, has been viewed 5 times, and has been purchased 3 times.Although A has sold more than B, B is the “better” product because it has a higher SPV.Now you may wonder how to obtain the number of product views on the PLP.It is possible to figure out how many rows of PLP users have seen by analyzing their scrolling patterns.This simple code may give useful insight into the problem:(this.scrollY+window.screen.height)/(document.body.scrollHeight)It is also possible to figure out how many rows users have seen by determining what percentage of the total height of PLP is covered by the height of every row of products.However, data do not have to be sent to the server every time users scroll the PLP. Rather, the number of viewed PLP rows (i.e. only a number) is sent to the server when the URL changes or when a browser tab is closed (or when a browser tab remains open for one hour).The products viewed on the PLP can be determined by having the above-mentioned number and the matrix of the viewed list of products.*Note: “Seeing” (done spontaneously) is different from “looking at” (done voluntarily); however, both are considered the same here for the sake of simplicity. In fact, distinguishing between the parts user see and the parts they look at something is impossible, even if sophisticated and expensive eye tracking equipment is embedded in every single user’s eyes. For more information, refer to “100 Things Every Designer Needs to Know About People”.If a product is viewed more often and purchased less often, there might be something wrong with it. So the product gets a negative point every time it is viewed but not purchased.Accordingly, is it necessary to count the number of times users view a series of PLPs without the intention of purchasing anything?Assume that a user looking for sunglasses views a PLP of various products (e.g. shirts, pants, etc.). Is it necessary to count the views of irrelevant products?The answer to both questions is negative.An in-depth analysis of the problem indicates that the following points should be taken into account in SPV calculation for more accurate results:A valid view is one that results in the purchase of a product. Unlike in-person purchases, many users visit online stores and just scroll pages without any purchase intention. So only views of products from the same category as the purchased product should be enumerated. In other words, if there are only two more pairs of sunglasses on the PLP, only the views of those two products should be counted. However, if all of the other products on the PLP are sunglasses, the view count of all viewed products is increased by one.Additional Notes:- If a user visits a product many times a day, the view count of the product will be increased only by one.- The views of recommendation widgets are counted like PLPs; however, the view counts of PDPs (product detail pages) are not excluded from calculations.3.2) I.S. Main Formula:SPV: main parameter (core parameter)Y­­j: complementary parametersX­i: customization parametersComplementary parameters are optional, but using them will improve the accuracy of the results. The effects of user reviews are considered in these parameters. The general opinion of the public on a product should be added to or subtracted from the SPV; however, it should not be so large as to overshadow the main parameter.The data employed in complementary parameters are extracted from different sections including average customer reviews, positive or negative reviews, recommendations for or against purchase, etc.Since not all websites feature these sections and because people from different countries and cultures have different opinions of a specific product, this data cannot have equal effects on every business. Therefore, this study has refrained from putting a specific value or percentage on these parameters. However, the effects of critical reviews should be given more weight than those of positive reviews because even a single unsatisfactory product can have severe consequences for the company.Personalizing parameters are used to convert sorts into recommendations, which are described in greater detail in the next section.4. Using Sort as RecommendationsIn the above formula, personalizing parameters are actually certain coefficients resulting in different sorts for different users. In other words, user behavior and log data can be analyzed to display the products that are both “good” and their personal favorites on top of other products.For instance, if user behavior and log data show that they have been looking for only black T-shirts with no interest in other products, the sorting system can assign a higher priority to their favorite products “boosting” them higher than the other products of the same tier.By default, the values of all personalizing parameters, including all features of a product (brand, color, material, size, etc.) are one. In the above instance, user experience can be improved by increasing the color coefficient of black products and displaying products that are both “good” and black higher on the list.In another case, if a user’s size is medium, medium-sized products will be shown first even if no filter is applied.5. Validating New Products:A serious problem of e-commerce businesses is that they are uncertain about where to put new products on the list of favorites.The online store will lose its appeal and customers if “good” new products are placed at the bottom of the list. The same is true if “bad” new products top the list.One validation method consists of exposing products to users for a limited period to gauge public reception.For this purpose, a value named SALE PER VIEW (SPV) is given to new products for a certain period and the product is put on top of the list for validation. (If the website traffic is high enough, a percentage of users can be selected as viewers instead of all users.) Also, the bottom products can be put through the same validation process for revalidation.It should be noted that ignoring the SPV altogether and using the sales rate as the only parameter is problematic.If only the sales rate is analyzed without considering the number of views, then, in the case of these two products, if:A is a newly-added product temporarily put on the first row to obtain its sales rate (0.0001), and B is an old product with the same sales rate (0.0001) put on the sixth row, then:The two products have the same sales rate. Should they be put next to each other on the list? The answer is no because A has achieved that sales rate because it has been on the first row (and viewed more often), and its sales rate will drop if it is put on the sixth row. This is exactly the type of problem that employing the SPV method can help solve.6. An Intelligent Step:After employing the I.S. to sort products and determine satisfactory and unsatisfactory ones, machine learning can be used to make the system more intelligent.In fact, I.S. and machine learning can be combined to determine where on the list to put new products without validation.Every product (e.g. a shirt) has certain specifications including price, color, brand, production year and season, size, material, pattern, etc. The “goodness” (popularity) of a newly-added product can be determined by analyzing the specifications of the “best” and “worst” (most and least popular) products.However, the learning process requires valid data obtained from the best products, and that data cannot be obtained without using the I.S. MohammadReza Khosravi MohammadReza Khosravi Tue, 03 Sep 2019 12:45:00 +0430