Anomaly Detection Framework Enabling Fraud Prevention In AdTech

Recommender System to Improve Monetization for Ad-Tech Leader

How our Demand Forecasting Engine Helped a Retailer Prevent Stock-outs

How Personalized Recommendations Increased Sales For a Cosmetics Retailer

Custom

Our vision is to stand at the vanguard of this transformative trend, leveraging the power of cutting edge algorithms, and ethical AI practices to pave the path towards a progressive future.

<br />
<h1>Revolutionising Gaming With Cutting Edge <span class="gradient-blue-purple">Technology!</span></h1>


Yugen, since its inception has built some of the finest ML Systems in the Industry enabling growth for our partners.

Experience

Yugen drives growth transformations for industry leaders, poised to do the same for your enterprise

Leadership

Yugen is a tech leader in ML & GenAI Platform space. Achieve 10X speed in executing your ML initiatives.

Inhouse Tech & Platforms

Strategic collaboration with academia and industry leaders fuels cutting-edge R&D at Yugen.

Collaborations

Contact US

<br />
<h1>Revolutionising AdTech With Cutting Edge <span class="gradient-blue-purple">Technology!</span></h1>


Contact Us

Our vision is to stand at the vanguard of this transformative trend, leveraging the power of cutting-edge algorithms, and ethical AI practices to pave the path towards a progressive future

<h1>Revolutionising Industries With Cutting Edge <span class="gradient-blue-purple">Technology!</span></h1>


Team of world class Engineers and Data Scientist with expertise in building ML Systems


<h2 class="wp-block-heading"><strong>Problem Statement</strong></h2>



<p>The client is one of the leading global players in the mobile advertising and app monetization space. It facilitates running offers for advertising companies across multiple platforms in its partner network. This in turn helps both advertisers as well as publishers in customer acquisition and retention.</p>



<p>The client has operations in more than 100 countries with 15M+ daily active users who complete 80M+ advertiser offers daily through its interface on publisher mobile applications. The carrot used by advertisers for users to complete offers is rewards in the form of in-app currency or dollars.</p>



<p>The client had multiple product lines which catered to multiple types of offers like Videos, Shopping Offers, Click Offers, Engagement/ Install offers, Survey Offers etc. All of these offers compete against each other to get user attention. However, just getting user attention isn’t enough. To get the most out of the interaction, the user has to complete the offer. Some of these offers are lengthy or complex offering substantial rewards whereas some of the offers are simpler offering lower rewards. Some users drop off during the attempt, some may be disqualified by the advertiser whereas some others may try to game the system. As an Ad- network, the client had to show contextually relevant offers to these users to improve the probability of conversion and hence monetization rates. </p>



<p></p>



<h2 class="wp-block-heading"><strong>Approach</strong></h2>



<p>We started off with a two pronged approach.</p>



<ol>
<li>Understanding KPIs &amp; Behavior
<ul>
<li>Identifying the current monetization rates / KPIs to set benchmarks: We did this by quickly using Google Data Studio to generate a few reports/ real time dashboards to understand the trends</li>



<li>Identifying the user preferences: We conducted various deep-dives to understand which features impact user behaviour like
<ul>
<li>Value proposition of the offer: Offers having lower time spent per dollar reward earned are more attractive</li>



<li>Difficulty Level: the offer’s having higher difficulty level have a higher chance of a user quitting midway and not completing it etc.</li>
</ul>
</li>
</ul>
</li>



<li>Understanding the Tech Specs of the System
<ul>
<li>Doing these analysis above, gave us deep insights into things like:
<ul>
<li>Scale of data to be managed by the system</li>



<li>Expected Latency of the API</li>



<li>Potential data sources &amp; refresh rates</li>



<li>Types of models/ approaches which can be tried</li>
</ul>
</li>
</ul>
</li>
</ol>



<p></p>



<h2 class="wp-block-heading"><strong>Solution</strong></h2>



<h2 class="wp-block-heading">How we focused on rapid prototyping to production!</h2>



<p>Generating Impact early and being able to iterate fast are cornerstones of our operating methods! We built a team consisting of Data Engineers &amp; Data Scientists with a goal of rapid prototyping and moving to production fast! This will set the stage for future experiments and model releases.<br></p>



<p>The initial analysis had yielded around 10-15 features which had a significant impact on the user behaviour. Our data engineers started building data pipelines for feature engineering by connecting various data sources (Bigquery, GCS &amp; AWS). This would later on evolve to become a feature store. Different pipelines served different used cases. The ones which involved Big Data using PySpark &amp; Scala whereas some others where the scale of data was lesser were built using simple python scripts. These scripts were deployed using Airflow on GCS &amp; EMR clusters.</p>



<h3 class="wp-block-heading">Recommendation Model</h3>



<p>While these pipelines were being built, our Data Scientists were able to create a Training Dataset using Pyspark using the same feature set. They started developing a baseline version of the algorithm on AWS Sagemaker. Given, we wanted to create impact fast. We first rolled out experiments with some simple models like user-item &amp; user-user collaborative filtering to help set performance benchmarks. We tried various other algorithms like Contextual Bandits etc.&nbsp;</p>



<p>We also had to tackle the problem of cold start for newly launched offers and we developed an auditioning framework for the same, along-with a dedicated algorithm to ensure better candidate selection even in the auditioning phase.&nbsp;</p>



<h3 class="wp-block-heading">Inference Pipeline (API)</h3>



<p>The inference pipelines were first built using AWS API Gateway &amp; Lambda but as soon as we started using much more complex models, we moved the inference infrastructure to a dedicated EMR cluster which had auto-scaling enabled. We deployed the system as an API. The engineering team just needed to make a Get request from our API whenever a new user’s offerwall had to be rendered. We had to make multiple tweaks related to optimization and caching etc. to ensure that the latency of the API could be &lt;200ms. </p>



<p></p>



<h2 class="wp-block-heading"><strong>Impact</strong></h2>



<p>Over the last 1 year:</p>



<ol>
<li>Started with just 1 Geography (US), serving &gt;1M requests per day but scaled to ~21 countries, serving &gt;10M requests per day.</li>



<li>Monetization rate increased by ~70% YoY</li>



<li>Customer Return Rates increased by ~10%</li>



<li>Interactions per user increased from ~9 to ~13</li>
</ol>



<p></p>


Social Media, Marketing, Digitally Generated Image, Engagement


<p></p>



<h2 class="wp-block-heading">Problem Statement</h2>



<h3 class="wp-block-heading"><strong>Business Context</strong></h3>



<p>The client is one of the leading global players in the mobile advertising and app monetization space. It facilitates running offers for advertising companies across multiple platforms in its partner network. This in turn helps both advertisers as well as publishers in customer acquisition and retention.</p>



<p>The client has operations in more than 100 countries with 15M+ daily active users who complete 80M+ advertiser offers daily through its interface on publisher mobile applications. The carrot used by advertisers for users to complete offers is rewards in the form of in-app currency or dollars.</p>



<h3 class="wp-block-heading"><strong>Situation</strong></h3>



<p>Over the last few years scale has grown tremendously, and so has the challenges with respect to fraudulent users exploiting the reward system. Dollar amount associated with fraudulent attempts is to the tune of $25,000 every day. These attempts if not handled correctly, often result in the client incurring losses to the tune of millions of dollars every year.</p>



<h3 class="wp-block-heading"><strong>Objective&nbsp;</strong></h3>



<p>Build a Machine Learning system that can identify fraudulent attempts with high precision and recall. While it is extremely important to identify fraudulent attempts, it is imperative for the business not to misclassify genuine attempts.&nbsp;</p>



<p><em>“Impact of misclassification of a genuine attempt is often much higher than that of a fraudulent attempt in client’s business”</em></p>



<p>Legacy framework at client &#8211;&nbsp;</p>



<ol>
<li>Rule based Risk Framework that scores every offer attempt based on various factors. Ones above threshold are blocked</li>



<li>Post-event scoring by third party products that share potential fraudulent attempts that should have been blocked but weren’t. </li>
</ol>



<p>Yugen’s objective was to build an in-house solution that can help the client  in achieving a high precision and recall for Risk Framework predictions as well as high Match Rate with the ML predictions by third party solutions, the latter being an area of unknowns as the client did not have much visibility into factors being considered by 3P products.</p>



<p></p>



<h2 class="wp-block-heading">Approach</h2>



<p class="has-text-align-center">Project execution was split into three phases &#8211; <img decoding="async" src="https://lh3.googleusercontent.com/6ZYRHTx7OE6_JU2fqI4TO9zc6yW4FN2RniPw_WPfNDBdvYlcokfQCcahueRS06hflgcPuU3eMHaxwdLk5CyZWfLW8uzai9VNv33YIpcnvOCM6qwck23o6RqrzOSh1pm46cx3PEKYbHaL2M0hgVmQ3YM" style="width: 1000px;"></p>



<h3 class="wp-block-heading">Diagnostic Phase</h3>



<p>Yugen’s diagnostic framework has been designed to lay a strong foundation for the study to continue more smoothly and at a much faster pace when it is kicked off.</p>



<p>Yugen conducted an accelerated two-week diagnostic phase with the client to &#8211; </p>



<ol>
<li>Understand the problem by conducting stakeholder interviews &#8211; A series of discussions were conducted with the stakeholders in business, risk, engineering and data science teams to understand the Fraud identification issue, how is it being solved currently and what are the missing gaps in the existing workflow</li>



<li>Understand data availability and get accesses sorted</li>



<li>Conduct preliminary analysis to identify top drivers for the problem &#8211; </li>



<li>Scoping for work streams and alignment on target</li>
</ol>



<h3 class="wp-block-heading">Solution Design &amp; Development</h3>



<p>An ML Project constitutes 4 key phases &#8211;&nbsp;</p>



<ol>
<li>Data Analysis</li>



<li>Feature Engineering</li>



<li>Model Development &amp; Evaluation</li>



<li>Model Deployment</li>
</ol>



<p>While all these 4 components look siloed, for efficiency study execution, Yugen undertakes “<strong><em>an iterative approach for faster deployment and continuous optimization”.</em></strong></p>



<p></p>



<figure class="wp-block-image"><img decoding="async" src="https://lh3.googleusercontent.com/_x-8bsezo1xdZVfrIT0vu6otuoz6nuz-oy-cVkH6OnoT-a5ebFnmIOxwz7yt-tlI5E-6wwOth3XxfxLyul3Y291lDQdeGlblR-QVATLumfI5vr3cBQmvZQ4QLI3rb3ZnWVdnjwAMH2C_G8fghLTrkxU" alt=""/></figure>



<h4 class="wp-block-heading">Data Analysis</h4>



<p>Team came up with all possible hypotheses that could distinguish behaviour of a fraudulent user from a genuine user. A few examples are  &#8211; </p>



<ol>
<li>Usage of VPN &#8211; Geo-hopping to exploit offers from other regions</li>



<li>Bot usage &#8211; Identifying patterns in data that distinguishes bots from a human</li>



<li>Identity theft &#8211; Features to identify a user from behavioural or demographic attributes if the ID is changing frequently</li>



<li>Click activity &#8211; Identify anomalous patterns in click events of the user</li>
</ol>



<p>and many more.</p>



<p>Every hypothesis was tested on tested on three fronts &#8211; </p>



<ol>
<li>Whether it helps in distinguishing fraudulent behaviour from rest or not</li>



<li>How much value it can drive in model explainability</li>



<li>How much history model needs to understand for the feature (1hr, 1day, 1month etc.)</li>
</ol>



<p>Based on the above 3, the team came up with around 200 unique features and more than 500 including different forms of the same feature, that appeared to have potential value in model development.</p>



<p>During the process, small batches of features (~5) went through the steps &#8211;&nbsp;</p>



<p><strong><em>Hypothesis testing &#8211; &gt; Feature deployment &#8211; &gt; Model development -&gt; Evaluation</em></strong></p>



<p>This was done to make sure that all components can run in parallel and make the study as efficient as possible.</p>



<h4 class="wp-block-heading">Feature Engineering</h4>



<p>The scale of the system that needed to be developed helped us understand during the diagnostic phase itself that Engineering will need to work very closely with the ML team throughout study to build the right infrastructure that will support ML System development.</p>



<p>The team designed an in-house Feature Store for the client with more than 500 features. This feature store catered to the Fraud Detection use case that the team was working on and at the same time a few other Data Science, Analytics and Dashboarding use cases as well.</p>



<h4 class="wp-block-heading">Model Development &amp; Evaluation</h4>



<p>The Fraud detection use case that the team was solving is often referred to as Anomaly detection in the Machine Learning space. That’s due to the fact that fraudulent behaviour of users can also be seen as an anomaly when looked together with that of genuine users.&nbsp;</p>



<p>Moreover, the data at hand was highly imbalanced. Only 0.5% of attempts by users were flagged fraudulent which made this an Extreme Rare Event problem to solve. Most of the ML algorithms fail when it comes to solving such rare event use cases. While there are techniques to tackle imbalancing, they all depend on either reducing the sample size of a class or inflating that of the opposite class or performing both. In all such use approaches, data landscape changes a lot and impacts model performance as well.</p>



<p>Above reasons, led us to leverage one of the Anomaly Detection deep learning algorithms for model training. This process comprised 3 steps &#8211; </p>



<ol>
<li>Train ML model on all features and <strong>identify the ones adding valu</strong>e through feature importance tools, exploratory analysis, and iterative improvement. Multiple iterations were performed to identify around 40 important features which were bringing in almost the same value as all 100 unique features together</li>



<li>Optimize the feature weights through techniques such as Genetic Algorithm to provide higher weights to features adding more value and eventually enhance model performance</li>



<li>Evaluate model metrics, performance &#8211; Precision, Recall, True Positive/Negative rate, Math Rate with external data etc.</li>
</ol>



<p>Following the above 3 steps, the team was able to reach desired level of performance in freeze the models.</p>



<h4 class="wp-block-heading">Model Deployment</h4>



<p>At first a Post event model was deployed in production to run inference in batches one time a day on the previous day’s attempt. The results were analysed over time to understand drift in performance.&nbsp;</p>



<p>Once, the model did well and remained stable on post-event predictions. Real time model was deployed to run A/B experiments.</p>



<p>On successful execution of experiments, the model recommendations were scaled to 100% transactions.</p>



<p>The current model that runs in production has following attributes &#8211; </p>



<ol>
<li>3 different models have been developed and deployed for 3 different types of offers</li>



<li>Models have been scaled from 1 country to 21 countries now</li>



<li>Model Performance &#8211;
<ul>
<li>98% correct predictions for genuine attempts</li>



<li>95% correct predictions for fraudulent attempts</li>



<li>30% match rate with attempts that are flagged by third party products</li>
</ul>
</li>
</ol>



<h2 class="wp-block-heading">Impact</h2>



<p>The Machine Learning System developed and deployed by Yugen&#8217;s team is expected to help client save more than USD 5M by the end of October 2024.</p>


Credit card payment security


<h2 class="wp-block-heading">Problem Statement</h2>



<h3 class="wp-block-heading">Business Context</h3>



<p>The client is an American subsidiary of a leading player globally in the Gaming Entertainment industry. Their products are in three core segments &#8211; </p>



<ol>
<li>Gaming Consoles</li>



<li>Games &#8211; Digital and Physical Games</li>



<li>Peripherals &#8211; Controller, VR etc.</li>
</ol>



<p>The client does annual ad spend of ~ USD 200M across all the product segments. However, the decisions on budget allocation have always been driven by business heuristics backed by basic data analyses.</p>



<h3 class="wp-block-heading">Situation&nbsp;</h3>



<p>Owing to the scale and nature of operations, the client has always had multiple marketing teams such as Social Media, TV, Digital Marketing, Channel Marketing etc. taking care of their own initiatives and coming up with budget requirements to fit their needs. The system lacked a scientific and centralized approach for marketing budget planning.</p>



<p>This led to frequent occurrences of sub optimal ROI from products or channels, failure of initiatives run by different teams. Above all, budget allocation was a big challenge for the leadership.</p>



<h3 class="wp-block-heading">Objective</h3>



<p>The two core problems that were identified to be addressed were to understand &#8211;&nbsp;</p>



<ol>
<li>Effectiveness of various channels to understand which channel should be allocated how much budget (ad-spend)</li>



<li>Product dynamics to identify optimal spend for every product the client is selling or going to launch</li>
</ol>



<p>Team had to design and develop a data backed scientific solution to cater to above needs.&nbsp;</p>



<p></p>



<h2 class="wp-block-heading">Approach</h2>



<p>The study was split into 3 phases &#8211;</p>



<h3 class="wp-block-heading">Data Analysis</h3>



<p>Team gathered data from all marketing teams and external media collaborators and conducted an extensive 4 week analysis sprint to build hypotheses and validate the same. Since, the objective was to achieve maximum ROI, the impact of features was always studied on Sales (volume). This included performing various uni, bi and multivariate analysis to understand impact of features such as &#8211;</p>



<ol>
<li>Impact of Television GRP on Sales</li>



<li>Impact of Clicks, CTR on Sales</li>



<li>Impact of partner spend (channel marketing) on Sales</li>



<li>Impact of Pricing on Sales</li>



<li>Collinearity or Multi-collinearity in features, especially marketing features</li>
</ol>



<p><em>and many more</em></p>



<p>A few key insights observed were &#8211; </p>



<ol>
<li>Most of the marketing features show extremely high correlation. Therefore for better performance and right attribution it is important to leverage techniques such as PCA.</li>



<li>Games have a very fast diminishing curve for sales. 80% of sales happen in the first 60 days. Therefore it is important to plan marketing before or around launch period</li>



<li>Adstock effect in Game marketing doesn’t persist for long</li>



<li>Marketing of other products or in association with other products creates a Halo effect leading to positive impact on game sales. For e.g. marketing of consoles or game and console combo will boost game sales</li>



<li>Marketing behaves very differently for Consoles, Small &amp; Big Physical Games, Digital Games etc.. Therefore, one blanket ML model may not bring in a lot of value.</li>
</ol>



<h3 class="wp-block-heading">Model Development</h3>



<p>An important decision one needs to make before beginning development work is Model selection. A few key driving factors in this study were &#8211; </p>



<ol>
<li>Do features have a linear or no linear relationship with the target variable? In our space, marketing features have diminishing returns curve which is similar to either an S-curve or in most cases log curve</li>



<li>Is model explainability a requirement? When we talk about attribution then explainability becomes an important area. Several ML/DL models perform well but can’t add much value when it comes to explainability of impact by features</li>
</ol>



<p>The data was not vast which allowed us more liberty to pick even basic models. And, as most will understand, marketing spend doesn’t have a linear relationship with sales. Spending on marketing gives diminishing returns which means after an extent, every incremental dollar we spend will start giving lower return than earlier and eventually negative ROI.</p>



<figure class="wp-block-image"><img decoding="async" src="https://lh3.googleusercontent.com/qafVpUoY1yRQFlD5E__2C1Ti7tTq1Bc_04sLK44WAIu6B7nvmUBLFeaA6-nJS6btAzN8zFqSdyqgBJ1wk1k4R0C9OdqGH9eFzfD0bUII-S10HvS9SdyzTrKLppwqGPi9hUQTDR73T9VXxA_oev9V-S0" alt=""/></figure>



<p>Therefore, the team leveraged a Linear Regression model which was converted to a log-log model to fit the need of marketing features.&nbsp;</p>



<p>Another important piece to be figured out was the granularity of prediction and how many models are needed to be developed. As described earlier, since the behavior of marketing is very different when it comes to Consoles vs Games or Digital vs Physical Games, the team built 3 different models &#8211; </p>



<ol>
<li>Gaming Consoles</li>



<li>Physical Games</li>



<li>Digital Games</li>
</ol>



<p>To tackle multicollinearity PCA was leveraged prior to model development.</p>



<p>The output from this step were the equations (and coefficients) for the three ML models. These equations were to be used in the next step.</p>



<h3 class="wp-block-heading">Optimization Framework</h3>



<p>Once the models for Sales prediction were developed, the next step was to leverage those to assess optimal spend that should be allocated to &#8211;</p>



<p></p>



<p class="has-text-align-center"><strong><em>Product → Marketing Channel → Week</em></strong></p>



<p>in order to achieve maximum ROI</p>



<p></p>



<p><strong>Decision Variables &#8211; </strong>Variables for which optimal values were needed to be identified were &#8211; </p>



<p><em>Decision variable -&gt; </em><em>P</em><em>i</em><em>M</em><em>j</em><em>W</em><em>k</em></p>



<p>Where,</p>



<p>P &#8211; Product</p>



<p>M &#8211; Marketing channel</p>



<p>W &#8211; Week</p>



<p></p>



<p><strong>Constraints &#8211;</strong></p>



<p>In media mix model optimization constraints on marketing spend for every channel play a very important role to achieve maximum ROI. Since, the diminishing curve for every product is different, the team created a framework to analyze diminishing returns from every marketing channel spend as well as overall spend for every product separately. This step helped in identifying the spend thresholds beyond which&nbsp;</p>



<ul>
<li>ROI starts diminishing&nbsp;</li>



<li>Sales saturates</li>
</ul>



<p>In addition to these, there were various other constraints from business stakeholders on how much should be the minimum and maximum spend for any particular product.</p>



<p>All these constraints were specified in the optimization framework which eventually brought a lot of value.</p>



<p></p>



<p><strong>Objective function &#8211;</strong></p>



<p>The objective of the study was to maximize sales within the limits of spend constraints, therefore the fitness function was also designed on same lines. Sales predictions model equations were already present from previous step, so that only thing required was to aggregate sales across all weeks for all products &#8211;&nbsp;</p>



<p>log(Sales Product1Week1) = 0 + log(1Price11) + m x log(2TV GRP11) + n x log(3Clicks11) + &#8230;.</p>



<p>log(Sales Product1Week2) = 0 + log(1Price12) + m x log(2TV GRP12) + n x log(3Clicks12) + &#8230;.</p>



<p>.</p>



<p>.</p>



<p>log(Sales Product2Week1) = 0 + log(1Price21) + m x log(2TV GRP21) + n x log(3Clicks21) + &#8230;.</p>



<p>.</p>



<p>.</p>



<p>.</p>



<p>log(Sales ProductnWeekn) = 0 + log(1Pricenn) + m x log(2TV GRPnn) + n x log(3Clicksnn) + &#8230;.</p>



<p>All the above equations when summed up represented total sales for the year. Therefore, the aggregated sum of all the sales prediction equations was fed to the non-linear optimization model that generated the optimal values for all the decision variables that could help in maximizing sales and ROI.</p>



<p></p>



<h2 class="wp-block-heading">Spend Planning Tool</h2>



<p>Final step of the solution was to build a tool that can help all the business stakeholders come up with budget allocation for any product. Key features of the tool were &#8211;&nbsp;</p>



<p>Input &#8211; </p>



<ol>
<li>Users needed to feed product details such as Price, Launch date, Max budget etc. (all features except marketing spend)</li>
</ol>



<p>Output &#8211; </p>



<ol>
<li>Tool had an inbuilt optimization engine that recommended optimal spend across all “<em>products x marketing channel x week” </em></li>



<li>Effectiveness of all the marketing channels</li>
</ol>



<p></p>



<h2 class="wp-block-heading">Impact</h2>



<p>The study when it went to the implementation phase helped the client realize a 25% drop in marketing spend with a lift of 20% in revenue for the Fiscal Year. 15% in ROI lift was attributed to better MMX Framework.</p>


Network gaming.

How MMX Framework Helped a Gaming Company With 15% Higher ROI


<h2 class="wp-block-heading"><strong>Problem Statement</strong></h2>



<p>The client is a major player in the Fashion Retail Industry having &gt;1000 stores across India. They had an inventory of &gt;25,000 SKUs.</p>



<p><br>The ability to forecast demand has always been one of the key problem statements for players in the retail industry. Anticipating sales better, has a direct impact on procurement, logistics, store inventory levels and eventually sales as well. </p>



<p></p>



<h2 class="wp-block-heading"><strong>Approach</strong></h2>



<p>Demand in the fashion industry is driven by numerous factors. Some organizations even have specific design philosophies and they prefer to cater to that niche target audience only. Hence, this scenario is unlike a general demand forecasting exercise where a simple time-series model could suffice. We started the diagnostic phase for the project by interviewing store managers, procurement stakeholders and even a few designers to understand the intricacies involved.</p>



<p>Yugen’s team then broke the problem down into 4 parts:</p>



<ol>
<li>Performing in-depth analysis to understand the features driving demand &amp; procurement decisions</li>



<li>Developing pipelines to get the required features to the database</li>



<li>Developing a customized algorithm to forecast the demand for different product categories</li>



<li>Develop a monitoring &amp; measurement system to ensure Quality of predictions and to identify drift</li>
</ol>



<p></p>



<h2 class="wp-block-heading"><strong>Solution</strong></h2>



<p>This system was designed as a post-event system which could be triggered as needed.</p>



<h3 class="wp-block-heading">Data Engineering</h3>



<p>We built ETL Pipelines which would bring in meta-data from various different DataWarehouses into a single DB. Scale of the data in this case wasn’t an issue however, the ability to manage various processes &amp; pipelines was a big consideration while designing it.&nbsp;</p>



<p>We also foresaw the need to put in a Data Quality Monitoring System which ensures that anomalies in the data are appropriately handled. The system also issues alerts to us and the stakeholders while showing if the alerts are critical or just warnings.&nbsp;</p>



<h3 class="wp-block-heading">Optimization Algorithm</h3>



<p>The fully customised algorithm coupled with a rule-set based framework was developed which could:</p>



<ol>
<li>Identify SKUs of a style which have depleted and recommend replenishment quantities based on it’s sell-through</li>



<li>Identify and maintain service levels for various pivotal &amp; non-pivotal sizes for each store &amp; attribute bucket combinations</li>



<li>Maintain Optimum Depth for SKUs across various attributes like Price Band, Sleeve Length, Occasion Type etc. per store </li>



<li>Benchmark styles with it’s peers and identify dead/slow moving inventory in stores for pullback</li>



<li>Maintaining acceptable levels of Stock Cover accounting for Buffer Stock, Lead Times &amp; Sell Through</li>



<li>Leverage True ROS, pivotal size availability, variant availability etc. for each store-style combination to arrive at recommendations for allocation of new styles to a store</li>



<li>Suggest replenishments to regional warehouses based on the sales velocity of SKUs in it’s catchment area.</li>
</ol>



<h3 class="wp-block-heading">Inference Pipeline (API)</h3>



<p>We used AWS EventBridge to trigger the system deployed on AWS Lambda. The lambda can also be triggered via an API. Different sub-components of the system are modular in nature and can be triggered independently </p>



<p></p>



<h2 class="wp-block-heading"><strong>Impact</strong></h2>



<p>Post release:</p>



<ol>
<li>Improvement in service levels of store inventory was observed (up ~4%-7% in different stores)</li>



<li>Lift of ~1%-1.5% in sales was observed due to reduction in number of stock outs driven by higher availability of fast selling sizes &amp; colours</li>



<li>Reduction in % dead stock across different stores</li>



<li>Savings in terms of manpower &amp; their bandwidth</li>



<li>Faster Turnaround and better management due to automated process</li>
</ol>


Business Workers using digital tablet computer cargo and tracking delivery for Transportation and global network of smart logistics in blur warehouse background.


<h2 class="wp-block-heading"><strong>Problem Statement</strong></h2>



<p>The client is a major player in the Cosmetics Retail Industry having &gt;200 stores across India. They had an inventory of &gt;50,000 SKUs from hundreds of brands. They had ~5Mn Customers which helped them generate &gt;$100Mn in Revenue.&nbsp;&nbsp;</p>



<p><br>In today’s world, multiple brands are adopting digital transformation to help them capture their customer’s attention. However, to increase the brand’s share of wallet, they need to recommend the right product, using the right message at the right time to the right person i.e. hyper-personalization. The Retail chain wanted to create an amazing customer experience by helping shoppers find relevant products faster, which would in turn improve their average basket value.<br></p>



<h2 class="wp-block-heading"><strong>Approach</strong></h2>



<p>We kicked-off with a diagnostic phase where we interviewed some stakeholders to narrow down on the problem statement. We then performed some exploratory analysis to understand the current trends and various KPIs. Using these KPIs as a reference, we set quantifiable acceptance criteria for the project. This diagnostic / exploratory phase also gave us insights into the scale of data which needs to be processed by the system so that we could design its architecture accordingly.<br><br>Yugen’s team broke down the problem into the following 3 components:</p>



<ol>
<li>Building scalable ETL Pipelines capable of handling data at scale</li>



<li>Building a recommendation engine to understand user behavior and show relevant products</li>



<li>Building a micro-service to enable the retailer to seamlessly consume the output of the engine via an API.</li>
</ol>



<h3 class="wp-block-heading"><br><strong>“ABDM” Framework</strong></h3>



<p>Experimentation is a key tenet of Data Science. Yugen’s team leverages it with the ABDM Framework:</p>



<ol>
<li>Analyze<br>Perform Deep Exploratory Analysis to identify trends, patterns, features which influence customer purchase behavior</li>



<li>Build<br>Build infrastructure &amp; pipelines capable of handling data at scale at the desired latency such that Engineering is not an after-thought </li>



<li>Deploy<br>Focus on Fast Prototyping to production, which enables us to make model releases faster and hence improve consistently. All of our promising model iterations make it to production.</li>



<li>Measure<br>Building model performance measurement framework because “What gets measured gets improved!” </li>
</ol>



<p></p>



<h2 class="wp-block-heading"><strong>Solution</strong></h2>



<h3 class="wp-block-heading">Data Engineering</h3>



<p>To support the scale of data, we needed to build pipelines capable of handling the data for ~4-5Mn customers making ~100k transactions a day. We built ETL Pipelines using a combination of highly optimised Python Scripts &amp; NoSQL Queries and scheduled them using Chron Jobs. We initially thought about orchestrating the system with Airflow but we decided that it might be an overkill and we could do so later down the line.&nbsp;</p>



<h3 class="wp-block-heading">Recommendation Model</h3>



<p>Performing a deep exploratory analysis yielded us a lot of insights and helped understand customer behaviour. We experimented with a lot of different approaches &amp; algorithms like Collaborative Filtering (based on user-user &amp; user-item interactions) &amp; content based (like probabilistic classification models). While we were trying to optimize the models for better precision &amp; recall, we also had to ensure that diversity (serendipity) &amp; explainability were not compromised.&nbsp;</p>



<p><br>Upon further evaluation, we understood that Collaborative approaches have higher bias but low variance whereas the content based approaches have a lower bias but higher variance which is why we decided to go ahead with an ensemble hybrid model which uses a combination of both the approaches to:</p>



<ol>
<li>Identify candidate products </li>



<li>Identify the probability of conversion</li>
</ol>



<p><br>We also addressed the problem of cold-start (for new users) by first rolling out a few experiments like:</p>



<ol>
<li>Showing top 5 highest selling products of a store</li>



<li>Showing top 5 highest selling products based on demography</li>



<li>Showing top 5 products with highest discount</li>



<li>Showing newly launched products</li>
</ol>



<p>We eventually concluded that this problem should be solved using a combination of Collaborative approach &amp; a rule-set based framework for higher ROI.</p>



<p>We developed this model using data from pre-covid transactions however, we observed that the user behaviour has changed significantly post-covid like&nbsp;</p>



<ol>
<li>Many new categories of products were introduced in the stores (like disinfectants, sanitizers etc.)</li>



<li>Many of the top selling products were experiencing significantly lower sales (like Nail Polish/ Nail Paints etc.)</li>



<li>While only 5% of their revenue was from online stores previously, it grew to around 15%-20% post covid.</li>
</ol>



<p>We rolled out a series of experiments and understand &amp; test our models in different situations to get the best results.</p>



<h3 class="wp-block-heading">Inference Pipeline (API)</h3>



<p>We needed to serve ~8 recommendations for each user within a latency of ~500ms. We decided to build a micro-service and expose it in the form of API, so that it could be easily used by the clients. API contracts were designed to enable us to get information like CustomerID, Loyalty ID, Gender, Items currently in the cart etc. The system was built using a combination of services like AWS API Gateway, AWS Lambda &amp; AWS EMR.<br></p>



<p>The inference pipeline comprised of 3 different stages:</p>



<ol>
<li>Getting outputs from Collaborative filtering Framework which needs fast querying &amp; throughput</li>



<li>Getting outputs from Content Based Models which calculates the probability of a user purchasing each of the individual candidate SKUs</li>



<li>Combining the results, post processing and generating the final output</li>
</ol>



<h2 class="wp-block-heading"><strong>Impact</strong></h2>



<p>Post release, in a span of 3 months, we observed that:</p>



<ol>
<li>We served recommendations to ~2Mn customers who made ~7Mn transactions</li>



<li>The average latency of our API was ~400 ms.</li>



<li>Average Customer Basket Value increased by ~2.5% in a span of 3 months post release</li>



<li># of unique products purchased per customer increased by ~5%</li>



<li>Customer Retention improved by ~0.5 ppt</li>
</ol>


Rectangle 34624766

Our vision is to stand at the vanguard of this transformative trend, leveraging the power of cutting-edge algorithms, and ethical AI practices to pave the path towards a progressive future.

Explore our creations,

showcasing

our finest work.

Explore our creations,

Explore

our

creations

showcasing

our

finest work.

Filter by:

Manual work decreased by 45% in a month

Anomaly Detection Framework Enabling Fraud Prevention In AdTech

Manual work decreased by 50% in a month

How Personalized Recommendations Increased Sales For a Cosmetics Retailer

Connect with Yugen.ai

Yugen.ai Headquarters, Bangalore 560068, India

Industries

Product

Company

Services

© 2023 Yugen.ai Inc. All rights reserved

Privacy
|
Terms
|
Security

Yugen.ai Headquarters, Bangalore 560068, India

© 2023 Yugen.ai Inc. All rights reserved

Privacy
|
Terms
|
Security

ML Platform

Demo

Case Studies

Demo

Explore our creations,

showcasing

our finest work.

Explore our creations,

Explore

our

creations

showcasing

our

finest work.

Filter by:

Manual work decreased by 45% in a month

Anomaly Detection Framework Enabling Fraud Prevention In AdTech

Manual work decreased by 50% in a month

How Personalized Recommendations Increased Sales For a Cosmetics Retailer

Subscribe to Yugen.ai exclusive newsletter & updates

Connect with Yugen.ai

Yugen.ai Headquarters, Bangalore 560068, India

Industries

Product

Company

Services

© 2023 Yugen.ai Inc. All rights reserved

Privacy|Terms |Security

Yugen.ai Headquarters, Bangalore 560068, India

© 2023 Yugen.ai Inc. All rights reserved

Privacy|Terms |Security

Privacy
|
Terms
|
Security

Privacy
|
Terms
|
Security