# Inside LinkedIn’s Generative AI Cookbook: How It Scaled People Search to 1.3 Billion Users Source: https://www.youtube.com/watch?v=wA0n8S1-tJc Recap page: https://rapidrecap.app/video/wA0n8S1-tJc Generated: 2025-11-14T20:04:08.9+00:00 --- ## Quick Overview LinkedIn successfully scaled its People Search feature to support 1.3 billion users by replacing the large, slow 7B parameter ranking model with a smaller, faster 220M parameter model, achieving this massive improvement by focusing on engineering mastery over brute-force scale, specifically by aggressively pruning the model size while maintaining relevance through careful data curation and a multi-teacher distillation process that encoded complex policies into the smaller model. **Key Points:** - LinkedIn scaled its People Search feature to support 1.3 billion users by aggressively pruning the ranking model size. - The initial ranking model used 7 billion parameters, which was deemed too slow and expensive for the required scale. - The new, scaled model utilizes only 220 million parameters, a significant reduction achieved through engineering optimization. - The key engineering technique involved training the smaller model using distillation from a much larger, complex model (the 'teacher model'). - The aggressive cut resulted in less than a 1% loss in relevance, demonstrating a massive leap in efficiency. - The engineering effort focused on distilling complex policies like fairness, ethics, and safety into the smaller model, rather than just focusing on model size. ![Screenshot at 00:19: The speaker introduces the core problem: the need to scale LinkedIn's generative AI people search, which launched about three years after ChatGPT, indicating the context of recent large-scale LLM deployment challenges.](https://ss.rapidrecap.app/screens/wA0n8S1-tJc/00-00-19.png) **Context:** This video details the technical approach LinkedIn took to scale its People Search feature, which had grown to serve over a billion users, a scale that their initial, massive generative AI model (likely a large language model) could not handle efficiently or cost-effectively. The discussion centers on the trade-offs between model size, speed, cost, and maintaining high relevance and policy adherence. ## Detailed Analysis LinkedIn encountered a significant scaling challenge with their generative AI People Search, which aimed to serve over a billion users. Their initial approach relied on a large 7 billion parameter model, which proved too slow and expensive, especially when considering the infrastructure cost of GPU usage for inference. The key breakthrough involved a massive model reduction: they pruned the model down to 220 million parameters. This reduction was not achieved by simply shrinking the model but through a sophisticated multi-teacher distillation process. The engineers used the large, complex model (the 'teacher') to train the much smaller model (the 'student') to mimic its behavior, specifically encoding complex policies related to fairness, ethics, safety, and business value into the smaller model's training. This process resulted in a 20x reduction in input size, yielding a 10x increase in ranking throughput, with only a negligible (less than 1%) loss in relevance metrics compared to the original model. The engineers successfully navigated the accuracy vs. speed trade-off by focusing on engineering mastery—creating a small, fast, compliant model—rather than relying on the costly brute force of simply using the largest available model. ### Scaling Challenge - Launching generative AI search 3 years after ChatGPT - Goal to serve over 1 billion users - Initial model size was 7 billion parameters. ### The Solution - Aggressively pruning model size down to 220 million parameters - Achieving a 20x reduction in input size and 10x ranking throughput increase. ### Technical Approach - Employing multi-teacher distillation to transfer knowledge from the large teacher model to the small student model. ### Policy Encoding - Distilling complex policies (fairness, ethics, safety, business value) into the smaller model via the distillation process. ### Results and Impact - Less than 1% loss in relevance metrics despite the massive model reduction - Outperforming the old method which relied on expensive lexical search. ![Screenshot at 00:05: Discussion begins on LinkedIn's technical playbook for scaling generative AI search.](https://ss.rapidrecap.app/screens/wA0n8S1-tJc/00-00-05.png) ![Screenshot at 00:12: The speaker highlights the scale: serving over a billion users globally.](https://ss.rapidrecap.app/screens/wA0n8S1-tJc/00-00-12.png) ![Screenshot at 00:23: The host notes that the old system relied on literal keyword matching, which was a chore for users.](https://ss.rapidrecap.app/screens/wA0n8S1-tJc/00-00-23.png) ![Screenshot at 00:35: The concept of aggressive pruning is introduced as the primary technique used to achieve scale.](https://ss.rapidrecap.app/screens/wA0n8S1-tJc/00-00-35.png) ![Screenshot at 00:54: The comparison between the slow, large model and the need for a faster, efficient solution is visually emphasized.](https://ss.rapidrecap.app/screens/wA0n8S1-tJc/00-00-54.png) ![Screenshot at 01:33: The speaker contrasts the old, rigid ranking system with the new approach.](https://ss.rapidrecap.app/screens/wA0n8S1-tJc/00-01-33.png) ![Screenshot at 02:25: A key takeaway is stated: the cost of building something that works at scale is the focus.](https://ss.rapidrecap.app/screens/wA0n8S1-tJc/00-02-25.png) ![Screenshot at 03:37: The VP of Engineering, Wenzing Zhang, is mentioned in relation to the ambitious goal of one giant AI system for everything at LinkedIn.](https://ss.rapidrecap.app/screens/wA0n8S1-tJc/00-03-37.png) ![Screenshot at 04:44: The focus shifts to the specific metrics achieved: a 10x throughput increase with a 20x input size reduction.](https://ss.rapidrecap.app/screens/wA0n8S1-tJc/00-04-44.png) ![Screenshot at 06:43: The massive cost change is highlighted: moving from large GPU clusters to a more efficient infrastructure.](https://ss.rapidrecap.app/screens/wA0n8S1-tJc/00-06-43.png)