Youku DSP Advertising System Architecture Practice

Article Author: Hongyan, Technical Expert at Alibaba Entertainment

Editor: Hoh Xil

Source: Alibaba Entertainment Technology

Produced by: DataFun

Note: Reprinting is welcome, please leave a message for permission.

Introduction: With the rise of the RTB network online advertising trading model, major companies have begun to build their own DSP (Demand-Side Platform) advertising systems to acquire customers. In recent years, Youku has also built a DSP system and is continuously iterating it. What technical explorations have been experienced during this process? What pitfalls have been encountered? What technical solutions have been accumulated? Below, I will share from a technical perspective, hoping to inspire everyone.

——Business Goals——

The core goal of DSP is user growth, enhancing user retention and activity through advertising acquisition and recall. From the perspective of monthly activity, we categorize users into layers: new users, low-active users, medium-active users, high-active users, and churned users.

Specific activity metrics can be defined based on the actual business situation. Once the audience segmentation is done, we can start recall efforts for each segmented audience. Our focus is to recall new users, low-active users, medium-active users, and churned users within the month.

• High-active users: Even without recall efforts, they will come back on their own;

• Churned users (users who haven’t visited in 30 days): The hardest to recall. This is because they may have uninstalled the app or switched to other apps. Data shows that the conversion ROI for churned users is also the lowest.

Therefore, our recall strategy is to advance the recall phase and start intervention during the low-active and medium-active stages. This is because the user acquisition cost during this phase is relatively low, and the ROI is high, allowing us to continuously reach out to these users, enhance retention, and migrate them towards high activity (of course, the user experience must keep up, forming usage habits and mindsets is very important).

——System Architecture——

Having clarified the business goals, let’s introduce our system architecture.

We will introduce the DSP system architecture practice and thoughts from several dimensions: channel capabilities, strategy capabilities, algorithm capabilities, real-time monitoring, and data.

❶ Channel Capabilities

Recall acquisition first requires strong channel capabilities to support it, ensuring we can reach and cover enough of the users we want. This article only involves channels that support RTB bidding ads, and there are many ADX in the market, such as Baidu ADX, NetEase, GuangdianTong, tanx, etc. Given so many channels, we cannot connect to all of them, and the traffic pressure is evident.

We first need to think about and evaluate which channels we should focus on connecting to, which channels have a large customer base, which channels have lower costs, and how the user segmentation looks for each channel. We need to have insights into channel quality.

Insights can be gained from the following indicators:

We need to analyze:

• How many users who installed Youku are from media traffic, and the user segmentation ratio;

• User overlap between multiple media, controlling cross-media exposure frequency;

• How many Youku users can be reached daily, and analyze the reasons for those not being reached;

• Media click-through rates and conversion rates (corresponding to Youku’s switching rates and playback rates), budget allocation, etc.;

• Achieve the best media combination for advertising;

• Optimize the network alliance channels, such as GuangdianTong.

For the GuangdianTong channel, we mainly connect to the network alliance, which may include thousands of APP media resources, each media’s effect and performance vary. This increases the optimization difficulty, and the traffic distribution from GuangdianTong is mostly based on the ratio of (successful bid PV/traffic distribution PV); the higher the ratio, the more traffic distributed. This brings a problem: if we want to acquire precise users, the PV participating in bidding will inevitably decrease, and the ratio will drop, ultimately leading to reduced traffic distribution and failing to cover more users, requiring a trade-off.

❷ Strategy Capabilities

Many advertising strategies are involved in the DSP advertising process. Here are a few common advertising strategies.

1. Frequency Control

Frequency control here includes: bidding frequency control, display frequency control, click frequency control, and switching frequency control across several dimensions.

Generally, frequency control mainly refers to display frequency control, and Youku has added three additional dimensions of frequency control.

• Bidding frequency control: This is media-specific; some media perform well, while others do not need to participate in bidding. Due to some media having a long delay between bidding and display, this can lead to no display, rendering display control ineffective.

  Users can continually participate in bidding and then display either in clusters or dispersed; in this case, display control becomes ineffective, and a user may be displayed hundreds of times.

• Click frequency control: Users who clicked on ads that day will not be purchased again to prevent fatigue.

• Switching control: Users who activated Youku that day will not continue to be purchased, as they have already been exposed to Youku, making further purchases unnecessary.

Below is an example illustrating the frequency control across different media; the effects of different media vary significantly, especially between network alliance channels and top media channels.

2. Material Racing Strategy

When delivering materials, an advertising plan may configure multiple sets of materials for delivery; without a racing mechanism, random delivery will occur.

The racing mechanism primarily solves the issue of providing high-CTR materials with as much exposure as possible while reducing exposure for low-CTR materials or not exposing them at all.

This is very similar to two classic problems in recommendation systems: the EE problem (from exploration to application) and the cold start problem.

Below is a simple racing logic implementation; the specific strategy is as follows:

Assuming delivery scheme A, there are advertising slots M, which meet the conditions for X sets of materials (X >= 2)

Material CTR = material clicks / material displays

Cold Start Process:

First, within a 15-minute time frame, the traffic for advertising slot M that meets delivery scheme A is evenly distributed among X sets of materials.

Second, calculate the CTR results of X sets of materials over the past 15 minutes [CTR1, CTR2,…, CTRx] and sort them, assuming the highest CTR is material N.

Third, in the next delivery (15 minutes) for advertising slot M meeting delivery scheme A during bidding:

• Allocate 80% of the traffic to material N

• Evenly distribute the remaining 20% traffic among the other X-1 sets of materials

Fourth, repeat steps 2 and 3 during delivery.

3. Smooth Delivery (budget smooth)

Smooth delivery ensures that advertisers can smoothly consume their budget under a fixed budget, rather than spending it all in the first few minutes.

Smooth delivery allows advertisers to reach more people and user groups over more time periods. Below are several consumption modes, illustrated with images from the web:

Previously, we used a flow-based smoothing mode; the issue was that during high traffic, more budget was allocated, but conversion ROI was not necessarily good. Currently, we use the third method, effect-based smoothing, where the effect mainly refers to click CTR. For example, during lunchtime, when users are more active on their phones, we allocate more budget for delivery during that time.

4. Advertising Retargeting Strategy (retargeting)

“Retargeting” is a term in advertising, also known as “visitor recovery ads,” “in-marketing ads,” “returning customer ads,” etc. It is an important way to help advertisers effectively acquire target audiences. For instance, during a consumer’s shopping process, they often do not complete the conversion due to various reasons. Retargeting ads display products to these consumers who did not complete the conversion, bringing them back to the relevant webpage. Since the displayed products are often what the consumers want the most, the conversion rate for retargeting ads is higher than that of general ads.

5. Image Visual Optimization Strategy

To provide users with a better experience and visual impact, enhancing their desire to click, we have conducted secondary processing on the images used in our ads, achieving good results.

Testing found that different channels have different visuals, so a unified design is not feasible. Therefore, we conducted visual design experiments divided by channel, for example, in a specific feed.

Through engineering combination capabilities, we quickly implemented and launched dozens of templates, conducting AB tests online to evaluate click-through rate effects and selecting excellent templates for long-term delivery.

❸ Algorithm Capabilities

Integrating algorithm capabilities is mainly to achieve personalized off-site advertising recommendations. Currently, the algorithm architecture is integrated.

1. Parallel Multi-Route Recall

• Recall based on playback history: This will recall based on user behavior within the platform, decaying over time;

• Recall based on trending programs: For example, new popular dramas released recently;

• Recall based on user preferences: For example, star preferences, series preferences, etc.;

• Recall based on on-site behavior: For example, favorites, clicks, browsing, search behaviors, etc.

2. Channel-Specific Models

Since we have integrated many channels, the characteristics of each channel’s performance vary, requiring targeted model training for each channel. Simple features are as follows:

❹ Data & Monitoring Capabilities

A complete data analysis system is crucial for DSP advertising. Through data analysis, we can optimize the conversion funnel and lower costs.

1. Conversion Funnel

First, we need to define the business conversion funnel, as shown in the figure below:

Next, let’s discuss the strategies we can intervene at each of these funnel stages.

• Participate in Bidding

  The volume of bidding participation is mainly limited by the user’s identification ability. If we can accurately identify Youku users based on device information, the volume of bidding participation can be greatly increased.

• Bidding Success

  The volume of successful bidding is restricted by several factors: the first is the bidding price; a low price leads to a high failure rate. The second is interface performance; currently, ADX requires a response within 100ms, which necessitates continuous optimization of the DSP’s response time to significantly improve the success volume.

• Display

  The display volume, such as title length, exposure frequency, etc. Another reason could be the media itself, as complex frequency control logic may lead to unsuccessful displays.

• Click

  This is first related to the accuracy of the algorithm’s recommendation capability; the more accurate the recommendation, the easier it is to click. Additionally, it relates to the attractiveness of images and titles as well as visual experience. For example, a captivating short video can significantly stimulate users’ desire to click, as can the previously mentioned visual image combination optimization.

• Awakening

  The switching rate is mainly limited by identifying whether the user has the Youku APP installed, whether the click was accidental, or if they are unwilling to open and watch due to timing issues. Another reason could be that the system starts too slowly, leading users to exit before the ad plays.

• Playback

  The playback rate is mainly limited by the performance of the playback page and pre-roll ads; if pre-roll ads are too long, users may exit before effective playback occurs. Additionally, network conditions such as 4G or WiFi also play a role.

• Next-Day Retention

  This is mainly related to the product’s user experience.

2. Comprehensive Monitoring

Detailed monitoring of all interactions in the advertising system is necessary to timely identify issues.

For example, sudden increases or decreases in expenditure, declines in click-through rates, or decreases in bidding participation volume all require detailed monitoring to avoid resource loss.

The monitoring system currently established includes the following indicators:

——Conclusion——

Finally, summarizing the entire DSP construction process, several points need to be considered:

• Channel construction, traffic access

• Accumulation of advertising strategies

• Utilization of algorithm models for recommendation advantages

• Comprehensive monitoring system

Of course, due to space limitations, this article does not cover another core aspect of DSP, which is traffic value estimation and bidding algorithms. This will be introduced in a separate article later. The system established by Youku has played an important role in customer acquisition and recall, laying a solid foundation for user growth. We are continuously building and improving it. Experts in the field of computational advertising are very welcome to join. Our team’s email is:

[email protected]

If you have related technical questions, you can also send them to the email for communication.

References:

1. https://zhuanlan.zhihu.com/p/52237643

2. http://www.swanlinux.net/2017/02/10/papers/real_time_bid_optimization/index.html

3. https://tech.meituan.com/2017/05/05/mt-dsp.html

4. https://zhuanlan.zhihu.com/p/28406017

5. https://help.aliyun.com/document_detail/59480.html

6. https://36kr.com/p/5189571

Author Introduction

Hongyan, Technical Expert at Alibaba Entertainment. Focused on the field of computational advertising, responsible for the construction of Youku’s digital marketing DSP platform.

——END——

Recommended Articles:

Decoding the Technical Architecture of Commercial Advertising Platforms

DataFun:

Focusing on knowledge sharing platforms in big data and artificial intelligence.

One “look,” one moment in time!👇