Ad Budgeting Component for Real-Time Bidding (RTB) Auctions

A budget control system to help demand-side platforms (DSPs) reduce budget overspend.

Clearcode built a distributed budget management component for real-time bidding, known as banker, to help tech companies prevent budget overspend and decrease the development time when building a demand-side platform (DSP).

CLIENT

Startup

INDUSTRY

Programmatic advertising

SERVICE

AdTech

COUNTRY

Poland

About the Ad Banker project

One of our AdTech development teams completed an internal research and development project where we aimed to reduce overspending in a DSP when buying impressions during real-time bidding (RTB) auctions.

We called this component ad banker and made it part of our AdTech Foundations — a series of essential components used to build AdTech and MarTech platforms.

Key points

Problem

Budget overspending is a big problem for both DSP vendors and their clients.

Solution

Clearcode developed a solution to this problem. We called it ad banker.

Outcome

The ad banker reduces budget overspend during real-time bidding (RTB) transactions by ensuring optimal distribution of the advertiser’s budget between multiple bidders.

Results

Our benchmark tests found that ad banker reduced budget overspend from 37.2% to just 1.2%.

Tomek Chmielewski-Head of Operations
Tomasz Chmielewski, COO

“Our benchmark tests found that ad banker reduced budget overspend from 37.2% to 1.2%”

Tomek Chmielewski

COO AND PROJECT MANAGER OF AD BANKER

The Goal

The goal of the project was to develop a solution to reduce overspending in a DSP when buying impressions during real-time bidding (RTB) auctions.

The problem

Controlling an advertiser’s media spend is a major component of any demand-side platform (DSP). If it’s not managed correctly, you’ll either underspend or overspend — both of which disadvantage advertisers.

While underspending means that an advertiser’s campaign doesn’t achieve the right amount of reach, overspending can cause big problems for both the DSP vendor — they’ll likely have to cover the costs — and the advertiser — they would have wasted their opportunity to show ads to their target audience.

Key features

    • Preallocating more resources than requested

      Each resource requested by the bidder is multiplied by a set value in the cashier and the reservation with multiplied values is returned to the bidder to use in the future (e.g. the bidder needs $1 to bid and it gets a $10 reservation from the cashier).

    • ASAP and Even budget distribution

      ASAP budget distribution aims to spend the budget quickly.

      Even budget distribution aims to spend the budget evenly over the campaign’s flight dates. 

    • Integrations with existing demand-side platforms

      The ad banker can be integrated into existing demand-side platforms (DSPs) and also used as a component for new DSP development projects.

    How impression tracking works in most DSPs

    On any given web page, there may be half a dozen or so ad slots. Each ad slot, once loaded, will send an ad request to the publisher’s ad server, which in turn will send a request to an SSP/ad exchange and then on to multiple DSPs.

    Once the DSPs receive the bid request, they have to return a bid response, typically within 250ms.

    This process occurs billions of times per day across the Internet.

    Due to the extremely tight time frames, DSPs need to ensure that their advertisers’ budgets are being managed correctly.

    There’s typically a several second delay between the time a bidder responds to a bid request and when the DSP receives information that the impression has been displayed. This delay can cause the bidder to overspend when there are multiple bidders handling large amounts of traffic (i.e. bid requests).

    Impression tracking explained

    Here is a step-by-step explanation of what’s happening in the diagram above:

    1. The publisher sends an ad request to an SSP/ad exchange.
    2. The SSP/ad exchange then sends a bid request to the DSP’s bidder.
    3. If the bid matches a campaign in the DSP, the bidder sends back a bid (e.g. $1.05 CPM) and ad markup.
    4. Assuming the DSP wins the impression, the SSP/ad exchange sends the DSP’s ad markup to the publisher.
    5. The publisher loads the ad markup and sends a request to retrieve the creative (ad).
    6. The ad server sends the creative to the publisher and displays it to the user.
    7. The tracker records data about the impression and click (if applicable).
    8. The tracker sends information about the impression to update the frequency capping conditions.
    9. The tracker sends event data to the campaign reporting database, which will be used to generate reports.

    With this setup, there’s no proper and efficient way to manage the budget, meaning overspending will occur.

    Examples of budget overspend in a DSP

    To highlight how overspending can occur, let’s take a look at 3 examples:

    1. Broad targeting

    When creating a campaign in a DSP, advertisers and AdOps specify the targeting parameters, such as location, device type, and ad size.

    If the targeting is too broad, then the bidder will match a large number of bid requests to the campaign. This can occur simultaneously or within a few milliseconds of each other.

    While broad targeting isn’t typically an issue, without a proper budget management system, the bidder will likely overspend the budget as it won’t be notified about the winning impressions, and therefore can’t update the budget, until after the impression has been served.

    2. Digital out-of-home (DOOH) campaigns

    If a DSP is integrated with a DOOH platform whereby advertisers show their ads on digital billboards, then, depending on numerous factors such as the DOOH platform’s setup and media-buying process, there could be a lengthy delay in receiving information that an advertiser’s ad has been displayed.

    The delay can range from 15 minutes to 48 hours, in which case you cannot accurately track budget spend.

    3. Bidders and spending databases in different regions

    Due to the location of the DSP, there are some situations where the DSPs bidder and spending will be located in the same region as the SSP/ad exchange (e.g. AWS region us-east-1).

    However, there may also be situations where different DSP components (e.g. bidder and spending database) will be located in different regions.

    Because of the latency between different data centers, it’s possible that the DSP will overspend a budget unless a proper budget management component is added.

    The solution

    To avoid overspending with online advertising campaigns, we built a budget manager called banker. It’s based on Edsger W. Dijkstra’s banker’s algorithm.

    The banker is used to ensure optimal distribution of limited resources (budget) between multiple processes (bidders). It allows several bidders to receive and respond to bid requests at the same time and avoid budget overspending.

    • One of the first main tasks we undertook on this project was optimising the backend and database. We redesigned the database structure to enable it to operate more effectively and to increase the process speed of the requests sent from the merchants’ sites.
    • Our expertise in this field and our experience with similar projects enabled us to minimise the learning curve and reduce the time spent on planning the project. This allowed us to start developing the application straightaway and maintain a fast pace throughout the development process.
    • We used mongoDB for the application’s database, which provides the stability, reliability and performance needed to support the many functions and user requests.

    How it works

    The diagram below illustrates the role the banker plays and how it interacts with other components.

    You can see here the flow from the example shown above (Diagram #1) in the broader context of a DSP. Implementing the banker into a DSP changes the flow, which now looks as follows:

    Ad banker explained

    Here’s a step-by-step explanation of what’s happening in the image:

    1. The publisher sends an ad request to an SSP/ad exchange.
    2. The SSP/ad exchange then sends a bid request to the DSP’s bidder.
    3. If the bid matches a campaign in the DSP, then the bidder asks the cashier (a component of the banker) for resources to place a bid (e.g. $1.05 CPM).
    4. If there are resources available for the given campaign, the banker gives them to the bidder, which actually gives more resources than bidder requested for use in future auctions.
    5. If the bidder receives the resources from the banker, it sends back a bid response and ad markup.
    6. Assuming the DSP wins the impression, the SSP/ad exchange sends the DSP’s ad markup to the publisher.
    7. The publisher loads the ad markup and sends a request to the DSP’s ad server (either an internal or external one).
    8. The ad server sends the creative to the publisher and displays it to the user.
    9. The ad markup sends a request to the tracker with data about the impression and click (if applicable).
    10. The tracker informs the banker that an impression has been served and the budget is updated. If the tracker doesn’t inform the banker about the event within set time range, the resources are considered as not spent.
    11. The tracker sends information about the impression to update the frequency capping conditions.
    12. The tracker sends event data to the campaign reporting database, which will be used to generate reports.

    The technologies we used

    TypeScript

    JavaScript

    Pyton

    React

    NodeJS

    GO

    Angular

    The results

    To test the performance of the ad banker, we created different algorithms.

    Below are the results from the initial test.

    What the algorithm descriptions mean

    Naive. An algorithm with no banker capabilities: It assumes that the whole budget is available for bidding as it hasn’t yet received impression requests, and therefore hasn’t updated the budget.

    Banker with 1 minute reservations: This means that the reservation from the banker cannot be used again within the first minute of being released.

    …releasing the earliest reservation on impression and …releasing the reservation from the proper bid time: This refers to how we attribute the impressions to the banker’s reservations.

    NOTE: The above tests were carried out using the as-soon-as-possible (ASAP) budget distribution model. The results would vary if a different model was used, such as the pacing model.

    How to interpret the results

    The spend ratios represent how much of a campaign’s budget was spent by the given algorithms.

    The ideal spend ratio is 1, which represents 100% of a campaign’s budget being spent.

    We can see that the naive algorithm’s maximum overspend was 1.372, which for a $10,000 budget would translate into an overspend of $3,720.

    The table below represents the amount of budget overspend each algorithm produced during the testing phase.

    A note about 0% overspending

    Although the goal of the banker is to reduce overspending by as much as possible, with Banker5 getting overspending down to just 1.2%, achieving 0% overspending is unlikely without considerable underspending.

    This is a common situation among real-time bidding (RTB) platforms due to fluctuations in traffic, the creative type, and the delays in receiving win or impression notices.

    Find out how we can help you with your project

    Schedule a call with us today and find out how we can help you with your AdTech or MarTech development project.