Supply-Side Platform (SSP) Case Study

Our Developed Solutions

Based on our knowledge of SSP/ad exchanges and RTB, as well as extensive research, we developed the following solutions to the aforementioned challenges.

The Solution to the Scalability Challenge

To ensure the ad exchange could scale – i.e. receive, handle and process more ad/bid requests – we deployed a microservice architecture coupled with a horizontal scaling pattern.

Microservice architecture is an architectural pattern used in software development projects to break down the application into individual and related services.

These services can be developed, deployed and maintained independently from the rest of the application and communicate between one another via application programming interfaces (APIs).

Adopting a microservice architecture provided us with the following benefits:

• It allowed us to achieve the target levels of scalability, as each microservice can horizontally scale independently.
• We can deploy these services individually rather than as a whole, which reduces deployment time.
• Having a microservice architecture means that if one microservice becomes faulty, it will be isolated to that service and won’t impact the rest of the ad exchange.

The Solution to the Performance Challenges

The main performance challenges in an SSP/ad exchange relate to its ability to:

• Handle large numbers of inventory changes made in the SSP.
• Propagate these changes to all exchange instances.
• Ensure the ad exchange instances can handle this resource-heavy task without causing the SSP to experience unexpected performance issues.

To solve these challenges, we introduced inter-service communication over the publish-subscribe (pub/sub) layer and developed a populator service.

Inter-service communication over the publish-subscribe (pub/sub) layer

Pub/sub is a messaging pattern designed to publish and receive messages by subscribing to a specific channel or subject.

To carry out the pub/sub pattern, we opted for NATS, an open-source messaging service.

Its role in the SSP environment is to inform exchange instances about any inventory changes.

Due to the nature of the NATS services, pub/sub provides a simple, secure and fast solution to the messaging distribution challenge.

Since pub/sub is a queueless solution, it eliminates many problems associated with queues and their limitations.

Populator service

Typically, when a change is made in an SSP, each exchange instance needs to download the changes in the inventory on startup.

If every exchange instance queries the SSP and downloads the inventory directly, serving those requests would require extensive resources and would likely crash the SSP.

To overcome this challenge, we introduced a populator service to function as a proxy and single point of communication between the exchange instances and the SSP, allowing us to achieve and ensure high performance.

The populator service notifies the exchange instances about new changes, updates the inventory in the exchange instances at a set interval, and prepares the inventory for easy loading.

This means the resource heavy task of providing the new inventory is performed in a predictable fashion, meaning there is no unexpected performance issues for the SSP.

Also, exchanges can always reliably download the latest inventory changes, even if the SSP is inaccessible.

How the populator service works

The populator service:

• Queries the SSP for the inventory on startup.
• Creates a unique hash for the response.
• Stores the response as a file in an object storage service, such as Google Cloud Storage.
• Publishes the file location on a pub/sub topic.
• Queries the SSP for the inventory on a schedule.
• Compares the response hash with the last one.
• If it differs, it stores the response and publishes the file location on a pub/sub topic.

In order to ensure it has the most up-to-date inventory, the exchange:

• Queries the populator for inventory file location on startup.
• Subscribes to the pub/sub topic to be notified of new inventory files.
• Downloads new files and replaces its inventory.

Once downloaded, the information is stored locally by the exchange instances.

Because of this setup, the exchange always has information about the inventory in its memory. In order to verify its validity, it doesn't need to download file storage, which would take a lot of time and resources; instead, it listens to the populator (pub/sub) for annotations regarding any changes in the inventory.

This results in faster performance, as there is no need to request the ad-slotIDs from the SSP each time an RTB auction is performed. The reason for this is because the exchange already has them stored in an in-memory database.

Faster performance translates to overall improvements in the ad/bid response time needed for conducting an RTB auction.