What is Ads.txt and How Does It Work?

Ad fraud has been a constant theme of online advertising and has often showed little sign of slowing down. Industry research reveals that it costs the digital ecosystem about $6 billion per year, with many online advertising companies completely unaware that it’s even happening.

For years, companies have looked at a range of possible solutions to combat this billion-dollar problem, even turning to blockchain for some inspiration, but as they say, sometimes the simplest ideas are the best ones, and a new IAB project called ads.txt is an example of this.

What is Ads.txt?

Ads.txt is an initiative developed by the IAB Tech Lab, aimed at combating certain types of ad fraud, mainly domain spoofing and illegitimate inventory arbitrage, and providing transparency in what is a highly opaque industry.

Released in May 2017, the ads.txt project is designed to clean up the online-advertising supply chain, help brands and advertisers purchase authentic digital media with confidence, and make it harder for fraudsters to profit from selling inventory that doesn’t belong to them.

While ads.txt revolves around advertising, the ads part is actually an acronym that stands for Authorized Digital Sellers.

What Problem Does Ads.txt Solve?

In particular, ads.txt aims to tackle domain spoofing (a type of ad fraud) and arbitrage, which isn’t exactly ad fraud, but is still a shady practice. Arbitrage is a process in which impressions are bought and then repackaged and resold at a higher price by a third party. Ads.txt helps to solve this problem by indicating who the authorized resellers of a publisher’s inventory are.

Domain spoofing, also known as domain hijacking, works in two ways—either by using malware that’s been installed on a user’s computer or by changing the URL in an ad tag.

Both methods produce the same result in that they trick ad exchanges and other programmatic platforms into thinking that the user is visiting a legitimate site, when in fact the ad will actually appear on a different, illegitimate site or displayed on a page in a hidden web browser.

Piracy sites, websites riddled with bot traffic, and other shady websites often use domain spoofing as a way to monetize sites that would otherwise be blacklisted or ignored by advertisers and brands.

The reason this fraudulent practice has existed for so long is because it’s hard for advertisers to confirm that the information passed in the bids is correct, meaning they can’t verify if their ads actually appeared on the sites they say the ads did.

Now, thanks to ads.txt, domain spoofing and selling illegitimate inventory have become a lot harder.

How Does Ads.txt Work?

First, a publisher adds an ads.txt file containing information about all the programmatic partners (supply-side platforms, ad exchanges, ad networks, etc.) that they work with to their web server and hosts it under their root domain.

Let’s take a closer look at the information in the ads.txt file and uncover what it all means.

Partner1, 678910, DIRECT, g45tg4e # banner

Partner1 – The AdTech platform the publisher uses to sell their inventory. Examples include appnexus.com, bidfluence.com, rubiconproject.com, doubleclickbygoogle.com, and pubmatic.com.

678910 – This field represents the Seller Account ID, which is also the publisher’s account ID for the respective AdTech vendors, and is used to verify the authenticity of the inventory during RTB auctions.

DIRECT / RESELLER – Direct means that the publisher works directly with the AdTech vendor to sell its inventory. Reseller means that the publisher has authorized another company (an ad network or digital advertising agency) to sell its inventory on its behalf.

g45tg4e – This optional field lists the Certification Authority ID, which identifies the advertising system within a certification authority, for example, the Trustworthy Accountability Group (TAG).

# banner – Some publishers include this extra field so they know which type of inventory the AdTech vendor sells (view cnn.com/ads.txt to see examples). As this hashtag represents a comment, it won’t be picked up by the crawling script unless certain configurations are made to it, but as this is purely for the publisher’s benefit, there’s no real need for buyers to have this information.

You can manually check to see if a website has an ads.txt file and view its contents by adding /ads.txt to the end of the root domain, e.g. businessinsider.com/ads.txt

Once a publisher has added the ads.txt file to their root domain, brands and advertisers can use a Python script to crawl the web (via a database they’ve created containing a list of domains) and see which publishers have an ads.txt file under their domain.

Once a brand or advertiser has a list of publishers that use ads.txt, they can reference this list against IDs in OpenRTB bid requests.

If the Seller Account IDs match, then the buyers (i.e. brands and advertisers) know that the publisher is who they say they are. If they don’t match or don’t exist, then it could mean that the domain is not coming from an authorized dealer or that the publisher hasn’t implemented ads.txt, and the buyers can choose not to bid on that particular inventory.

Ads.txt is an effective solution to combating domain spoofing because the publisher is the only one that can add the ads.txt file to its domain, meaning it can’t be altered by a different site and used to sell counterfeit inventory.

Is Ads.txt Being Adopted by Publishers?

Initial reports suggested that the ads.txt project wasn’t being adopted at the rate most would expect, considering it’s a pretty straightforward process and eliminates one of the biggest problems in AdTech.

However, since its release in May 2017, many top publishers have adopted the ads.txt project, including:

To put the adoption of ads.txt into some perspective, here are some figures pulled from reports that highlight the adoption rates:

• An Ad Ops Insider report found that 12.8% of publishers have an ads.txt file under their domain (released in September 2017).
• GetIntent found that only 1.3% of domains on its available inventory used ads.txt (released in August).
• A study by MarTech Today discovered that only 34 out of the top 500 (6.8%) most-trafficked sites had adopted ads.txt (released in August 2017).

The reasons for these low adoption rates vary with some publishers stating that although creating a text file and uploading it to their web server is easy enough, they still need to manually check the details with their partners to ensure they include the correct IDs to avoid being left off the list.

Interestingly enough, it looks like Google has played a large role in bumping up the adoption rates through its announcement that several of its AdTech platforms will start to filter for ads.txt files.

Ads.txt for Mobile Apps

Since its introduction in May 2017, ads.txt has successively been adopted by over 2m companies in the AdTech industry, and is currently making it much more difficult for fraudsters to spoof domains. However, there is another kind of ad fraud rampant in the industry today: mobile app fraud. We’ve written about it in a comprehensive post about the different methods of ad fraud on our blog.

The initial release of the IABs guidance for ads.txt implementation did not provide proper measures to prevent mobile app fraud. Luckily, the IAB Tech Lab released an update to the guidelines in June 2018, and made it available for public comment for a month (the period for feedback ended July 6, 2018). The organization called app publishers, AdTech vendors, and app stores to provide technical feedback to help refine and improve ads.txt for mobile apps.

Why Is Ads.Txt for Mobile Apps a Thing?

The problem with ads.txt in the context of mobile apps is that apps don’t have a convenient way to store the list of authorized domain sellers (ads.txt) – unlike in the case of the web, there is no web domain where the file can be stored.

To identify legitimate domains within the ecosystem, the IAB Tech Lab has envisioned three possibilities:

• Metadata fields in app stores. When developers register an app in iTunes, they create a bundle ID, which actually resembles a reverse URL, something like com.apple.pages (an example for the Pages app). Google Play Store uses a similar system.However, there is a certain reluctance from the stores, which prefer to work in their own interest. For example, Apple has a limitation on how many times a bundle ID can be looked up, which would make its application in ads.txt impossible. The IAB, as of yet, has made little progress on the matter with Google and Amazon.
• Implementing a standardized API created by an independent third-party to retrieve an app’s identifying metadata is an alternative option that the IAB Tech Lab has devised if Google and Apple refuse to collaborate.
• App stores (Google, Apple etc.) developing an API to support ads.txt, which would take longer to implement, and is fairly unlikely.

Dennis Buchheim, IAB Tech Lab’s general manager, believes the solution with metadata fields would be the most secure and reliable mechanism, and would also require the least technical effort from developers and website owners.

Bottom line: we have yet to see what the future holds for ads.txt in the context of mobile apps. Still, we must remember that ads.txt will only effectively curtail ad fraud when combined with the ad request encryption and authentication method ads.cert – we’ve written more extensively about the solution in our other posts here and here.

Reports of Ads.txt Botnet Scam Vulnerability

A 2018 scam recently reported by DoubleVerify revealed the vulnerability of ads.txt as scammers could have stolen between $70 and $80 million in advertisers’ money over the period of a year.

The situation results from two main reasons.

The first is that most publishers have far too many Resellers listed in their ads.txt file that they don’t have a direct relationship with. According to the DoubleVerify report, botnet operators created accounts with these resellers and used them to sell their fraudulent inventory by spoofing and posing as legitimate websites.

In a recent interview with MarTech Today, OpenX’s Chris Hallenbeck stresses the importance of introducing contractual requirements for exchanges to only sell inventory directly from publishers, or with a maximum ‘one-hop’ relationship.

The second reason relates to the fact that ads.txt is missing relevant authentication and verification measures. Specifically, ads.cert – a validation method which works alongside ads.txt to enable cryptographically signed bid requests – has not been widely adopted in the industry.

Ads.cert is intended to put an end to fraudulent activity which plaques the supply chain by making bad inventory look like premium inventory.

But ads.txt’s failure to permeate the industry is not only because there’s a lack of cryptographically signed bid requests via ads.cert, but because certain actors in the programmatic industry not thinking long-term and prioritizing profit over trust. It also results from certain actors in the programmatic industry not thinking long-term and prioritizing profit over trust.

Perhaps the next big step towards combating fraud and dishonest behavior in the online advertising industry is updating AdTech platforms to OpenRTB 3.0 (currently finalized and ready for adoption) as it contains the beta version of Ads.cert: Signed Bid Requests.

However, as with any new version of OpenRTB, the challenge is for AdTech companies to update their infrastructure to implement the new version. And unfortunately, there’s no backward compatibility option with OpenRTB. With all that’s happening in the AdTech industry, this probably isn’t a high priority at most companies.

Final Thoughts

While ads.txt won’t solve all of AdTech’s fraud woes, it will bring an end to one area and get companies used to an online advertising ecosystem that promotes transparency and proactively combats ad fraud and other shady practices. Most companies will embrace this new world, while others who have made a business of abusing the transparency issues in AdTech won’t.