Ads.txt – What it is and why it can’t protect you from invalid traffic and click fraud

Image source

While the initial release of ads.txt in 2017 was aimed at ads on websites, mobile ad fraud continued to grow. As a result, the IAB TechLab published app-ads.txt in 2019, an update to the existing specification that can also be used to fight ad fraud in mobile apps.

The new standard was also quickly picked up by Google and AdMob and now has an even higher adoption rate than ads.txt for websites. 68% of the top 1,000 Google Play apps have an app-ads.txt, but only 42% of the top 1,000 apps in Apple’s App Store.

The app-ads.txt deserves its own detailed article on our website. Until then, you can find the latest specification and instructions for implementation in apps on the official IAB website.

Vulnerabilities of ads.txt

Now that we have looked at the intentions and benefits of the ads.txt file, we need to take a look at potential vulnerabilities. First of all: There are some gaps in the system that could be exploited by fraudsters and the biggest error factor is, as so often, human error.

Lack of attention and maintenance from publishers

The ads.txt file is not a solution that you simply create and forget. Publishers need to actively keep track of listings, maintain it and also review it periodically.

When more and more parties are authorized to sell ad inventory on behalf of the publisher, the ads.txt file gets bigger and bigger. This not only makes it very difficult to read and maintain, but also no longer accurately reflects who is authorized to sell the actual inventory.

Every time a new partner is signed, they have to add that particular information to the file. The other way is the same: every time a partnership ends, the information must be removed from the file. Otherwise, publishers run the risk that long-expired partnerships still have access to their advertising inventory – at least according to the ads.txt entry. This situation can be very quickly exploited by malicious partners, which brings us back to the issue of faking ad request.

Take the New York Times example above. It currently has 21 authorized partners (as of June 2021) who are allowed to manage the advertising inventory. A direct relationship has been established with all partners, so no one can resell inventory without further ado.

If you look at ESPN.com, on the other hand, you are almost blown away. A whopping 460 partners are listed there, and more than half of them are resellers.

404Bot – Keep an eye on your reseller

At the beginning of 2019, a new scam made the rounds: 404Bot. Fraudsters leveraged the functionality of ads.txt by opening accounts with ad networks that were listed as approved resellers with major publishers.

It is estimated that due to the aforementioned lack of overview within a large ads.txt file, the scammers were able to steal between $15 and $80 million over a period of several months.

The scheme was pretty simple:

1. Fraudsters created accounts with advertising networks listed as resellers with major publications.
2. Then they sent bot traffic to fake but real looking URLs on the publishers’ website that didn’t exist and returned a 404 status code – hence the name 404Bot. HTTP status code 404 indicates that the browser was able to communicate with a particular server, but the server could not find the requested file.
3. However, there was ad code embedded on these fake URLs that was executed in the usual way.
4. The scammers were then paid by the ad network for successfully selling ad inventory from premium publishers.

The big advantage of this approach was also that the URLs appear legitimate in all reports. If someone wanted to visit the URLs, they were only shown the notice that the page was not found. However, it could also have been that the article had been deleted again by the publisher in the meantime. The perfect cover for the fraud.

As a publisher, it is essential that you keep an eye on the content of your ads.txt file and avoid resellers as much as possible. All domains that were affected by the 404Bot had one thing in common: Their ads.txt files were huge and contained a lot of reselling partners. The more resellers are listed in it, the higher the probability that you are affected by fraud.

Social engineering to be included in ads.txt

With the launch of ads.txt, publishers received more and more requests from fraudsters pretending to be digital marketing agencies who wanted to secure an official place as a reseller in the list. The scam looked something like this:

Scammers contacted major publishers to be included in their ads.txt. In doing so, they pretended to have acted as resellers in the past and to already have sold publishers’ ad inventory. One of these providers made it to over 60 lists in a short period of time because publishers did not verify the requests properly, but gave out many free passes, especially at the beginning, out of fear of losing revenue.

You can probably guess how the story ends. Once the scammers got on the list, they were able to sell ad space on premium publishers for a high price and use bot traffic to drive up profits.

In this case, it was not directly a vulnerability in ads.txt, but social engineering and thus humans as the weakest link.

Ads.txt can’t protect you from invalid traffic and click fraud

In this article we have covered everything worth knowing about ads.txt: What it is, how it works and also what the vulnerabilities are. While the current state is a step in the right direction to reduce ad fraud – in this case, especially domain spoofing – it’s important to keep one thing in mind: Ads.txt does not protect your ad spend from invalid traffic and click fraud!

Even if your ad is displayed on a website in a legitimate way, it does not guarantee that real people will see it or even click on it. Ads.txt makes sure that your ad is only displayed by authorized partners, not that real people interact with it.

As an advertiser, you will therefore continue to see invalid bot traffic and fake clicks on your ads, which will drain your advertising budget, even if your ads are served directly to publisher websites.

Very worrying in this context is a report at the end of 2019, which came to the conclusion that websites that use ads.txt have about 10% more invalid traffic (IVT) than websites that have not implemented ads.txt.

To solve this problem, you need ad fraud detection software that can detect fake bots and prevent them from clicking your ads and wasting your advertising budget.

Sign up for a free 7-day trial with fraud0 today and see the results for yourself – no strings attached and no credit card required.