How AI can fix the Internet
Advertising giveth; advertising taketh
In my last post I wrote about how AI is helping reduce the capability gap that emerged between the billions of ads and advertisers on their platforms self-publishing on the one hand, and the US legal system on the other. Digitals self-service model of ads publishing created a huge regulatory hurdle to comply with legal privacy and security laws that big tech companies have been attempting in vain to comply with for a decade. As AI improves, those gaps are closing. As those gaps close, so too will the window to successfully grift off of cheap digital tricks. What will emerge out the other side is a pathway to restore the best parts of the original world wide web.
Websites serving junk content in attempts to juice ad dollars arguably killed the original internet. The internet of the late 90s revolved around niche content by hobbyists, directing users towards static sites with (maybe) an accompanying web forum. You still see many examples of this legacy model today, especially in sports content e.g. at LetsRun.com, the largest independent Track & Field news website. Sites drew large audiences of like-minded people together for decades before digital ‘audiences’ came into the nomenclature, but ads systems quickly followed. In 1994, 11 million households had an internet-connected computer of some kind and the first web banner ads were introduced the same year. By 2004, 70% of the US had internet connection of some kind and the digital ads market was just under $10 billion in the US and growing fast. Today, twenty years later, that market is $270 billion.
What people figured out between 1994 and 2004 was that the static sites of hosted, curated content could also easily host digital ads right alongside. The idea that you could turn internet celebrity or user bases into a revenue source used to be considered a joke, peaking with South Park’s 2008 ‘Canada on Strike’ episode that includes a scene where content creators storm Canada’s Department of Internet Money’ episode only to receive ‘theoretical dollars’ for views across platforms. Ironically within a year or so of that episode, we saw the founding and/or launching of some of today’s largest ad tech platforms – Criteo, AppNexus, recently-bankrupt MediaMath, Integral Ad Science, etc.
Users of the early internet in those days saw websites change from clean, simple websites and blogs to ad-laden behemoths. The worst hit were simple function sites that did things like track the weather or share sports scores. Recipe websites turned into rambling monologues, misclicks opened new tabs and pages with hidden background buttons, and sound would auto-play from everywhere.
The rise of the web2.0 companies did two things at once to this model. First, automation brought down a lot of single-function websites. Sites with map, weather, scoring, etc functions fell away fast with changes to Google’s SEO. Second was that the consolidation of users into social media, and therefore consolidation of media spending under a smaller and smaller set of actors meant that ad policies could be more easily applied. Google took about 10% of the digital ads market in 2007 with Facebook still a college campus project. By 2017 they combined for over 54% of the total digital ads market globally.
Facebook and Google’s UX was incredible by the standards of the era, and so too were their privacy and security policies in Ads systems. In 2006 Google was already building ML systems to combat click fraud, arbitrage issues, and more while folding these models into the recently-acquired DoubleClick purchase that became its mainstay programmatic arm. Those policies and models got more niche over time, restricting common bad actors in insurance, news & opinion, dangerous products manufacturers, and more.
And today the industry is coming for ‘made for advertiser’ sites, or MFAs, a nice way to say ‘a website made to shove as many ads as possible at you, and provide the tiniest amount of content possible.’ If you have ever tried to stream a show or game you didn’t have the app subscription for, or found yourself waiting minutes behind an impossible-to-exit ad layered on top of the content you're trying to reach, you’re familiar with an MFA. Even mainstream sites like Forbes.com can use their website content to explicitly drive traffic and juice ad-based revenues.
Yet from an advertisers perspective, MFAs are a huge money pit. Low quality MFA sites can drain over 20% of ad budgets that are optimized for cheap impression reach. “MFA sites are a great working example of a programmatic system being gamed,” said Damon Reeve, CEO of the U.K-based publisher alliance Ozone. “Advertisers don’t like them, publishers don’t like them, and yet advertiser budgets still flow to them. And that’s because they are designed to perform according to the ad-tech metrics that advertisers value for their digital budgets.”
Legislative policies like CCPA, industry regulations like FTCs ban on drug paraphernalia in ads, and industry standards like Apples ATT combined to help root out the worst sins of the internet from the mainstream. MFA laden content is now finally getting a similar skeptical eye, with major content publishers like Magnite, Sharethrough and PubMatic starting to use AI to find and restrict ads from going to MFA sites. As the AI sharpens, vendors like Google and the Trade Desk who once touted tools to combat MFA sites as industry-leading innovations in AI are finding these models now mainstream industry standards.
As the major publishers weed out ad impressions and click farms from the core of the internet, it could finally help bring back the best of the early days of the internet while avoiding the worst. As earlier pieces have discussed, emerging from the web2.0 era will not be simply transferring from Facebook to a new app. The confluence of new platforms to fill micro-niches from Substack to Shopify is like a supercharged version of the blogosphere of old, but now we don't all have to learn HTML.
AI has quietly stepped in to tackle some of the digital ecosystem's hardest problems, from the most sensitive topics like CSAM to the most sociologically challenging ones, like ‘what’s the tipping point between a lot of ads, and too many ads?’ for modern MFA sites toeing the line of usefulness. Despite how it may feel, the internet is already a much cleaner and more useful place than it was ten or twenty years ago, and with new capabilities we may still only be at the starting line of the internet age.