How TikTok is Evolving the Next Generation of Search

Edwin Chen
Oct 25, 2022
How TikTok is Evolving the Next Generation of Search

The power of Reddit threads in Search has been discussed with great fanfare — and for good reason! Finding a Reddit thread on the SERP is like a kiss on a rainy day.

Kiss me, you beautiful Reddit thread.

But what about that other social media giant? TikTok is poorly understood by Silicon Valley engineers — “I'm too old for these shenanigans!”, they say — but it's an important way younger users discover new info and recommendations.

So where does Gen Z find TikTok helpful and why? To help old-school engineers get insight into those Search-forward millenials, we ran a human evaluation of TikToks on the search results page. Read on to learn more!

A superintelligent AI studying TikTok's role on the SERP.

Google vs. TikTok

Imagine your fiancé has booked you a $1000 reservation at Michelin 3-star Alinea. In order to learn more, are the Google SERP and a TripAdvisor page really your best bet?

Boooring.
Google's second search result. Only TripAdvisor could make Alinea look like a cheap candy store.

Or would you rather experience the magic through a short, firsthand TikTok?

Take all my money. https://www.tiktok.com/@celinelinarte/video/7014209427372788997

Google recently admitted that TikTok is eating into its core products, especially among younger users, who prefer to discover information and recommendations on TikTok and Instagram instead.

After all, if you’re visiting Los Angeles and making dinner plans at Din Tai Fung with your friends, do you really have time to read a 500-word Yelp wall of text?

Nobody has time for this.

Or would you prefer to dive in with a sexy, 18-second TikTok?

Google recently started displaying TikToks in its search results… But social discovery has always been its weakness.

  • Pinterest images are the bane of my Google life.
  • I love Reddit search results, but half the time, Google won’t even show the subreddit they come from.
  • Quora was amazing in its day, but showing them as a source of helpful information now…? Come on!

So when are TikTok results actually useful, and how can they be improved?

Human evaluation of search quality is one of our flagship applications. We partner with the top search engines in the world to help them deliver high-quality search results to users every day.

So to understand this better, we ran a large-scale search evaluation investigating the role of TikTok on the Google SERP. Let's dig in.

Search Evaluation: TikTok meets the SERP

To investigate TikTok’s place on the SERP, we launched a personalized human evaluation asking Surgers – the raters on our platform who evaluate search quality, train large language models with human feedback, and more – to rate thousands of <query, TikTok> pairs.

  1. First, we asked 150 Surgers to track their search queries, until we gathered 500 search result pages containing TikTok content.
  2. They then evaluated each <search query, TikTok> pair on a 5-point Likert scale (“Was this a helpful result for your query?”).

Here are some examples!

Search Query #1: toy storage hacks

Query: toy storage hacks

Intent: I wanted to find creative ideas that look nice for storing my kids’ toys.

Result:

Rating: Amazing. I love how there is a variety of videos showing many different shapes and sizes of toys that can be stored in different ways. I love that these videos inspire me to be more creative too.

It’s like Pinterest, but way better, because they show me videos of the entire room filled with actual kids’ toys. Not just generic stock images from a manufacturer’s website.

For example, here’s the first result on Google: It’s just a generic montage, filled with ugly images they just copy-pasted from somewhere. The person who wrote the article has never even used these in real life (and probably doesn’t even have any kids).

Google’s first search result: an ugly montage of copy-pasted stock images. The author probably doesn’t even have any kids!

In contrast, on Tiktok, you get the whole shebang. You see the storage hacks in the context of the whole room, you see the toys being placed in and out, it’s way more fun to watch, and if I like someone I can subscribe to their account to see more (whereas nobody subscribes to blogs anymore, except maybe boomers).

Search Query #2: heatless curlers trend

Query: heatless curlers trend

Intent: One of my friends mentioned that she had been using “heatless curlers” recently to curl her hair. She looked gorgeous, so I wanted to learn more.

Result:

Rating: Amazing. The tik toks were very helpful because I could see why the trend is so popular. There were multiple videos showing before and after results of using heatless curlers that were amazing. I decided it was worth getting these curlers to avoid heat damage, so I even went out and bought some afterwards.

Overall, I liked these much better than youtube videos (too long, the pre-video ads are annoying) and reading an article (too lazy, and most of them only contain still images). The comments are fun too and have some helpful tips!

Google’s first result. Who has time to watch an 11-minute video? This is what you get when you optimize your metrics for watch time over serendipity and pleasure…
Google’s second result. But who cares what Andrea Hans says? The page doesn’t even contain a single relevant image – on a fashion query, no less!

Search Query #3: best curly cucumber recipe

Query: best curly cucumber recipe

Intent: My friend showed me a video recipe of "curly cucumbers", and I wanted to find them again.

Result:

Rating: Amazing. The videos are short and sweet, and show how easy it is to make this dish. The videos are all less than a minute long, which is shorter than it would take me to get past a Youtube ad or navigate past blogger fluff.

I don’t think these videos would be sufficient, since I like having detailed instructions too, but I like starting with these first in order to see whether these are easy and pleasing to make in the first place.

The cucumbers are also curly, and the videos show you how to do that! Which is a key part of why I want to make this. But the first few non-TikTok search results on Google are just boring cucumbers like my dad would make.

The first result on Google – not even curly!

Search Evaluation: TikTok meets the SERP

Overall, here’s how Surgers rated the TikTok results.

The results of a large-scale, personalized human evaluation of TikToks in the Google SERP

Surgers found the TikTok results that Google displayed Amazing or Pretty Good 79% of the time. Pinterest, watch out too!

Qualitative Feedback on TikTok in Search Results

Let’s dig in a little more. After seeing TikToks in their search results for a couple weeks, where did Surgers think they would and wouldn’t be interested in seeing more?

In general, they called out fashion, recipes, travel, celebrities, tutorials, and product reviews. Here are a few Surgers in their own words.

Surger #1: “People want authenticity”

I’d absolutely love to see more Tik Tok results on the search results page.

I save tons of recipes on Tik Tok (coffee recipes, crockpot recipes, starbucks drink hacks) etc. I would totally use Google to search for that if it'd let me (but I feel like Tik Tok's rarely come up when I search on Google). I also look for local restaurants, movie reviews, and I spend a lot of time on 'booktok' (basically book reviews on Tik Tok). Those are all things I'd love to see on Google's search page. BookTok is so big that Barnes and Noble and other bookstores have their own sections dedicated to it, but Tik Tok was one of the last results on the first page when I searched it (similarly to the result I'm submitting, the Tik Toks are far down towards the bottom). I think that's a missed opportunity for Google, Tik Tok and the user!

I also spend a lot of time on Tik Tok looking at outfit ideas (IE fall outfits 2022). I don't want to watch a whole Youtube video on it, but I like to scroll through and see the different options on Tik Tok. That was a query that could have pulled a lot of Tik Tok's up on Google, but it didn't pull any up.

Similarly, on Instagram, I spend a lot of time looking at restaurants to try or things to do in my area, so I think reels/Instagram posts being in the top result would be nice.

I think people want authenticity and quick videos/answers, so Tik Tok and Instagram are really valuable in that sense. Especially for things like recipes, a lot of people don't want to scroll through a blog and a story just to get to the recipe content (and it's so much easier when you can visually see a recipe).

Surger #2: “On Google, I have to sift through ¾ of the page being filler”

There are quite a few instances where Instagram/TikTok videos would be beneficial.

Recently we started eating healthier, so we’ve been searching for recipes (mostly on TikTok). If I just went to a recipes page or one of the big recipe databases (Epicurious or anything that usually shows up on Google), I have to sift through 3/4 of the page being filler or stories the users out in for SEO. On TikTok though, they get straight to the point.

Another instance would be with decoration inspiration.

On the other hand, some things are just better on YouTube or text-based sites. Things like troubleshooting issues with my car or computer, or searching for local restaurants or businesses. Most of the restaurants around here aren’t active on TikTok/IG, at least that I’ve seen, so seeing results for a random restaurant when I want to see the hours of the restaurants around here would be nothing but an annoyance. There could also be troubleshooting steps on TikTok, but I’ve never seen them and it doesn’t seem like the greatest platform for something like that.

If I’m searching for entertainment, like a comedian, results from TikTok or IG would also be useful, since most of the comedians I know have a presence on TikTok/IG. I could find similar results in longer form with YouTube results, but honestly the ads have been putting me off a lot lately, so I’ve almost gone solely TikTok for that sort of content.

Watch Out, Google

First TikTok came for Meta. Then it came for… Google?!

Remember the 2010s, when upstarts like Facebook, Quora, and Pinterest had the potential to threaten Google Search’s throne? When Google was so worried that it built its ill-fated Google+ and offered Snapchat $30B to fend off the social hordes?

The theory:

  • Why search for a new restaurant or TV to buy, when you could ask your Facebook friends instead?
  • Why search for math tutorials and startup news, when you could ask gossipy Silicon Valley insiders on Quora?
  • Why search for furniture and fashion, when design inspo awaited on Pinterest?

Of course, those fears never materialized. The last time I searched for “burritos near me” on Facebook, it showed me a dead restaurant 45 minutes away.

But times change. Who doesn’t shudder at the SEO farms and huckster blogs that rise to the top of Google these days? Does your mouth crack wide with joy when a magical Reddit result appears like a ray of sun?

TikTok has toppled Facebook and Instagram's throne. Is it coming for Google next?

Build the next generation of Search

Interested in learning more about the role human evaluation plays in building modern, first-class search engines?

Neeva recently used the Surge AI platform to run their own search insights studies on the role of TikTok in Search. For example, they found that TikTok search results were especially useful for Cooking/Recipes queries and Beauty queries, but worse at Sports and Legal!

Check out their tweet thread for the full insights.

And if you want to read more about search quality, the evolving tastes of Gen Z, and human evaluation, check out our other posts on the Surge AI blog!

Edwin Chen

Edwin Chen

Edwin oversees Surge AI's Engineering and Research teams — whether it's helping customers train large language models on human feedback, building content moderation algorithms to detect hate speech and spam, or scaling up an elite data labeling workforce. He previously led AI, Data Science, and Human Computation teams at Google, Facebook, and Twitter, and studied mathematics and linguistics at MIT.

surge ai logo

Data Labeling 2.0 for Rich, Creative AI

Superintelligent AI, meet your human teachers. Our data labeling platform is designed from the ground up to train the next generation of AI — whether it’s systems that can code in Python, summarize poetry, or detect the subtleties of toxic speech. Use our powerful data labeling workforce and tools to build the rich, human-powered datasets you need today.

Data Labeling for the
Richness of AI

Get Started

Want to learn more? Schedule Demo

Follow Surge AI for the latest in AI, language, and data labeling!