The power of Reddit threads in Search has been discussed with great fanfare — and for good reason! Finding a Reddit thread on the SERP is like a kiss on a rainy day.
But what about that other social media giant? TikTok is poorly understood by Silicon Valley engineers — “I'm too old for these shenanigans!”, they say — but it's an important way younger users discover new info and recommendations.
So where does Gen Z find TikTok helpful and why? To help old-school engineers get insight into those Search-forward millenials, we ran a human evaluation of TikToks on the search results page. Read on to learn more!
Google vs. TikTok
Imagine your fiancé has booked you a $1000 reservation at Michelin 3-star Alinea. In order to learn more, are the Google SERP and a TripAdvisor page really your best bet?
Or would you rather experience the magic through a short, firsthand TikTok?
Google recently admitted that TikTok is eating into its core products, especially among younger users, who prefer to discover information and recommendations on TikTok and Instagram instead.
After all, if you’re visiting Los Angeles and making dinner plans at Din Tai Fung with your friends, do you really have time to read a 500-word Yelp wall of text?
Or would you prefer to dive in with a sexy, 18-second TikTok?
Google recently started displaying TikToks in its search results… But social discovery has always been its weakness.
- Pinterest images are the bane of my Google life.
- I love Reddit search results, but half the time, Google won’t even show the subreddit they come from.
- Quora was amazing in its day, but showing them as a source of helpful information now…? Come on!
So when are TikTok results actually useful, and how can they be improved?
Human evaluation of search quality is one of our flagship applications. We partner with the top search engines in the world to help them deliver high-quality search results to users every day.
So to understand this better, we ran a large-scale search evaluation investigating the role of TikTok on the Google SERP. Let's dig in.
Search Evaluation: TikTok meets the SERP
To investigate TikTok’s place on the SERP, we launched a personalized human evaluation asking Surgers – the raters on our platform who evaluate search quality, train large language models with human feedback, and more – to rate thousands of <query, TikTok> pairs.
- First, we asked 150 Surgers to track their search queries, until we gathered 500 search result pages containing TikTok content.
- They then evaluated each <search query, TikTok> pair on a 5-point Likert scale (“Was this a helpful result for your query?”).
Here are some examples!
Search Query #1: toy storage hacks
Query: toy storage hacks
Intent: I wanted to find creative ideas that look nice for storing my kids’ toys.
Result:
Rating: Amazing. I love how there is a variety of videos showing many different shapes and sizes of toys that can be stored in different ways. I love that these videos inspire me to be more creative too.
It’s like Pinterest, but way better, because they show me videos of the entire room filled with actual kids’ toys. Not just generic stock images from a manufacturer’s website.
For example, here’s the first result on Google: It’s just a generic montage, filled with ugly images they just copy-pasted from somewhere. The person who wrote the article has never even used these in real life (and probably doesn’t even have any kids).
In contrast, on Tiktok, you get the whole shebang. You see the storage hacks in the context of the whole room, you see the toys being placed in and out, it’s way more fun to watch, and if I like someone I can subscribe to their account to see more (whereas nobody subscribes to blogs anymore, except maybe boomers).
Search Query #2: heatless curlers trend
Query: heatless curlers trend
Intent: One of my friends mentioned that she had been using “heatless curlers” recently to curl her hair. She looked gorgeous, so I wanted to learn more.
Result:
Rating: Amazing. The tik toks were very helpful because I could see why the trend is so popular. There were multiple videos showing before and after results of using heatless curlers that were amazing. I decided it was worth getting these curlers to avoid heat damage, so I even went out and bought some afterwards.
Overall, I liked these much better than youtube videos (too long, the pre-video ads are annoying) and reading an article (too lazy, and most of them only contain still images). The comments are fun too and have some helpful tips!
Search Query #3: best curly cucumber recipe
Query: best curly cucumber recipe
Intent: My friend showed me a video recipe of "curly cucumbers", and I wanted to find them again.
Result:
Rating: Amazing. The videos are short and sweet, and show how easy it is to make this dish. The videos are all less than a minute long, which is shorter than it would take me to get past a Youtube ad or navigate past blogger fluff.
I don’t think these videos would be sufficient, since I like having detailed instructions too, but I like starting with these first in order to see whether these are easy and pleasing to make in the first place.
The cucumbers are also curly, and the videos show you how to do that! Which is a key part of why I want to make this. But the first few non-TikTok search results on Google are just boring cucumbers like my dad would make.
Search Evaluation: TikTok meets the SERP
Overall, here’s how Surgers rated the TikTok results.
Surgers found the TikTok results that Google displayed Amazing or Pretty Good 79% of the time. Pinterest, watch out too!
Qualitative Feedback on TikTok in Search Results
Let’s dig in a little more. After seeing TikToks in their search results for a couple weeks, where did Surgers think they would and wouldn’t be interested in seeing more?
In general, they called out fashion, recipes, travel, celebrities, tutorials, and product reviews. Here are a few Surgers in their own words.
Surger #1: “People want authenticity”
I’d absolutely love to see more Tik Tok results on the search results page.
I save tons of recipes on Tik Tok (coffee recipes, crockpot recipes, starbucks drink hacks) etc. I would totally use Google to search for that if it'd let me (but I feel like Tik Tok's rarely come up when I search on Google). I also look for local restaurants, movie reviews, and I spend a lot of time on 'booktok' (basically book reviews on Tik Tok). Those are all things I'd love to see on Google's search page. BookTok is so big that Barnes and Noble and other bookstores have their own sections dedicated to it, but Tik Tok was one of the last results on the first page when I searched it (similarly to the result I'm submitting, the Tik Toks are far down towards the bottom). I think that's a missed opportunity for Google, Tik Tok and the user!
I also spend a lot of time on Tik Tok looking at outfit ideas (IE fall outfits 2022). I don't want to watch a whole Youtube video on it, but I like to scroll through and see the different options on Tik Tok. That was a query that could have pulled a lot of Tik Tok's up on Google, but it didn't pull any up.
Similarly, on Instagram, I spend a lot of time looking at restaurants to try or things to do in my area, so I think reels/Instagram posts being in the top result would be nice.
I think people want authenticity and quick videos/answers, so Tik Tok and Instagram are really valuable in that sense. Especially for things like recipes, a lot of people don't want to scroll through a blog and a story just to get to the recipe content (and it's so much easier when you can visually see a recipe).
Surger #2: “On Google, I have to sift through ¾ of the page being filler”
There are quite a few instances where Instagram/TikTok videos would be beneficial.
Recently we started eating healthier, so we’ve been searching for recipes (mostly on TikTok). If I just went to a recipes page or one of the big recipe databases (Epicurious or anything that usually shows up on Google), I have to sift through 3/4 of the page being filler or stories the users out in for SEO. On TikTok though, they get straight to the point.
Another instance would be with decoration inspiration.
On the other hand, some things are just better on YouTube or text-based sites. Things like troubleshooting issues with my car or computer, or searching for local restaurants or businesses. Most of the restaurants around here aren’t active on TikTok/IG, at least that I’ve seen, so seeing results for a random restaurant when I want to see the hours of the restaurants around here would be nothing but an annoyance. There could also be troubleshooting steps on TikTok, but I’ve never seen them and it doesn’t seem like the greatest platform for something like that.
If I’m searching for entertainment, like a comedian, results from TikTok or IG would also be useful, since most of the comedians I know have a presence on TikTok/IG. I could find similar results in longer form with YouTube results, but honestly the ads have been putting me off a lot lately, so I’ve almost gone solely TikTok for that sort of content.
Watch Out, Google
First TikTok came for Meta. Then it came for… Google?!
Remember the 2010s, when upstarts like Facebook, Quora, and Pinterest had the potential to threaten Google Search’s throne? When Google was so worried that it built its ill-fated Google+ and offered Snapchat $30B to fend off the social hordes?
The theory:
- Why search for a new restaurant or TV to buy, when you could ask your Facebook friends instead?
- Why search for math tutorials and startup news, when you could ask gossipy Silicon Valley insiders on Quora?
- Why search for furniture and fashion, when design inspo awaited on Pinterest?
Of course, those fears never materialized. The last time I searched for “burritos near me” on Facebook, it showed me a dead restaurant 45 minutes away.
But times change. Who doesn’t shudder at the SEO farms and huckster blogs that rise to the top of Google these days? Does your mouth crack wide with joy when a magical Reddit result appears like a ray of sun?
TikTok has toppled Facebook and Instagram's throne. Is it coming for Google next?
Build the next generation of Search
Interested in learning more about the role human evaluation plays in building modern, first-class search engines?
Neeva recently used the Surge AI platform to run their own search insights studies on the role of TikTok in Search. For example, they found that TikTok search results were especially useful for Cooking/Recipes queries and Beauty queries, but worse at Sports and Legal!
Check out their tweet thread for the full insights.
And if you want to read more about search quality, the evolving tastes of Gen Z, and human evaluation, check out our other posts on the Surge AI blog!
- Is Google Search Deteriorating? Measuring Google's Search Quality in 2022
- Why Instagram is Losing Gen Z: We Asked 100 Users to Compare TikTok vs. Reels
- I wanted burritos. Facebook's search engine sent me to a dead restaurant 45 minutes away.
- Search Behind-the-Scenes: How Neeva Uses Human Evaluation to Measure Search Quality
- Google Search is Falling Behind
Data Labeling 2.0 for Rich, Creative AI
Superintelligent AI, meet your human teachers. Our data labeling platform is designed from the ground up to train the next generation of AI — whether it’s systems that can code in Python, summarize poetry, or detect the subtleties of toxic speech. Use our powerful data labeling workforce and tools to build the rich, human-powered datasets you need today.
