How GenAI is changing the way consumers behave online

I’m desperate to go on holiday somewhere hot and sunny and I don’t have time to plan my own itinerary. No biggie, I’ll just ask ChatGPT. Sure enough: as if by magic, and almost instantly, everything I need appears on my screen.

Three years ago, I’d have used a search engine like Google or Bing, entering a few keywords – “winter sun holidays” and in return getting a list of tour companies’ websites and reviews-based sites such as TripAdvisor – before clicking on the results (the “search-and-click” model). If I wanted a more personalised itinerary, I’d probably have followed up with a call. It all seems so cumbersome now.

Using a chatbot is more like asking a friend – “Where can I go and sunbathe in the morning, get my cultural fix in the afternoon and eat great food at night? Need your advice.” It’s a conversational experience, and the answer you get is often complete in itself.

Will traditional searches decrease in number now we have GenAI? And will people browse fewer websites as a result? According to new research by Anja Lambrecht, Professor of Marketing at LBS, and Nicolas Padilla, Assistant Professor of Marketing at LBS, the answers are yes and yes, sizeably. These are just two findings in their paper 'The impact of LLM adoption on online user behavior', authored with two academics from UCLA, H. Tai Lam, Assistant Professor of Marketing and Brett Hollenbeck, Associate Professor of Marketing.

Are LLMs substituting for traditional search or serving as a complementary tool?

Large Language Models (LLMs) are a kind of generative artificial intelligence (GenAI); “large” because they learn to read and write human by digesting immense datasets, “generative” because they can create original content. The researchers explored how LLMs have in a very short time transformed how we use the internet, and what this means for content providers and the broader web ecosystem: search engines, online publishers and advertisers.

“If users shift queries away from traditional search engines that direct traffic to a wide range of websites toward LLMs that provide synthesised answers and fewer pathways to external content, the underlying revenue model for content creators may be jeopardised,” they explain. “These shifts also raise competitive questions, as LLMs increasingly operate as alternative gateways to information that challenge the central role long held by search engines.”

At the same time, LLMs may serve as a complementary tool, they suggest, fulfilling a distinct function that is complementary to traditional search and online content consumption: “For example, LLMs excel at understanding complex queries and pre-structuring broad sets of information in an accessible way, but at times struggle to reliably providing specific factual details or credible primary sources. This balance of strengths and weaknesses could result in their adoption, facilitating more efficient initial search, which then results in a greater overall search volume and visits to publisher websites.”

Becoming aware of shifts in consumer behaviour and being able to base that awareness on firm evidence is vital for businesses seeking to get their products and services noticed above the competition. Understanding whether LLMs substitute for or complement traditional online search and content consumption is important for evaluating their impact.

LLM adoption: three areas to investigate

The research paper investigates three aspects of the adoption of LLMs:

1. How does it affect online search? Although LLMs are able to answer questions, which might suggest substitution, concerns around the quality of those answers could mean people still continue to search in other ways as well, potentially even increasing the overall search volume.

2. When does LLM use substitute for broader online activity, and when doesn’t it – and what is the impact on web publishers? The researchers evaluated the impact on adopters’ online traffic overall, for smaller vs larger websites, and for two distinct content categories: educational websites and user-generated content sites.

3. Does LLM adoption affect users’ advertising exposure? If so, that might shift web publishers’ ability to monetise content, and retailers’ ability to reach consumers. Till now, brands have typically invested in a combination of search ads, which are triggered by particular keywords in a search, and display ads, which show up when a person is online but not necessarily searching, based on their demographic, browsing habits and interests.

To tackle these questions, the researchers analysed a huge set of clickstream data – almost 1.2 million url-level data from desktop browsers from late 2022 through to mid-December 2023 (OpenAI launched Chat GPT, the first user-friendly chatbot, on 30 November 2022). They narrowed this down to the online usage of 2014 American households that had used LLMs for three consecutive weeks.

Firstly, they looked at search, the entry point to online browsing for most users. As anticipated, the number of traditional searches dropped – slowly at first, as users got acquainted with the new technology – and by 20 weeks in, by more than 20%. The decline affected question-based searches but not navigational searches (i.e., when someone is looking for something specific, such as a brand name).

Then, they examined how this had affected website traffic. Frequently visited websites were not affected, but smaller websites suffered a significant drop in business. The researchers point out that the results suggest large, well-established sites remained relatively insulated, but at least some of the smaller sites, which are most dependent on search-driven referrals, lose out.

The paper’s other big finding is that the number of display ad exposures dropped, especially to consumers who were making a lot of online purchases: “Substitution away from direct website visits has implications for monetisation. Display ad exposures fall significantly after LLM adoption, especially among users with high levels of retail activity (the people whose rich browsing activity provides more data for advertisers, who often retarget them based on what they’ve searched for previously). In contrast, search ad exposures do not decline significantly, because navigational searches that are more likely to trigger paid search placements haven’t been affected.

What does this mean for publishers, which need to monetise the content they create? If users are now less likely to bother visiting smaller websites, is it worth it for a company that probably has a limited budget to invest in producing brilliant content? The trend towards getting answers elsewhere is disincentivising, but at the same time it might provide an incentive for websites to publish high-quality differentiated content that has the ability to more directly attract users.

Anja and her colleagues also looked at two very different kinds of websites: education-related and those publishing user-generated content. Students have been quick to adopt LLMs; a lot of educational tasks such as making summaries and drafting essays are directly automatable. The researchers found that LLMs serve as a strong substitute for a broad range of education-related activities, and could negatively impact many online education content providers.

The user-generated content sites included Stack Overflow, a knowledge-sharing platform that students and developers use to ask questions about coding (to give one of the simpler examples: “How do I add an extra item to a list in Python?”). The researchers found people were making fewer visits to Stack Overflow now they could use GenAI to get their answers. In contrast, the number of visits to Wikipedia and Reddit were unaffected. The adoption of LLMs also made no difference to the traffic to email, retail and news sites.

GenAI has evolved and improved since the time period assessed in this study. The researchers suggest that as LLM capabilities expand, the behavioural and economic forces they identified will persist and potentially intensify. What are the implications for content creators, GenAI firms and policymakers?

Implications for web publishers, GenAI firms and advertising

• With LLMs drawing on website content without generating much traffic, content providers will find it harder to make money from advertising or subscriptions. Faced with this, web publishers may need to invest into high-quality differentiated content that has the power to directly attract consumers and potentially allows them to charge LLMs for access to their content.

• Alternatively, web publishers can consider how to adjust their revenue models more broadly. For example, Stack Overflow has pivoted to using their accumulated Q&A database for Enterprise SaaS solutions while also licensing their data, including for AI training purposes. Notwithstanding, the researchers expect that at least some web content providers – likely those producing low quality imitable content – will find it difficult to adjust to the weaker monetary incentives for producing online content.

• GenAI firms will continue to rely on high-quality content, both for training LLMs and for retrieval-augmented generation. But online content providers will only be incentivised to continue producing this content if they can monetise it effectively. Previous approaches to monetising content, such as putting subscription-only content behind paywalls or “freemium” models, have worked up to a point but there has been a trade-off in terms of the number of people willing to pay. Multiple different revenue streams may be at risk.

• New advertising markets are emerging. If LLMs become the main route for people to access information then advertising funds may shift towards AI platforms which may further hurt existing web content providers. And as the boundaries between search engines and LLMs continue to blur, both incumbents and AI entrants face strategic pressure to differentiate on features, accuracy, live updates and user experience.

The researchers note: “A large share of the digital economy depends on the steady production of high-quality online content, often by smaller or specialised creators. If traffic and monetisation decline due to substitution by LLMs, the incentives to produce future content might weaken, with implications for information diversity and the long-run health of the web ecosystem.”

That said, LLMs clearly provide significant value to consumers, for example in that search for a holiday destination, and the example of Stack Overflow demonstrates how innovation in consumers accessing content can drive broader business model innovation. It will be exciting to monitor how LLMs together with content publishers shape the digital landscape of the future.