AI in search experience and knowledge building

These thoughts are from my article on UX Matters and were published on December 2023 and largely inspired by our work with Polaris Media.

As a UX designer, navigating complex user needs relating to information-seeking and knowledge-building tasks can be daunting. However, by employing artificial intelligence (AI), in either its traditional or generative form, UX designers can create experiences that significantly improve the way people find and make sense of information online. But designers must give extra care when transitioning to an AI-augmented search experience.

To deliver a successful design, building user trust in the AI itself, avoiding the abrupt disruption of habitual search patterns, and maintaining the right degree of user control are all essential factors to consider. With the right technical knowledge and an unflinching focus on human value, designers can leverage AI to build real knowledge systems that truly empower their human users.

Understanding How AI Can Improve Users’ Knowledge-Building Processes

AI-based applications can help users find the knowledge they need quickly and efficiently.

AI is best known for automating processes and tasks and, since the advent of generative AI, for creating outcomes that human agents can then leverage for a variety of purposes. In combination, these two capabilities can be incredibly significant for the knowledge-building problem space, in which most of users’ painpoints typically center around two key experiences:

• retrieval — finding relevant information sources such as documents
• sensemaking — extracting valuable information and insights from these information sources

Whether by automating the tedious process of retrieving information from extensive collections of documents, diverse data points, and other information sources or by deploying generative AI to summarize and make disparate information more digestible, AI-based applications can help users find the knowledge they need quickly and efficiently. When any enterprise implements such systems at scale, this can contribute to building a comprehensive knowledge-base for the enterprise.

Challenges of Designing AI-Based Applications

Typical AI outputs are subject to frequent changes because these systems are both dynamic and constantly learning from users’ inputs. They perfect their results over time and are probabilistic because they make decisions based on probabilities and statistical reasoning. Thus, users’ inputs could potentially produce different outputs each time.

With such probabilism and variability comes a design challenge: an AI-based application must accommodate a user experience that is characterized by change, and this change is based on these different user outcomes.

Considering Users’ Current Search Habits

When the goal is shortcutting time-consuming research, cutting to the chase, and surfacing only the most relevant information, that is precisely where AI shines.

When your goal is to improve any information-seeking experience, the first thing to consider is how users actually perform their searches. Do they customarily carry out comprehensive research, checking each and every result? When they discover an information source — for example, a long document — do they read it in its entirety? Are they seeking information in the way they would for their job, to comply with some corporate rule, or just because of a long-established way of working? Whatever the reason, understanding users’ needs requires careful investigation up front.

When the goal is shortcutting time-consuming research, cutting to the chase, and surfacing only the most relevant information, that is precisely where AI shines. However, disrupting a search experience too abruptly could confuse users, particularly when an AI system is new and still in its learning phase. Trust issues might creep in at this stage, leaving users in doubt whether the AI is providing the most relevant information.

Designing a Collaborative Search Experience

The purpose of the AI system is to help augment the user’s capabilities. At the same time, the AI system requires the best possible conditions to perform its job smoothly and efficiently.

Once you’ve thoroughly examined your users’ behavior, you should be able to create a great user experience for your AI-based app.

The key idea here is collaboration between user and the AI system. You need to strike a balance between human and machine agency. It is important to remember that the purpose of the AI system is to help augment the user’s capabilities. At the same time, the AI system requires the best possible conditions to perform its job smoothly and efficiently.

With that said, there are four important user-experience factors to consider when designing an AI-based, knowledge-building app, as follows:

1. Conversational search
2. Different options for exploring search results
3. Using generative AI to aid information sensemaking
4. Explaining the outcomes of artificial intelligence

1. Conversational Search

Large language models are introducing a new paradigm for looking up knowledge. In comparison to traditional enterprise and consumer search engines, which rely solely on keywords and human feedback, applications that deploy generative AI can produce human-like, more accurate responses. Plus, depending on users’ observed behaviors, you could consider implementing conversational features that would make the search experience more immediate and accessible.

2. Different Options for Exploring Search Results

Human knowledge-building processes are profoundly complex and inherently subjective, so different users might evaluate the relevance of retrieved information in different ways.

Display the search results that the AI deems most relevant to the intent of a users’ search in a position of prominence. After all, the purpose of implementing AI is for a machine to do the tedious legwork of perusing data sources and selecting the right content on the human agent’s behalf. Therefore, the user experience should emphasize this added value.

Be careful though! If your users have hitherto engaged in exhaustive searches — that is, if they habitually check all the search results, leaving no stone unturned — give them the option to continue to do just that. You can achieve this goal in various ways, for example:

• Add a relevance score for each search result. This allows users to evaluate and compare different results.
• Provide a filter that lets users widen or narrow the results depending on their relevance. This is very important in situations where users need to connect the dots across different types of information, but the model has not yet been trained to do that.

Human knowledge-building processes are profoundly complex and inherently subjective, so different users might evaluate the relevance of retrieved information in different ways. Knowledge that is important to one user might be buried in sources that a model overlooks. This is an inherent drawback of a probabilistic context.

Another way of supporting multiple information-seeking patterns in a user interface is to use different widgets to display specific AI outcomes. For instance, content cards could highlight the top results, while lists or tables of search results could support exhaustive search patterns. Figure 1 provides examples of different user-interface layouts that can accommodate changing information-seeking behaviors, when transitioning to an AI-based application. Tables support exhaustive searches for information, while cards highlight the system’s recommended content. The transitional layout can help users gain confidence with a new user experience.

Create a unified environment such as a dashboard, in which users can feel in control and decide by themselves how they want to navigate and act upon the results that the AI provides.

3. Using Generative AI to Aid Information Sensemaking

Generative-AI capabilities [can] create useful content such as summaries of longer documents or self-explanatory visualizations that make data and numbers clearer and more accessible.

Just locating a document or any other knowledge source can take users only so far if they cannot decipher or process its content within a reasonable amount of time. UX designers could leverage generative-AI capabilities to create useful content such as summaries of longer documents or self-explanatory visualizations that make data and numbers clearer and more accessible.

4. Explaining the Outcomes of Artificial Intelligence

As AI occupies more and more space in our lives, issues around user trust, compliance, and ethics are multiplying. For users, knowledge-building processes are almost always the starting point in making decisions — and for the enterprises for which they work. In some cases, these decisions can have a significant impact on people’s lives. Although implementing consistent AI-governance practices is crucial to preventing bias, model drifts, AI hallucinations, and other undesirable outcomes, UX designers can support the appropriate use of AI outcomes by including explainability affordances in a user interface. These could be anything from brief text that explains why a particular information source is relevant to a more detailed, page-long explanation of how the model has made its selections.

There is no one-size-fits all approach to explainability. You must consider your target audience and their needs around explainability, and thus, what level of detail you should provide. By incorporating explainability in the design of an AI-based system, designers can promote trust in the system and add to the user’s sense of agency and better control.

Final Thoughts

The inherently dynamic and probabilistic nature of AI-based systems makes their user interfaces seem a bit like works in progress.

AI-powered systems are becoming ubiquitous for both enterprise and consumer use cases that involve knowledge-building processes. By combining traditional automation with the latest generative-AI capabilities, knowledge-building systems can help users effectively find and make sense of information. However, for an enterprise, creating these systems can be costly. To ensure their success, your responsibility as a UX designer is to find ways of fostering their adoption and ensuring their correct usage by users.

But don’t worry about the difficulty of getting everything right the first time. The inherently dynamic and probabilistic nature of AI-based systems makes their user interfaces seem a bit like works in progress. Just remember, by creating a collaborative user experience that lets people can make the most of the AI’s outcomes without losing their agency and control over the process, you can make the most of the power of this technology, without compromising users’ trust in AI-based systems.

Reference

IBM. IBM Design for AI. Retrieved November 27, 2023.