Blog
Domain Experts are Essential for Creativity in the Algorithmic Age
How experts can unlock creativity and burst ideation bubbles with exploration-based algorithmic design
4 December 2025
Moran Lazar, Hila Lifshitz, Charles Ayoubi and Hen Emuna
Domain experts generate 11% more creative solutions when using exploration-based algorithms that surface diverse information, but perform no better than novices with standard search tools. The design of algorithms, operating invisibly beneath the surface, fundamentally determines whether experts deliver their full value for innovation.
As artificial intelligence tools become ubiquitous in workplaces, an important question has emerged: do we still need deep domain knowledge? When AI can produce sophisticated outputs in seconds, the traditional path of building expertise over years seems inefficient, perhaps even obsolete. Our research provides some new elements to answer that question. We find that domain expertise is valuable for creative problem solving in the age of algorithms, but with a critical caveat: its value depends critically on the type of algorithm experts use. We ran two experiments testing how different search algorithms affect idea generation for sustainability challenges. In a laboratory setting with 104 participants and a global field experiment with 245 creators from over 40 countries, we discovered something striking. When participants used conventional search tools like Google Search, domain experts generated ideas no more creative than complete novices. But when they used a modified search tool we designed to surface more diverse information, experts generated solutions that were 11% more creative and substantially outperformed novices. The invisible architecture of these digital tools determined whether expertise could fulfill its potential.
When Standard Search Tools Trap Experts in Familiar Thinking
Consider how you typically use Google Search. You type a query and the algorithm returns results ranked by relevance and popularity. This design makes perfect sense for many tasks. If you need to know the capital of France or find a nearby restaurant, you want the most common, validated answer. The algorithm’s job is to efficiently narrow down billions of web pages to a handful most likely to satisfy your needs. But this same efficiency becomes a liability during creative work. When generating novel solutions to complex problems, exposure to the most popular perspectives can inadvertently reinforce conventional thinking. For domain experts, this creates what we call algorithmic confirmation bias. Because experts have deep familiarity with their field’s established knowledge, they naturally gravitate toward information that aligns with their mental models. When search algorithms predominantly surface mainstream information, they amplify this tendency rather than counteracting it. Experts end up seeing their existing beliefs reflected back at them, strengthening the very cognitive patterns they need to escape for creative breakthroughs. In our experiments, we saw this play out clearly. Sustainability experts using standard exploitation-based search algorithms generated ideas that clustered tightly around conventional approaches. They proposed familiar solutions such as improved food labeling, consumer education campaigns, and donation programs. These ideas were competent but rarely surprising. The algorithm had effectively channeled their thinking along well-worn paths. Meanwhile, novices using the same tools produced similarly conventional ideas. Without deep knowledge to draw on, they simply absorbed the mainstream information the algorithm presented. The playing field between experts and novices was surprisingly level, but level at a modest altitude of creativity.
Designing Algorithms to Enable Rather Than Constrain Creative Thinking
What if we could redesign search algorithms specifically for creative work? We built a tool called XYZ to test this possibility. Instead of ranking results purely by relevance and popularity, XYZ uses natural language processing to identify distinct conceptual clusters within search results, then deliberately surfaces options from across these different clusters on the first page. A user searching for “reducing household food waste” might see results about composting technology, behavioral nudges, supply chain logistics, cultural practices in low-waste societies, and economic incentives, all presented together rather than filtered down to the single most popular category.
This design reverses the typical algorithmic approach. Rather than narrowing and filtering to show users what they most likely want, it broadens and diversifies to show users what they might not expect but could potentially use. The familiar search interface remains, but the underlying logic shifts from exploitation to exploration. We believe the principle has wide applicability: when the task is creative rather than informational, algorithms should be optimized for diversity rather than consensus. Our field experiment put this to the test with a real-world innovation challenge. We partnered with sustainability organizations, incubators, and Freelancer to create a competition engaging hundreds of creators to develop solutions for household food waste. Participants came from more than 40 countries and ranged from sustainability professionals to passionate amateurs. Each creator used either a standard exploitation-based search tool or our exploratory XYZ tool before submitting their idea. Independent expert judges, blind to which tool participants had used, evaluated all submissions for creativity.
The pattern that emerged was unambiguous. With standard search, experts and novices performed similarly. With exploratory search, novices improved slightly, but experts excelled dramatically, generating ideas judged 11% more creative on average. The gap between experts and novices widened substantially. Expertise finally mattered, but only when the algorithmic environment supported it.
Why did exploration-based algorithms unlock expert creativity? We found that domain expertise enables what we call recombinant innovation. When presented with diverse, unfamiliar information, experts can identify unexpected connections between distant concepts because they understand the deeper principles underlying each domain. One expert in our study combined insights about community sharing platforms, smart home sensors, and behavioral psychology to propose an automated neighborhood food rescue network. This synthesis required recognizing that food waste is simultaneously a coordination problem, a technology problem, and a behavioral problem, then seeing how solutions from each domain could complement each other. Novices exposed to the same diverse information lacked the conceptual foundation to make these leaps.
From Individual to Organizational Creativity
The implications extend beyond individual idea generation. We analyzed the semantic content of all ideas submitted in our field experiment, using natural language processing to group conceptually similar proposals. When novices used the exploitation-based search tool, their ideas clustered into a single dominant grouping. When experts used this tool, we saw two distinct clusters emerge. When novices used XYZ, they also produced two clusters. But when experts used XYZ, the ideas spread across five distinct clusters, each representing a fundamentally different approach to the problem. This matters because innovation at the organizational or societal level depends on cognitive diversity. When everyone generates similar ideas, even if each idea is individually decent, the collective portfolio lacks the variety needed to address complex challenges. We call these convergent groupings ideation bubbles. Breaking these bubbles requires both the diverse information that exploratory algorithms provide and the recombinant capabilities that domain experts possess. If domain experts use exploratory tools, they pioneer distinct solution territories. One expert might focus on circular economy redesign, another on behavior change interventions, another on technology-enabled transparency, and another on incentive restructuring. The organization gains a portfolio of genuinely different approaches rather than variations on a theme.
What This Means for Practice and Research
These findings have direct implications for how organizations can approach innovation. Standard search engines and the increasingly used generative AI assistants optimized for efficiency work well for implementation tasks, but for early-stage innovation, tools optimized for exploration may be more valuable. Organizations can develop explicit guidance about which tools to use when, recognizing that algorithmic design choices have real consequences for creative output. With generative AI, this means adjusting parameters like temperature settings for creative tasks and crafting prompts that explicitly request diverse, unconventional information rather than consensus solutions. The deeper implication concerns how expertise itself functions in the age of algorithms. Our findings challenge the notion that AI diminishes the value of specialized knowledge. Instead, as algorithms become more powerful, expertise becomes more valuable but in a transformed way. The role of experts shifts from being the sole source of knowledge to being the synthesizers and recombinators of knowledge that algorithms help surface. Organizations that invest in both deep expertise and the algorithmic infrastructure that allows that expertise to flourish will hold the competitive advantage.
Our research also demonstrates the value of experimental approaches that build practical interventions. By creating XYZ as a working tool, we could test our ideas in realistic conditions and generate findings with immediate applicability. The field experiment illustrates how partnerships with institutional and business actors enable research that is both rigorous and relevant. Working with sustainability organizations, incubators, and a freelance platform allowed us to engage hundreds of real creators tackling an actual challenge, producing insights grounded in practice. This collaborative model, where researchers design interventions and organizations provide access to authentic innovation contexts, offers a pathway for generating knowledge that advances both theory and real-world impact. When experiments unfold in actual organizational settings rather than artificial laboratories, and when the tools we test can be freely deployed by practitioners, the resulting insights have immediate pathways to application.
About the Research
This research was published in the Academy of Management Journal and involved two complementary studies: a controlled laboratory experiment with 104 participants generating ideas for reducing overconsumption, and a global field experiment with 245 creators from over 40 countries tackling household food waste reduction. The field study was conducted in collaboration with leading sustainability organizations, incubators, and businesses.
Citation: Lazar, M., Lifshitz-Assaf, H., Ayoubi, C., & Emuna, H. (2025). Would Archimedes Shout “Eureka” with Algorithms? The Hidden Hand of Algorithmic Design in Idea Generation, the Creation of Ideation Bubbles, and How Experts Can Burst Them. Academy of Management Journal, 68(5). https://doi.org/10.5465/amj.2023.1307