“Research in seconds, not weeks!” “Democratization of design!” Promises like these suggest that experience designers and researchers might soon find themselves out of a job. These magical AI tools claim to make the time-consuming, expensive, and often messy design process cheap, fast, and effortless.

I primarily work on B2B solutions with highly specific requirements, often for companies with limited knowledge of the design process and users who aren’t particularly tech-savvy. The workflows I’m tasked with optimising are highly complex, and the interfaces must support them while making the user experience as seamless as possible.

Against this backdrop, I have to admit — I was skeptical that these magical AI tools could make a big difference in my work. So, I decided to put them to the test with one of my recent projects.

(Disclaimer: Due to various privacy and corporate policies, my use of AI tools is heavily restricted. So in some cases I rather pretended I was working on a typical project and used mock data.)

UX design tools aren’t exactly rare — they’re practically spilling over. If you look at the image below, listing every UX-relevant tool, you’ll get a sense of just how overwhelming the landscape is. Most of these tools are either enhanced by AI or have AI as their central feature.

The goal of my sample project was to improve the overall UX of an internal web application for a multinational corporation. I was the sole designer responsible for the entire UX process (which is quite typical in the B2B world, where design is still often seen as an afterthought).

Here’s how AI helped — or didn’t help — at each step of the process:

One of the first and most important steps in any project is to understand the topic at hand, its terminology and its concepts. Here, AI was fantastic! I primarily used ChatGPT to help me quickly grasp complex concepts. With the help of AI, I could quickly get an initial understanding of the main tasks users typically perform with this type of application and the steps involved in completing them.

Without AI, gaining this level of understanding would have been far more time-consuming and difficult.

🤩 My verdict: Extremely helpful!

If you’ve already defined your research goals and chosen a format, tools like ChatGPT can produce incredibly helpful results. Of course, the interview guide needs to be reviewed and fine-tuned, but it’s a real time-saver compared to starting from scratch.

🤩 My verdict: Extremely helpful!

There are plenty of tools for AI-supported interviews, meaning an AI is actually conducting the interview or the testing, which sounded quite impressive to me. Although I couldn’t use them for my project, I ran openly available demos with Outset.ai and Listenlabs.ai.

The demo by Outset did not really convince me, because the experience felt a bit bumpy: Questions were written on the screen, to answer you first needed to push a button — so it did not feel like a conversation at all. The experience with Listenlabs on the other hand really blew my mind! The AI actually talks with you, making the interview feel natural and pleasant. You can try out different formats right on their homepage. (Tip: Their AI Personality Test is worth checking out — you’ll be surprised! And no, I’m not being paid to say this. 😊)

I see huge potential in AI-supported interviews. That said, I wonder how participants would feel about being interviewed by AI. Would they drift less since there’s no need to build a relationship with a machine? Or would they bail the moment they lose interest — after all, it’s not like they’d offend a real person.

I’m excited to see how these tools perform in real-world scenarios and how their overall usage will develop.

😃 My verdict: Lots of potential!

Even though I haven’t had any hands-on experience with synthetic research yet, I want to briefly touch on the topic. (Of course, I’ve used Chat GPT to impersonate a specific persona when answering certain questions. But I’ve never really considered this real research — more like a quick way to get an initial sense of a topic, especially in the early stages of understanding.)

What is synthetic research? It’s research conducted with AI-generated users. With tools like Synthetic Users, you just provide information about the user group, research goal, and interview format, and in return get persona-like profiles along with corresponding interview transcripts.

The topic has sparked heated debates in the UX community and could easily warrant an entire article of its own. The experts at NN Group have written a highly recommended piece on this.

🤔 My verdict: I can only agree with the NN Group experts’ conclusion that “If you’re using synthetic users in your research process, they should complement, not replace, real research.” And their major concern is spot on: “If stakeholders feel that they’re getting some insight for a tiny investment, justifying the need for investing in real research may become difficult.”

Because let’s be honest — securing funding for UX research has always been hard. And with this faster, cheaper alternative, the temptation for stakeholders to take the easy path will only grow.

Sadly, I don’t currently have access to a dedicated research repository. Tools like Dovetail — which I’ve used in the past — are amazing, even without AI features. And I’m sure their AI capabilities have improved a lot since then. Looppanel is another tool often mentioned, though I haven’t tried it myself.

For my test, I relied on ChatGPT (many UX tools use OpenAI’s model under the hood anyway). I transcribed the interviews with MS Teams, manually removed sensitive data, and then fed the transcripts to ChatGPT for analysis. It did a decent job identifying recurring themes and major problem areas.

I have to add that these interviews weren’t easy: The interviewees kept getting lost in details, making it difficult to steer them back on track — even for an experienced researcher, a real challenge! And let’s be honest: this kind of situation is more the rule than the exception.

This is exactly where it becomes crucial to not just listen to what users say, but also to observe what they do, read between the lines, and take a lot of contextual knowledge into account — things like company structures and goals, technical background information, or even team dynamics.

In summary, using AI to analyse the transcripts provided a solid first overview of recurring patterns. However, extracting the truly relevant insights still required my own observations, interpretation of what was said, and a good deal of background knowledge.

🙂 My verdict: Great for a first overview, but the heavy lifting still falls on you.

Once I had identified the key insights, I asked ChatGPT to draft a research report for the product owner. Similar to creating the interview guide, the AI provided a helpful starting point. But the real work — crafting a clear, compelling narrative with actionable recommendations — was still on me.

That said, ChatGPT was a great brainstorming partner when I got stuck.

🙂 My verdict: Useful for structure and fresh ideas.

I keep hearing about how AI design tools can generate clickable prototypes in seconds, supposedly democratising design (à la “Everybody can be a designer”).

First of all, to all non-designers reading this: design is not just about making things look pretty. Every visual decision is based on data gathered through research (or at least it should be). Can we all agree to retire the stereotype of designers as mere pixel pushers? Thanks.

This said, I tried to somehow feed the AI with first ideas derived from my research results. I tested three tools: Galileo, Uizard, and UX Pilot (web version).

The results? Disappointing. UX Pilot performed best, while Galileo was the weakest. Sure, I used free versions, and I don’t have a degree in prompt engineering (I have read the Galileo Prompt Guide and the UX Pilot Handbook though 🤓). But honestly, explaining my requirements to the AI would’ve taken longer than building a prototype myself (what I did).

AI loves patterns. So, for familiar interfaces like e-commerce sites, dating apps or product landing pages, these tools probably provide interesting results. But for unique, complex requirements, they fall short.

(And let’s be real: many AI-generated designs look polished at first glance but make no sense upon closer inspection.)

😒 My verdict: Not very helpful for complex applications. If we reach a point where research insights feed directly into AI design tools, things could get really interesting, though.

Expertise is still essential
AI is great for providing an initial starting point, and in many cases, it can help overcome the fear of the blank page. But to give effective instructions and to verify the results, experienced experts are still necessary — at least if you want to base decisions on reliable, high-quality data. AI tools simply aren’t advanced enough yet to be fully trusted with critical tasks. In many cases, it’s actually more efficient to just do the work yourself. The 2024 AI in UX Research Report comes to the same conclusion, stating that “the primary benefit of AI — speed — is slowed by the need to ensure its accuracy.”

Good prompting is key
This might seem obvious, but I don’t think it is. It’s not that easy to articulate your ideas or vision for an interface or a concept in written language and then ask highly specific questions about it! This presents two challenges at once:

1. You need to know exactly what you want to find out or create.
2. You need to phrase it in a way the AI can actually understand.

And here we come full circle — experience and expertise matter. Because only someone who truly knows what they’re doing can give clear and effective instructions and ask the right questions.

AI is here to stay
I know you hear this everywhere, but I have to agree: this isn’t just a passing trend. So buckle up and get ready! BUT — don’t just blindly use AI for everything out of fear of being left behind. It’s not a magic solution that works on its own. Developing your own expertise, learning new skills, and — most importantly — using your own brain is still essential.

As magical as these tools may seem, at the end of the day, they are just that — tools. And tools are only as good as the hands that wield them.

I believe that if there is the opportunity to embed the UX design process within an in-house LLM that has all the company’s knowledge and background information (e.g. about the industry, the market and relevant competitors etc.), many exciting possibilities will arise. This is because, in such a case, the model would already know the “bigger picture” and could potentially make connections and generate ideas that are much more interesting than those derived from isolated queries.

Harnessing the full potential of AI and integrating it purposefully into company structures — in a way that people can use it effectively and, ideally, enjoy doing so — is just one area that is opening up to us experience designers with the ongoing growth of AI technologies. Because no matter how sophisticated the technological solution may be, at some point, there are always people who need to interact with it.

🧠 🧠 🧠 P.S.: This article originates from an organic brain 🧠 🧠 🧠
(AI was only used for translation and for generating an image)