Taking Search to the Next Level – ChatGPT, Bard, and Others

ChatGPT, Bard, Minerva, and others, a class of tools called Large Language Models (LLMs), are poised to revolutionize how we search the internet. In the simplest terms possible, LLMs are software programs trained on vast (think Carl Sagan’s billions and billions) amounts of text-based information with the goal of being able to respond using human-like language. Those datasets include fiction and non-fiction books, website content, court filings, blogs, social media posts, chat logs, and other documents that can be compiled and accessed repeatedly. To hold conversations, LLMs are trained to respond using natural language – or to write as a human might write. This chat experience isn’t all that new – the first recognized chatbot, E.L.I.Z.A, was introduced in 1966, and it’s not unusual to encounter a chatbot as part of most e-commerce sites.

What then is the amazing leap that LLMs represent? Capability, speed, and agility are three attributes that are often recognized as part of the LLM advancement. Each release is an exponential improvement on its previous version. ChatGPT-4, released less than six months after ChatGPT 3.5, can now process video and images, and without getting too technical, has added the equivalent of approximately 22,000 new words (https://www.moveworks.com/insights/what-is-gpt-4-and-why-is-it-better-than-gpt-3). There have also been improvements in complex problem-solving and creativity, leading one analyst to quantify the updated ChatGPT as “ten times more advanced” (https://www.searchenginejournal.com/gpt-4-vs-gpt-3-5/482463/#close).

Their faster-than-humanly-possible speed allows them to respond to queries with impressive speed. When a user provides ChatGPT, as an example, with a request to create an abstract on eDiscovery written as if the author were Dr. Seuss, in less than two minutes it returns:

Oh, electronic discovery, what a task it can be
With so much data to find, it’s like searching the sea
From emails to documents, it’s all there to see
But how can we manage it, let’s you and me

First, we must identify where to start
With key custodians and sources to take part
To avoid endless searching, and work smart
And find what we need, from the very start

Next, we must preserve the data with care
To avoid tampering and any despair
With legal holds, we can do our share
And protect it all, with a forensic flair

Then, we can collect the data, oh so fine
With methods that capture it, in its prime
With metadata intact, and nothing left behind
So we can analyze it, and make up our mind

Now it’s time to review, with a keen eye
To separate what’s relevant, from what’s a lie
With analytics and tools, we can give it a try
And cull it down, to a size that’s right

Next, we produce it, in a format that’s clear
With organization and structure, to lessen any fear
So it’s easy to find, and easy to hear
And nothing’s lost, in the e-discovery sphere

Finally, we present it, with a story to tell
With charts, graphs, and timelines to sell
To show what’s important, and what’s not so well
And make it all clear, like a ringing bell

So that’s the story of electronic discovery
With Dr. Seuss style, and a little bit of discovery
Remember these steps, and you’ll do just fine
In the world of e-discovery, it’s all about the shine!

Despite all the impressive results, there are concerns about LLMs. The most notable is that LLMs can “hallucinate” responses. While ChatGPT reports that it cannot deliberately produce a false answer, it can make errors or provide incomplete answers based on gaps in the training data or the complexity of the question. How this flaw shows up can include accrediting the wrong degree or university affiliation to a specific person or providing a link to a website that no longer exists. OpenAI’s own testing demonstrated that the more complex the process or the longer the conversation in which ChatGPT was engaged, the more likely it was to produce erroneous responses.

Perhaps more worrisome, technology tools are only as immune to the nefarious motivations of their users as their developers anticipate. Incompletely trained LLMs can be prompted to provide harmful, dangerous, and offensive responses, all of which have garnered media attention, and prove that for all the innovation language-based models present, they’re still developed by humans, at least for now.