Value in the Time of LLMs
You may be experiencing a similar problem that Florentino Ariza had. Florentino was in love with Fermina Daza, but Fermina’s father did not approve. It was not until much later when Fermina’s husband, Dr. Urbino, who was approved by her father, died and Fermina and Florentino were finally able to take their voyage down the Magdalena River. Perhaps you are not dealing with decades of heartbreak (or maybe you are, I do not know your life) but you are dealing with a question about how to use large language models (LLMs) in your organizations. Not particularly dissimilar from Florentino’s predicament, your boss or your organization has forbidden the potential presented by LLMs for any number of reasons. Maybe you are in love with the potential that LLMs bring but are not allowed to take your own voyage down the Magdalena River toward new value. Do not worry. You need not wait decades like Florentino did. You can begin your life journey anytime with custom LLMs that are developed for your specific use case.
LLMs have become synonymous with the big names such as ChatGPT or Bard (or Gemini or whatever Google’s marketing team rebranded it as). They are also well understood to hallucinate and provide potentially inaccurate responses to queries. This is true, and any output of LLMs should be checked by a human. LLMs are not, and are unlikely to become, “human out of the loop” systems. The value of LLMs is an interface through which humans interact with computers and other forms of artificial intelligence (AI). Expecting perfect results from an LLM after a single query is not reality—just ask the New York lawyers who were sanctioned for using ChatGPT for a court filing only to have it hallucinate precedent cases. LLMs are excellent at augmenting humans, not replacing them.
With that out of the way, what about using LLMs in an organization that has sensitive data? Perhaps you deal with regulated financial data or law enforcement sensitive information or medical information. Are you prepared to send your sensitive data to an open cloud and use an LLM you do not have control over? Depending on your data, you could not even if you wanted to. Fermina’s father foils us again.
However, the use of LLMs does not have to be synonymous with the big names and it is possible to securely develop your own LLMs in house that do not connect to the cloud and can operate securely on an air gapped laptop. You can train them on specific sets of data rather than the entire internet and reduce the chances of hallucinations. The data can stay internal and not have to go to a cloud, nor even be connected to the internet if you desire. So, under what circumstances should you consider developing a custom LLM?
- Define your business problem and how an LLM can help create value. It may be in time saved, accuracy, reduction in force, or in customer service. In any case, make sure the problem you are trying to solve can be solved by an LLM.
- Inventory your data and your data policies. Do you have data to take special precautions with? Does your data require audits to your leadership or third parties? If so, a cloud based LLM is out of the question for you, but an in-house custom LLM is a great path to pursue.
- Evaluate your workforce. Does your workforce understand the capabilities and limitations of LLMs? Ensure they understand your vision of LLMs as an augmentation to human expertise, not a replacement for it.
- How quickly do you need to move? Hiring and creating a custom LLM development shop in house is expensive and likely to take a long time. Consider whether you have the resources and desire to create this capability or if it makes sense to partner with an AI firm that specializes in model creation, technology governance, data automation, and other important components of a successful AI program.
- Security, Compliance, Regulation. Not all data can be transmitted to a cloud that is not under your control via a third party. If you have specific security, regulatory, and/or compliance concerns, LLMs should not be a pipedream for you. Regulatory environments change so the tools you invest in need to be ready for new developments. The cloud is not a necessity anymore and you can still use the best tools available.
It is painful going through life with something missing. It is even worse when that missing piece translates into lost value. Cloud based LLMs have already changed how humans interact with AI and computers in general, but the future of LLMs is customization. There will always be large LLMs trained on the entire internet, and they will continue to entertain us with sonnets about the joys of eating In-N-Out Burger and frustrate professors as students try to shortcut assignments. But the real value is in targeted use cases where focus can be applied to specific problems, trained on specific data sets and unplugged from the cloud.
This is your Fermina and your ticket to a dreamy boat trip down the Magdalena toward new value.