A new report from Predibase highlights emerging use cases among organizations with LLMs in production and the growing demand for customizable, open-source LLMs

Predibase, the first commercially available declarative AI platform for engineers, today released a new report, “Beyond the Buzz: A Look at Large Language Models in Production.” Based on survey data from organizations experimenting with LLMs, the report offers insight into real-world concerns, opportunities, and priorities for organizations as they embrace AI and LLMs. Among the key findings: enterprises are looking for ways to customize and deploy open-source LLMs without giving commercial vendors access to proprietary data, and they are exploring other use cases beyond generative AI capabilities.

“It is now open season for Large Language Models (LLMs). Thanks to the widespread recognition of OpenAI’s ChatGPT, businesses are in an arms race to gain a competitive edge using the latest AI capabilities. Still, they require more customized LLMs to meet domain-specific use cases,” said Piero Molino, co-founder and CEO of Predibase. “This report highlights the need for the industry to focus on the real opportunities and challenges as opposed to blindly following the hype.”

The report highlights emerging trends from LLMs in production using responses from 150 executives, data scientists, machine learning engineers, developers, and product managers at both large and small enterprises across 29 countries. Key findings include:

• Less than a quarter of enterprises are comfortable using commercial LLMs. Roughly a third (33%) cite concerns about sharing sensitive or proprietary data with commercial LLM vendors, leading to increased interest in privately hosted, open-source alternatives.
• Open-source LLMs are gaining momentum. Nearly 77% of respondents either don’t use or don’t plan to use commercial LLMs beyond prototypes in production, citing concerns about privacy, cost, and lack of customization, leading to an uptick in open-source alternatives. Meta, for example, has moved away from building closed-source LLMs like LLaMA-1, replacing it with LLaMA-2, available as open-source and free for commercial and research applications.
• While generative AI use cases remain popular, enterprises see the potential of other applications to provide business value. Information Extraction is the second most popular use case (selected by 32.6% of respondents). This involves leveraging LLMs to convert unstructured data like PDF documents or customer emails into structured tables for aggregate analytics. Next was Q&A and Search (15.2% of respondents), the brain in chatbots that provides accurate and relevant responses to user queries in real-time.
• Organizations are turning to customized LLMs to achieve more accurate and tailored results. Most teams plan to customize their LLMs by fine-tuning (32.4%) or reinforcement learning with human feedback (27%). The roadblocks team face with fine-tuning continue to be a lack of data (21%) and the overall complexity of the process like managing infrastructure (46%).

“We see clear potential to improve the outcomes of our conservation efforts using customized open-source LLMs to help our teams generate insights and learnings from our large corpus of project reports," said Dave Thau, Global Data and Technology Lead Scientist, World Wildlife Fund.

“Clearly, companies are investing in the personnel and technologies necessary to work with emerging generative AI technologies to support production-scale outcomes,” added Bradley Shimmin, Chief Analyst AI platforms, analytics, and data management at Omdia. “The trick, of course, will rest not in building these outcomes but in ensuring that they deliver consistent, secure, responsible outcomes. With an increasing desire to customize and deploy open-source models, enterprises will need to invest in operational tooling and infrastructure capable of keeping up with the rapid pace of innovation in the open-source community.”