Laying the Data Foundations of Your Organization’s GenAI Journey

AI has already become an integral part of our daily lives, enhancing our experiences through digital voice assistants, personalized movie recommendations and a plethora of other applications. However, the recent advancements in generative AI (GenAI) applications, such as ChatGPT, has triggered a profound shift in the way we interact with and harness the power of AI. 

Now, AI is more conversant and readily available for anyone to use, empowering individuals and organizations alike to create compelling text and images from just a few inputs. This democratization of AI presents a unique opportunity for businesses to deliver unprecedented value to clients. Our research indicates that GenAI has the potential to unlock productivity gains — sometimes up to 50% improvements for straightforward tasks.

While both individuals and organizations can benefit from GenAI, the key differences at an organizational level lie in data, scale, purpose and compliance. Many organizations have been eager to embrace this revolutionary technology, but unfortunately lack the necessary components to start the process. The following elements are critical considerations for an organization embarking on its GenAI journey.

Establish a Well-Governed Enterprise Data Platform

To set the stage for success, you should prioritize establishing a robust and well-governed enterprise data platform. This foundational step ensures that you can scale effectively for the growing data needs of AI. Data lineage and quality must also be managed effectively to be secure and readily accessible to enable AI-driven insights and applications. 

One essential benefit of GenAI is massively improving an organization’s ability to condense, annotate and categorize unstructured documents and data sources. Modern data platforms, like the Databricks Lakehouse, have a major role to play in unifying structurally different datasets in the same platform to create a single version of truth for enterprise data. Effective governance helps organizations maintain and share enterprise-grade data, thereby enabling your business teams to build analytics and AI solutions at scale. 

Due to the inherent cybersecurity risks with large amounts of data in use, it is extremely important to establish governance, but not only for security purposes. It’s critical to ensure that data quality, stewardship, trust and governance are always optimized by leveraging a combination of people, processes and tools. Solutions like Azure Purview, Unity Catalog and Collibra, to name a few, are designed to effectively govern data for AI uses. 

Focus on Practical Uses

Once you’ve established a governed data platform, the next step is to identify the business areas and use cases which will benefit most from the application of GenAI. Large language models (LLMs) are not an ideal solution for all problems, so careful evaluation is necessary.

Always start with a proof of concept (POC). Representative use cases and effective POCs not only help inform leadership and executives on the overall suitability, cost and path to final maturity for a project, but they also help you assess platform readiness for production. At the end of the POC phase, an adoption strategy should cover priority use cases, implementation approaches and tools, best practices in LLM operations and a high-level road map to reach production.

While operational use cases are always top of mind for CTOs and CMOs, sometimes it’s the seemingly random use cases of AI that can have a major impact. Think broadly and always justify the return on investment (ROI). Choose use cases that utilize AI to allow employees to spend their time focusing on areas of higher business value, like strategy.

While it is exciting to adopt LLM applications like ChatGPT Enterprise and other GenAI-enabled solutions, think seriously about your organization’s roadmap from experimentation to production — or risk significantly impacting your ROI.

Choose the Right LLM Models

One dilemma you might face is whether to choose a third-party LLM or an open source model. Important factors, like response quality, usage, transparency and the specificity of the task, must be considered. 

Third-party models, like ChatGPT, typically require minimal set-up, offer a high response quality and have low costs for low usage. However, these models carry a risk of data exposure and can quickly become expensive if your organization’s usage volume increases. 

Open source models on the other hand are a viable solution for focused use cases. They are transparent and can be finetuned, allowing a choice of underlying infrastructure and making them cost-effective for high usage. Typically, open source LLMs require more experts for training and maintenance and can sometimes produce lower-quality outputs than third-party models.

As more open source models, like Meta’s LLaMA2 and Databricks’ Dolly 2.0, appear and improve output quality, you will probably begin to move towards open source to help keep costs down and maintain control for security and compliance. 

Importantly, regular model validation and monitoring is essential to mitigate GenAI hallucinations and errors. By finetuning the generative model and implementing effective machine learning operations (MLOps) through testing and validation processes, you can help eliminate shortcomings and reduce biases in outputs.

Responsible Use & Ethical Guardrails

AI is no longer just hype; it has become the new reality, and with it comes new risks. As GenAI is further democratized, organizations will need to involve diverse teams to ensure they can handle potential downsides, like biases or privacy, copyright and security concerns. Actively engaging with your data privacy, legal, ethics and compliance teams — or establishing those teams — to review policies and validate them to defend against the perilous side of GenAI is crucial. It is essential that companies have “humans in the loop” to prove the output of GenAI is acceptable and usable. 

Individual employees also need to learn and understand LLM responsibility with prompt-writing best practices. An established and thorough LLM operations process, including model monitoring and validation, that embeds responsible AI practices into every aspect of the process, can help mitigate many of these problems.

Make the Most of the Unfolding AI Opportunity

Capitalizing on the emerging opportunities presented by GenAI is an exciting prospect for businesses, but as with any new technology, a bit of caution is required. By carefully planning your organization’s AI foundation, you can position yourself to take full advantage of the new paradigm.