Skip to content
All posts

Stumbling Out of the AI Starting Blocks

It goes without saying that there is tremendous hype around generative AI. But there is also lots of misinformation or, at best, glossing over of the more complex points that make it real. For generative AI to deliver real value to an organization, it must connect its knowledge base to the AI. This argument was well articulated in Forrester’s recent contributed article in Forbes, “How Generative AI Is Affecting Knowledge Management.” I’d like to use this article as a launch pad to point out at least two major obstacles that AI practitioners face when they stumble out of the starting blocks.

The first mistake is thinking a large language model (LLM) app like ChatGPT (GPT-3/4) is all you need to implement generative AI. Clearly GPT-3 is great at generating text like stories, summaries, and lists out of thin air. For some people, connecting to organizational knowledge translates to copying and pasting snippets of texts into ChatGPT and continuing the cycle when prompted to do so until the final result is achieved. This is obviously not scalable nor very practical beyond simple use cases. Even when “knowledge” is fed to GPT-3/4 through APIs as a form of automation, the step of automating the retrieval of knowledge to feed the LLM is glossed over. It’s assumed that the user has the ability to do this easily, that facts are easy to find, and that somehow it is easy to package up in just the right format for the LLM to process effectively. Company AI leaders need to figure out a scalable approach to this problem, and strategy and IT consultants need to have good solutions for their clients or everyone is going to waste enormous amounts of money and time.

The second major stumbling block is underestimating the important step of ingesting or transforming organizational data and preparing the data for AI. Structured data is fairly straight forward as the data is “structured” or organized in a well-defined tabular format. Even then, the AI team needs to work with the data owners for access and devise a strategy for handling security and privacy. Unstructured data, which accounts for about 70% of an organization’s data, is a much more difficult challenge. Language or textual information, which is the most useful for LLMs, resides in documents and files. Companies can easily have thousands of documents and upwards of millions of files. Most files are in PDFs, and many PDFs can be gnarly and near impossible to decipher. If you’re at a company that has upwards of hundreds of thousands or millions of files, then you must determine a viable strategy for scale. Then you may choose to embed vectors or not into your data (I’ll tackle this topic in my next blog). Given all these challenges of extracting knowledge from documents, I suppose you can forgive these articles and media discussions for skipping the boring data ingestion and preparation steps and going straight to the sexy part of miraculously generating content. However, to deliver real business value from generative AI, you should do it right from the start and devise a solid strategy for getting your enterprise’s structured and unstructured data ready for AI.

Don’t skip the hard first steps of “AI-ification” of your data. It’s the foundation of your AI infrastructure. Do it right and your AI journey will be a smooth one.