Methods of AI Implementation in Companies, AI Functions over Corporate Know-How, and Related Aspects of Corporate Data Security

Summary: How LLM-based AI works, the methods available to implement such AI in a company, how AI implemented through these methods functions, and what to focus on to make enterprise AI implementation effective and purposeful—all of this is the subject of this article. Implementing AI on top of enterprise know-how also introduces a new perspective on enterprise data security; existing corporate security policies will likely not cover the post-AI environment, and this article provides actionable solutions for that as well.

Abstract

Implementing artificial intelligence (AI) based on Large Language Models (LLMs) in the corporate environment represents a major opportunity to increase employee productivity, streamline information workflows, improve how employees, customers, and partners utilize enterprise know-how, and partially or fully automate selected business processes. At the same time, this implementation introduces new risks in safeguarding corporate know-how and ensuring the quality and integrity of outputs used for decision-making both inside and outside the company.

To properly implement and use AI on top of corporate know-how, it is essential to have a foundational understanding of how LLMs and related technologies work. This enables organizations to understand boundaries—where AI is effective and where it is not, what hallucinations are, how LLMs process corporate know-how, and how decisions are made. All of this is explained in the chapter How LLM-based AI Works with Corporate Know-How, while related technologies are detailed in AI Implementation Methods and Their Impact on Data Security, especially in the section RAG Architecture. Without understanding these fundamentals, an enterprise AI initiative risks not only failing to achieve the expected benefits, but also causing costly and hard-to-repair damage both internally and externally towards customers.

There are 3 primary methods for implementing LLM-based AI on top of corporate know-how:

• Using Public AI Tools such as ChatGPT, Microsoft Copilot, or Google Gemini: A fast, low-cost approach requiring no complex integration. However, it lacks connection to proprietary internal know-how, limiting its value for specialized workflows. Processing internal data with public tools also creates severe data leakage risks.
• Training a Custom Internal LLM on corporate data: A technically complex and capital-intensive path. While offering full control over infrastructure, it suffers from major drawbacks: extremely difficult knowledge updates, hallucination risks, and the complete inability to enforce role-based access permissions within the model's neural weights.
• RAG (Retrieval-Augmented Generation) Architecture: The modern, recommended, and industry-standard approach. It combines the strengths of both preceding methods while mitigating their weaknesses:
  • The LLM works dynamically with up-to-date corporate know-how without retraining the model.
  • Answers are strictly grounded in retrieved source documents, minimizing hallucinations.
  • Access permissions can be enforced granularly at retrieval time based on the user's identity.

The individual AI implementation methods are explained in detail in AI Implementation Methods and Their Impact on Data Security; here is a concise comparison table:

Criterion	Public AI Tools	Internal LLM Training	RAG Architecture
Costs	Low	High	Medium
Security	Low	High externally, low internally	High
Accuracy	Low	Medium	High

It is important to emphasize that successfully adopting AI via any of the methods above requires not only mastering technical implementation, but also actively involving employees throughout the process. Organizations must guide this transformation using proven change management methodologies—focusing on human factors, open communication, continuous education, and leadership. These organizational dimensions are beyond the technical scope of this article.

To ensure this analysis provides practical, actionable insights rather than pure theory, the concluding chapter Example of AI Implementation with RAG in the iTutor Platform details a concrete software implementation of LLM-based AI over corporate data featuring an Agentic RAG architecture.

How LLM-based AI works with corporate know-how

This article deals with implementations where AI works with corporate know-how and, via Chatbots, Agents, and other means, helps employees efficiently complete their work tasks, or even takes over some of these tasks and performs them itself.

To determine which method of such AI implementation to choose and what new data security rules to adopt, it is good to understand the specific AI implementation methods and generally grasp how AI functions over corporate know-how.

In this first chapter, we will explain very simply how AI operates on corporate know-how stored as texts or materials convertible to text, such as audio, video, or images. Such texts constituting corporate know-how are usually stored in many places, e.g., in a corporate document management system, on drives, on the company intranet and websites, in corporate databases, discussion forums, emails, CRM or ERP systems, etc.

If you know what an LLM is, the principles it works on, how it learns, how it answers a prompt, what an embedding is, how embeddings for RAG differ from embeddings inside an LLM, or how a vector database works with embeddings, you can skip the rest of this chapter and continue to the second chapter AI implementation methods and their impact on data security.

Those who find this first chapter too technical and detailed can also skip it and continue to the second chapter. They can always return here to specific concepts if a term used in the second chapter is unclear and they would like to learn more about it.

LLM

The "brain" of AI for working with corporate know-how is the so-called LLM, Large Language Model. It can be imagined as "someone" who has read "all the information in the world" (publicly available texts on the internet) and answers user queries or solves their requests posed in natural language based on that. LLM answers are also in natural language.

This "brain" does not think exactly like a human brain; we still know too little about how the human brain thinks to simulate it computationally. An LLM is built on an architecture called an artificial neural network. This is inspired by the activity of neurons in the brain. An artificial neural network consists of a massive amount of computationally represented neurons; each neuron is simulated by a single real number. Computational neurons are placed in layers; each neuron in a layer is connected to all neurons of the previous layer so that the value from each connected neuron of the previous layer is transferred to it via weights. The weighted values of the input neurons are aggregated into a resulting number, to which a constant (bias) is added, and an activation function (e.g., ReLU, Sigmoid, or Tanh) is applied to the result, introducing nonlinearity into the system. This creates the value (number) of the output neuron. The knowledge of the largest LLMs is thus represented by the architecture of neuron connections in layers and the values of hundreds of billions to trillions of weights and biases (there are fewer biases than weights; there are usually a thousand or more weights per bias), where the values of weights and biases are set during training (learning).

Simply put, you input some numbers into the learned neural network's input layer—as many numbers as there are neurons in the input layer. These numbers are gradually transferred via the set weights and biases to neurons in subsequent layers until they appear as some other numbers in the neurons on the output layer. The output of the neural network is again as many numbers as there are neurons in the output layer.

When a neural network learns, it is given samples of input numbers, and for each such input sample, it is always shown what sample of output numbers is expected at the output. During the learning phase, the neural network adjusts its weights and biases using a process called backpropagation so that for the given input samples, its output samples resemble the expected output samples as closely as possible.

An LLM does not work with numbers but with texts; however, the principle described above is similar:

• In the learning phase, "all available texts from the internet" are taken. This in itself is a complex process involving the filtering of this data, called curation, where instead of simply "throwing" all available text into the LLM input, the data is carefully filtered to prevent bias, the spread of false information, and the leakage of sensitive data; the use of licensed information is also monitored. Every such cleaned continuous text from the internet, e.g., one webpage, one downloaded document, is sent in parts to the LLM's input. For each input part of the text sent, the expected next word is shown as the LLM's output. We actually slide a window over the downloaded internet text, which changes in width from a single word up to the maximum supported number of words at the LLM's input, which is called the LLM's context window size. For example, for the text "The cat crawls through the hole, the dog through the window," "The cat" is sent to the LLM input in the first window, and it is shown that the expected output word is "crawls." Then an expanded window "The cat crawls" is sent to the LLM input, showing that the expected output word is "through," and so on, until finally "The cat crawls through the hole, the dog through the" is sent, showing the expected word "window." If the LLM's context window size were only 3 words (in reality, it is much larger and usually holds 150 to 300 pages of text), after reaching 3 words at the input ("The cat crawls"), a shifted window "cat crawls through" would be sent as the next LLM input, and thus a 3-word window would traverse the entire input text. In the learning phase using backpropagation, the LLM sets up to trillions of its weights from all texts sent to its input, embedding not only the syntactic rules of natural language but also the knowledge hidden in the sent texts within its weights.
• The trained LLM then resolves user or AI agent requests or queries by passing the textually formulated requests or queries, whose length does not exceed the LLM's context window size, to the LLM input. The text thus formulated and passed to the LLM as input is called a prompt. From the input prompt, the LLM "calculates" the first word of its output answer using the above-described passage through all layers of neurons with weights set during training. The LLM appends this calculated word to the end of the input prompt text and sends this modified prompt back to itself as input. This is how the LLM calculates the second word of its answer. This could repeat indefinitely, which is why the LLM can at a certain point "decide that this is enough" and hand over the generated answer with a finite number of words to the user.

This very simply explained function of an LLM is enough to understand 2 of the three basic methods of implementing AI in a company over corporate know-how, described in the second chapter. There is also a third, most widely used AI implementation method utilizing RAG technology. Because of it, we will also briefly explain something about converting texts to numbers.

Embeddings

An LLM is a neural network; it works with numbers, not texts. It is good to have a basic understanding of how the input texts sent to the LLM in a prompt are converted into numbers, and how a word is subsequently made from the numerical output of the LLM neural network. For RAG technology, converting texts to numbers is also crucial for another reason: to search for background data for the LLM's answer.

The method of converting text to numbers is built on something called an embedding. An embedding is a numerical representation of a word's contextual meaning. Every word can be converted into an embedding; 2 words that are contextually close in meaning also have close embeddings. Such a numerical representation of a word's meaning cannot be expressed by a single number, e.g., a table having the number (embedding) 4 and a chair 6. These are indeed quite close numbers, as a table and a chair are also somewhat close in meaning, but with this numbering method, we would soon find that it is impossible to express the meaningful proximity and distance of various words without them mutually excluding each other numerically. Therefore, an embedding is not a single number, but a vector of numbers; sometimes there are up to thousands of numbers in a single embedding. In such a multi-numerical vector space, the proximity/distance of individual embeddings (essentially points in this space, representing the contextual meaning of individual words) can be mathematically determined, e.g., using Cosine similarity or Euclidean distance. At the same time, the meaning of all words can be "encoded" into such a vector representation of a word's contextual meaning so that their proximity works out "correctly."

Embeddings can be created not only from words but also from entire texts; the embedding of such a text has the meaning of the entire text encoded within it.

Something extra

For those who feel we are oversimplifying here, and that explanations of many things are missing, a few brief concluding notes.

• Embeddings are actually not made from words, but from so-called tokens. A token can even be just a part of a word. The LLM can then deal even with words it has never seen, it can better handle the grammar of complex languages like Czech, and relatively small token dictionaries can be created (around 100,000 tokens), which can cover all natural language words. Of course, there are many more of those; e.g., there are around 250,000 words in Czech just in their base form; thanks to declension, conjugation, technical and slang terms, it runs into the millions. Methods of making tokens vary; e.g., the word "unbelievable" could break down into tokens "un", "believe", and "able". As you can see, these tokens and others like them can be used to compose many other words without these words needing to be in the dictionary.
• An embedding from a token is calculated using a neural network. An LLM contains a dictionary; technically, it's the first layer of its neural network. If this dictionary has, for example, 100,000 tokens and an embedding contains 16,384 dimensions (numbers), it is a matrix of 100,000 x 16,384 numbers, where each row represents the embedding of a specific token from the dictionary. At the beginning of training, this matrix is filled with random numbers; during the training phase, backpropagation refines not only the weights of the LLM neural network but also these numbers representing the token values.
• The input to the LLM neural network is not a series of numbers, as explained for a general neural network above, but a matrix where each row is an embedding of one token from the input text. Furthermore, positional encoding is added so the neural network can understand that it is not just a "pile" of words (tokens), but that these tokens have an order and a relationship to surrounding tokens.
• Embeddings cannot be easily converted back into words. For an LLM, it is simply done by having as many neurons on the output layer of the neural network as there are words (tokens) in the dictionary. The value of each such neuron is recalculated into the probability with which the dictionary word represented by this neuron should appear in the LLM's text output. Either the word with the highest probability is chosen as the final word. With this choice, however, the LLM acts a bit like a robot; the answers are stable but not very creative. Or, uncertainty is introduced into the answers based on so-called "temperature" and random selection, which can lead to more creative answers, but also to "babbling" or nonsense (hallucinations) in the answers.
• For RAG technology, which will be explained in the second chapter, slightly different embeddings are used. These embeddings typically have a smaller dimension than embeddings inside an LLM, e.g., 1,536 numbers in one embedding. This is because they must be stored in a vector database, and then embeddings must be quickly searched for in it by proximity. A vector database is a specialized type of database for storing and managing data in the form of numerical arrays, known as vectors. Unlike traditional databases, which search for an exact match (e.g., name or ID), a vector database can search by meaning similarity (embedding proximity). Embeddings for RAG are usually created using a specialized neural network. For RAG, embeddings are made from complete paragraphs of text, not just from individual words (tokens). This specialized neural network internally generates embeddings for all words (tokens) in the text, and ultimately groups them (so-called pooling) into a single embedding that contains the meaning of the entire text.

AI Implementation Methods and Their Impact on Data Security

There are several methods to implement AI processing corporate know-how using LLMs. Here we will describe 3 basic ways of these implementations.

The methods are described here from a technical and security perspective. At least in the introduction, it is necessary to mention the human factor, which must not be neglected in any AI implementation. Implementing AI often brings changes to corporate processes and culture; it is necessary to overcome employees' fear of being replaced, which is often a bigger obstacle in AI implementation than the technology itself. It is essential to correctly apply change management, explain the reasons for introduction, combine implementation with the educational process, prepare a pilot project, select AI ambassadors, and do a number of other related things. This is not the subject of the article; describing how to work with the human factor during AI implementation would take up a whole article of a similar size to this technical one. If readers are interested, we will include an article about the human factor and corresponding procedures during AI implementation in Kontis Insights in the future.

3 basic methods of implementing AI processing corporate know-how using LLMs:

Using ready-made AI tools without connection to corporate know-how

These are tools like ChatGPT, Microsoft Copilot, Google Gemini, Claude, Perplexity AI, and other similar LLM-based tools.

Using ready-made AI tools brings zero development costs, an almost instant solution launch, or low operating costs.

These tools are not inherently connected to corporate know-how; the LLMs used "know nothing" about corporate know-how. During the learning phase, the corresponding LLM might have learned something about the company from internet sources, e.g., it was trained on corporate websites or other public sources about the company. This learned knowledge was merged within the LLM's weights with billions of other facts read from the internet, which necessarily leads to generalization. Based on a user's request in the prompt, the LLM might also decide to search the internet for some information about the company additionally and add it as another input to the prompt. The LLM can access this corporate information via an internet search engine, so the texts it can find are similar to the input texts during learning, just more current, for example. There won't be much internal corporate know-how in them.

An LLM used in this way can greatly help solve work tasks for which employees do not need corporate know-how. Before using it, it is good to train employees so they have a basic understanding of how an LLM works and how to properly construct a prompt for the LLM to get the best possible answers. See the first chapter explaining LLM functions; if the user doesn't enter a sufficiently comprehensive prompt that well explains the request, the LLM cannot provide a quality answer.

The problem arises if the employee wants to solve AI queries or requests related to corporate know-how. The LLM does not have the answer "encoded" in its weights because it was not trained on corporate know-how. It probably won't say it doesn't know; instead, it will start making things up, so-called hallucinating. For example, if you are a manufacturing company producing on assembly lines, and you ask such an LLM "How is product X clamped into the line's mechanism on the production line," the LLM will not say it doesn't know, but will construct a relatively credible answer based on what it learned during the learning phase from the internet about various production lines or product clamping. For your company specifically and your procedures, it may answer credibly, but completely incorrectly.

This leads employees to insert documents containing corporate know-how into the LLM prompt, based on which the LLM is supposed to answer. For instance, in the manufacturing company example, the user inserts a query and a corporate document with work instructions for the given production line into the prompt, instructing the LLM to answer only based on the information in the attached document. Similarly, for example, a manager might attach a contract with a customer to the prompt and ask if they are violating anything with the activity specifically described in the prompt, and if so, what the penalties are.

Problems with using LLMs without connection to corporate know-how

If users in such an implementation want to resolve queries or requests related to corporate know-how:

• The user must be able to find documents with the related corporate know-how to attach to the prompt. Finding all such documents and other information sources, which could be in emails, chats, bug lists, the company ERP, and the like, is not easy. The user often does not find everything from the corporate know-how related to the query.
• Even if the user finds all the necessary background data for a quality LLM answer, these materials may not fit into the prompt, see the LLM context window size explained in the first chapter.
• Even if the materials fit into the context window, if they are large, the LLM has a harder time finding relevant parts for the answer in them and loses detail from a long text.

Data Security of This Method

With the above-described insertion of background materials into the prompt, entire corporate documents travel outside the company. Even if the communication with the LLM is secured en route and the LLM provider guarantees that they will not train their LLM based on this data, or use it for anything other than generating the LLM answer, in many companies this will not be approved by the departments responsible for corporate data security. Especially if, for example, customer contracts travel out via the prompt, see the previous example, or documents containing sensitive data from a GDPR perspective, it can be problematic. At the very least, further data security measures must be taken here, such as:

• Training users on what documents they can insert into a prompt. This is usually associated with introducing a corporate document classification system and assigning a specific classification to each document, so users can recognize which documents and under what conditions they can insert into a prompt.
• Implementing DLP (Data Loss Prevention) systems controlling data sent out of the company.
• Implementing an AI Gateway that filters prompts, anonymizes data, and audits usage.

Such measures entail additional costs, and they cannot guarantee 100% protection against corporate know-how leaking outside the company. If this is the goal, the LLM needs to be "closed" within the company's intranet, see the next 2 methods. A custom generally trained LLM can also be closed inside the company in the case of this method, but it doesn't make much economic or functional sense without simultaneously implementing the RAG architecture. What custom generally trained LLMs can be closed within the company and what requirements are associated with this is therefore described in more detail in the paragraph Data security outside the company under the RAG architecture method.

Training and implementing an internal LLM

On the opposite spectrum of cost, implementation time, and also security against corporate data leaking outside the company is the method requiring the training of a custom LLM model on corporate know-how. Such an LLM can then operate completely closed within the corporate intranet.

There are 2 ways to train an LLM on corporate know-how:

Full Training

With Full training, you start from zero; the LLM doesn't know a single word, its weights are not set. It's necessary to use the "whole internet" in training, the data of which underwent curation as described in the first chapter, and additionally include corporate data, again cleaned, into the input training data. From the description of LLM training in the first chapter, it is evident that this involves high demands on technical experts, complex data cleaning, and very high computational power costs—to give you an idea, Full training usually requires thousands of GPU hours, and the training phase takes a long time. The entire training process usually takes several months. For these reasons, Full training of a custom LLM is economically nonsensical and unattainable for 99% of companies. The fundamental problem with training on corporate data is that corporate know-how is not a static thing; on the contrary, it changes very rapidly over time. Every such change in corporate data would require a new retraining of the LLM or fine-tuning it.

Fine-Tuning

With Fine tuning, you start with an already set up, i.e., generally trained publicly available LLM such as Llama or Mistral. The generally trained LLM is then further trained on corporate data. This is economically and temporally more acceptable than Full training, but this method is also associated with a longer implementation time, a demand for technical experts and cleaning of input corporate data, and the need for constant retraining of the LLM related to the change of corporate know-how.

Problems with Training an LLM on Corporate Know-How

Both Full training and Fine tuning carry critical disadvantages when training on corporate know-how, including, for example:

• The LLM cannot cite corporate documents in its answers based on which it generated the answer. It is therefore impossible to verify the accuracy of the LLM's answer, which is a serious problem given the potential for LLM hallucination described above.
• During fine-tuning, there is a risk that the LLM will forget some already learned things when adjusting weights; this is called "catastrophic forgetting." If we fine-tune the LLM only on corporate data, even if we always use the entire set of current corporate data for each such fine-tuning, there is a risk that the LLM will "forget" something from the initial Full training over the entire internet; the LLM becomes an expert on corporate know-how, but stops handling general queries or logical reasoning. There are various methods to correct this, but they are relatively complicated and demanding.
• During fine-tuning, corporate data is usually very "small" compared to data from Full training, creating a risk that the LLM learns the training examples "by heart". As a result, the LLM answers perfectly on data from the training set, but fails in real operation with slightly different queries.

Data Security of This Method

In terms of securing corporate know-how, if the trained LLM is closed inside the company within its intranet, it is fully secured that the corporate know-how does not get outside the company.

However, during the LLM training on corporate data, the corporate know-how became part of the LLM's internal weights, see the first chapter on how LLMs work. Such a trained LLM cannot distinguish in its answers which user had what rights to specific corporate data based on which the LLM answers. If the model knows the answer, it answers anyone who asks correctly. This makes the method of training a custom LLM unusable for most companies; from the outside, the corporate know-how is perfectly secured, but inside the company not at all; any employee with access to the LLM prompt automatically gets access to any information from the entire corporate know-how.

See the manager example above who asks about something regarding a specific contract; a trained LLM on corporate know-how will most likely answer the manager correctly. But if any other employee with access to the LLM prompt asks something about this contract and the customer, the LLM will provide them with all the information it knows, even though this employee did not have the right to look into the company's contracts with customers.

RAG Architecture

Implementing AI over corporate data based on the RAG architecture is currently the most widely used method for teaching an LLM-based AI to work with corporate know-how.

For the AI implementation methods described above, the problems associated with them are briefly outlined. Using the RAG architecture solves almost all these problems.

The acronym RAG stands for:

• Retrieval, i.e., finding the information based on which the LLM should answer the query or request in the prompt.
• Augmentation, meaning expansion; the retrieved information is attached to the prompt with the user's query/request, including instructions on how the LLM should answer solely based on this attached information.
• Generation, i.e., generating; this is the classic LLM generating an answer based on the prompt, augmented by related information from the corporate know-how.

Searching in Corporate Know-How

The essential part of RAG is the Retrieval, finding information in corporate know-how. In the RAG architecture, this search is based on embeddings, the essence of which is explained in the first chapter in the section Embeddings.

Preparation of embeddings

First, smaller pieces of text, called chunks, are cut from all texts representing corporate know-how. The goal is to divide the corporate text know-how into parts that can be best assembled as context for the user's query in the LLM prompt. If a chunk were too small, e.g., one word, the meaning would not be sufficiently contained in it. If a chunk were too large, e.g., an entire chapter, it would be too extensive, with multiple topics, it would be hard to search by meaning, and the LLM would process it less efficiently. There are various methods for dividing text into chunks, e.g.:

• simple division into text parts with a fixed character length
• more advanced division by paragraphs or subchapters
• sophisticated methods where the ends and beginnings of topics in the text are again determined using AI
• sometimes a hierarchical structure is made from chunks, a wider context of the chunk is sent to the LLM, etc.

Usually, the length of one chunk is around 256 to 512 tokens, which are again explained in the first chapter. Various improvements are used, such as slight text overlaps; each chunk contains 10-20% of the end of the previous chunk at its beginning so that context is not lost. Or, the context of the entire document from which it originates is assigned to the beginning of each chunk, something like "This paragraph comes from the work instructions for production line X" or "This paragraph comes from the contract about X with customer Y".

Embeddings for RAG, described in the first chapter, are calculated from the chunks created this way. These embeddings are stored in a vector database along with the original texts they were created from.

Retrieving context for the user's prompt

The user enters their request/query in the prompt. Either an embedding is made from it, or the entered prompt is processed beforehand. Ranging from simple methods like cleaning the prompt of unnecessary words or generating an embedding from each word, to sophisticated methods using AI, such as generating query formulation variants, extracting topics from the user's prompt, or generating fictitious answers from which embeddings are then created.

The result of prompt processing is one or several embeddings from the user's entered prompt. For this resulting one or more embeddings, the closest embeddings in the vector space are searched for in the vector database with stored embeddings from the preparation phase. These most likely contain information from corporate know-how related to the user's query or request. For these retrieved embeddings, the original texts from which they originated are also stored in the vector database. These original texts are added to the prompt as context for the request, with instructions that the LLM should answer based only on them.

The RAG architecture thus does something similar to what a user does when Using ready-made AI tools without connection to corporate know-how by attaching corporate documents to the prompt according to which the LLM should answer. RAG does it more sophisticatedly and efficiently. If we take the above-listed Problems with using LLMs without connection to corporate know-how, the following applies here in the same order:

• Unlike a user, RAG will likely find many more semantically (meaningfully) close pieces of information from corporate know-how that relate to the user's query in the prompt. The LLM thus has more related information in the prompt based on which to generate an answer.
• The background materials found by RAG are not entire corporate documents, but only their fragments related to the query. So these texts will most likely all fit into the LLM context window.
• Because the texts contain only information cut from documents truly related to the query, the LLM can efficiently create an answer from them without getting lost in the detail of a long text largely unrelated to the query, see LLM function.

Supplementing answers with source links

RAG architecture removes the critical shortcoming of Training and implementing an internal LLM, where the LLM cannot cite links to corporate documents in its answers from which it drew information for the answer. Metadata can be attached to the embeddings stored in the vector database, such as the already mentioned links to source documents, or even their specific parts, from which the embedding was created in the preparation phase. RAG can attach these links to LLM answers; the questioner can verify the answer in the original source to see if the LLM is hallucinating.

Securing corporate know-how in RAG architecture will be discussed in a separate subchapter; RAG can eliminate almost all data security disadvantages named in the previous two AI implementation methods.

If we use the previously mentioned example, where a user inserts into their prompt "How is product X clamped into the line's mechanism on the production line" a corporate document with work instructions for the given production line, RAG architecture does this for the user more sophisticatedly. The description of "clamping" is often spread across the text manuals of multiple work instructions, technical drawings with labels, bills of materials, and it may also relate to information in occupational safety documents, etc. Given a properly posed query, RAG will likely retrieve most of these texts, cleaned of information unrelated to the query. The LLM's answer to the prompt enriched with these texts will be more accurate and concise. Thanks to the links attached by RAG to the LLM's answer pointing to the source texts, the questioner can verify the answer in the referenced documents.

Similarly, when a manager asks in a prompt whether a specifically described activity violates anything with a specific customer, and if so, what the penalties are, unlike a questioner who attached an entire contract to the prompt, RAG will retrieve not only this contract with the customer and only the passages related to the query within it, but RAG may also find similar texts, if they exist, e.g., in the contract's appendices, in saved communication with the customer about the subject of the contract, in directives specifying the company's general rules for working with customers, etc. The LLM's answer to the query will again be significantly better than just with the attached contract to the prompt, with the possibility of verifying in the affected documents that it is indeed the case.

Data Security of This Method

Inside the company

When producing embeddings from the cut know-how texts, described in their preparation phase, various metadata can be stored with the embeddings, such as the aforementioned links to source texts. This metadata can also be used to store and subsequently determine the questioner's rights to specific texts related to the produced embedding. There are multiple methods for doing this, e.g.:

• Filtering by metadata. Information is stored in the embedding's metadata according to which only embeddings the user has the right to will be filtered in the vector database when searching for embeddings close to the user's query. Such filtering by the vector database engine enables efficient and fast retrieval of embeddings close to the query embedding. However, it does not allow for complex rights management. For example, a list of allowed users, a group or role the user must belong to, etc., is stored as metadata for the embedding. From this, it's evident that a complex rights management system that changes over time in a company cannot be built this way.
• Relationship-based access control. The ID of the object from which the embedding was generated is stored with the embedding. When searching for close embeddings to a user's query, the vector database returns, say, the 50 closest results regardless of rights. The RAG architecture asks an external service providing complex rights management in the company which of these 50 specific objects the questioner has the right to see. Only texts from embeddings whose source objects the user has the right to are added to the LLM prompt. If there are few such embeddings among the retrieved 50, RAG iteratively searches the vector database for other embeddings, e.g., by selecting a different metric or threshold value, which may not be as "close" to the user's query as the previous 50 retrieved, but the user has the right to see the objects associated with the texts.

In this way, from a data security perspective, it can be guaranteed that unlike a trained LLM on corporate know-how, the LLM will only return information to the user in response to their query or request that the user has the right to see within the corporate know-how.

Outside the company

If necessary, similar to securing a trained LLM on corporate know-how, the RAG architecture can be fully closed within the company's intranet, thus fully protecting corporate know-how from any leakage outside the company.

In this case, in addition to the corporate platform where corporate know-how is stored, the following must also be operated on corporate servers:

• A vector database for storing embeddings. There are large scalable solutions on the market with advanced search functions, e.g., vector DBs Qdrant or Milvus, as well as solutions simpler to set up and operate, suitable for small to medium projects, like ChromaDB.
• A model for producing embeddings. There are top models on the market like BGE-M3, which is multilingual and therefore suitable even for Czech, or models from MixedBread.ai, TEI from Hugging Face, or Nomic Embedd.
• A custom generally trained LLM. There are LLMs on the market with top performance like Llama 3.1 or newer models of the Qwen3.5 series, which however require powerful hardware, multiple GPUs with high VRAM. Or there are models less demanding on hardware available, like Mistral Large 2, Ollama, or vLLM. Such a generally trained LLM can additionally be fine-tuned in this use case, e.g., to the corporate communication style, which is a quite interesting extension of the general LLM function.

The costs of such a solution must be considered; for top-tier solutions, hardware acquisition will run into millions, and significant amounts will also go into operations. Besides electricity and cooling, these will mainly be costs for experts who will maintain and update the named systems. Costs for experts are often higher than the hardware itself and its operation.

For this reason, closing a vector database, an embedding generation model, and an LLM inside the corporate intranet is usually only approached in cases where privacy is an absolutely critical requirement, or massive AI operations are expected, with volumes over 10 billion tokens monthly, where it sometimes also makes economic sense.

Most companies will make do with an operation where they access the vector DB, embedding generation model, and LLM via API to already ready-made SaaS solutions. There are high-quality solutions on the market from OpenAI, Azure OpenAI Service, Anthropic, DeepSeek (here there might be a security issue in some regions, e.g., in the EU, because the servers are in China), Groq, or Mistral AI.

There are universal adapters, such as LiteLLM, which can be used to switch between individual solutions and potentially change SaaS providers according to changing requirements for security, performance, and quality, without the company having to change the entire corporate system.

SaaS solution providers usually guarantee that they will not train their LLMs on data sent via API nor use it in any other way; some providers guarantee operations in EU data centers, e.g., Azure OpenAI Service or Mistral AI.

The costs of SaaS solutions are usually calculated based on the amount of tokens that pass through the API. Unless a company sends billions of tokens via API monthly, a SaaS solution will be much more economical than running everything on its own servers.

Agentic RAG

Request Processing in Classic RAG

The RAG workflow described above is commonly referred to as Classic RAG. It processes a user request or inquiry as follows:

1. The user submits a request or inquiry.
2. The prompt containing the user query is enriched:
   • Using the various vector methods described earlier, the vector database is queried for embeddings that are semantically (by meaning) closest to the user's request.
   • The final prompt for the LLM is assembled from: the user query, the raw texts from which the retrieved embeddings were generated, and guiding instructions (system instructions) detailing how the LLM should formulate its response using the query and attached texts.

Accompanying system instructions typically define how to leverage the attached texts (e.g., "Answer exclusively using information provided in the attached context."), what persona or identity the LLM represents and what its specific role is (e.g., "You are Assistant Carl – an AI assistant focused on inventory management tasks for brake components."), what tone and style to adopt (e.g., "Respond in a friendly, clear, and helpful manner."), and various behavioral guidelines (e.g., "Keep answers concise and actionable. Proceed step-by-step only when strictly necessary (1–3 sentences per step). If clarification is required, ask only ONE clarifying question at a time."). Based on this enriched prompt, the LLM generates its response. The generation mechanism itself is detailed in the preceding sections of this article.

Limits of Classic RAG

This classic approach to RAG encounters several fundamental limitations:

1. Not all types of information are well-suited for storage in a vector database as embeddings and subsequent semantic similarity retrieval:
   • Information suitable for embedding storage typically includes data that remains relatively stable over time and carries strong semantic meaning. Prototypical corporate know-how suitable for embeddings and vector DB storage includes internal policies, operating procedures, contracts, training course contents, process descriptions, product specifications, and company guidelines.
   • Information unsuitable for embedding storage changes frequently over time, fluctuates dynamically based on context, and generally lacks broad semantic conceptual meaning on which vector similarity search relies. For example, regarding the company products mentioned above whose technical descriptions are stored as embeddings, actual stock levels on warehouse shelves change from hour to hour; continuously regenerating and re-indexing embeddings with real-time stock quantities makes no sense, and vector similarity search across numeric inventory levels is completely ineffective or fails entirely.
     Similarly, while training course catalog descriptions can be embedded effectively, the assignment of specific courses to individual users should not be stored as embeddings. Doing so would require generating separate embeddings for every user's current course enrollment list. Moreover, completion status (e.g., whether employee John Doe completed his fire safety exam yesterday) changes dynamically, requiring continuous re-indexing. Storing this type of information—which we refer to as "live data"—in embeddings is impractical.
     Classic RAG cannot effectively handle such live operational data, yet an employee or manager querying the AI will frequently expect answers that combine conceptual know-how with real-time operational status.
2. Not all queries or requests can be resolved in a single-pass "query → prompt enrichment → LLM output" workflow. Complex user inquiries often require multiple sequential actions and reasoning steps to formulate an accurate answer—for instance, first searching for products matching the user's technical description (found in the vector database), and then retrieving current inventory counts across multiple branch locations (queried from an ERP database). Classic RAG is not designed for multi-step workflows, intermediate planning, or programmatic iteration.
3. Classic RAG operates in a strictly passive, read-only mode. It cannot trigger changes in surrounding software systems, such as registering an employee for a webinar, creating an IT support ticket, initiating an approval workflow, or scheduling an appointment.

Extended Capabilities in Agentic RAG

To overcome these limitations of Classic RAG, the concept of Agentic RAG was developed. In Agentic RAG, the LLM is transformed from a passive text synthesizer into an active decision-making agent equipped with tools to interact dynamically with external systems and live data sources.

Tools

In Agentic RAG, the LLM is provided with a curated library of callable Tools. A Tool is essentially a programmatic interface (such as a REST API endpoint, database query, or function call) that performs a specific action or retrieves specific real-time data. For each Tool, the LLM is provided with a structured schema detailing: the tool's name, a natural-language description of its exact purpose and when it should be invoked, and the parameters it accepts along with their data types and descriptions.

When a user submits a prompt, the LLM analyzes whether its available tools can assist in fulfilling the request. If the LLM determines that a tool call is needed, it pauses text generation and outputs a structured tool request specifying the function name and exact arguments. The host application executes the function, retrieves the real-time data or performs the action, and returns the result back to the LLM. The LLM can then incorporate these fresh, live results into its final answer or decide whether additional tool calls are necessary.

Autonomy in Request Resolution

Unlike Classic RAG's single-shot pipeline, an Agent in Agentic RAG operates with autonomous reasoning loops (often based on patterns such as ReAct – Reasoning + Acting). The agent can formulate a plan, break a complex objective into subtasks, invoke multiple tools sequentially or in parallel, evaluate intermediate findings, correct course if a query yields insufficient data, and synthesize a comprehensive final answer only when all prerequisites are fulfilled.

Modifying the State of the "Surrounding World"

Beyond reading live data, Agentic RAG enables action-taking tools that modify state in external enterprise systems. For example, an agent can submit an expense report, assign a mandatory compliance e-course in LMS iTutor, reserve a seat in a classroom seminar, or update employee competency assessments. This elevates AI from an informational search assistant to an autonomous enterprise assistant capable of handling end-to-end business transactions.

Security Requirements in Agentic RAG

Empowering an AI agent with autonomous tool execution and system write permissions introduces significant security challenges that require rigorous enterprise architecture:

First, every tool that queries data or performs actions must strictly enforce the authenticated identity and permissions of the end user (the human caller). When the AI agent invokes a tool to check inventory or retrieve employee certifications, the backend function must verify that the requesting user possesses the necessary read permissions for that specific data. If the user lacks permission, the tool must return an explicit access denial rather than exposing restricted corporate records.

Second, tools should supply verifiable reference links wherever possible. Just as Classic RAG provides citations to source documents, tools retrieving live data can return deep links to external systems (e.g., linking directly to an ERP inventory view or an LMS course ledger) so users can independently verify the veracity of the AI's statements.

Crucially, for tools that modify enterprise data or trigger external transactions, organizations must establish clear governance: AI agents lack legal personhood, and the company remains legally and financially responsible for all actions executed by the agent. Organizations must implement the following safeguard rules:

• Strict Tool Triggering Criteria: Tool descriptions must explicitly forbid invocation unless the user has unequivocally stated intent. For example, a course enrollment tool description should mandate: "USE ONLY if the user has EXPLICITLY expressed a clear desire to register or enroll in a specific session (e.g., 'Sign me up', 'Register me for the June 12 session'). NEVER call this tool if the user is merely asking questions about course content or checking seat availability."
• Strict Parameter Descriptions: Parameter definitions must instruct the LLM never to guess or infer missing identifiers from conversation history. The LLM should be instructed: "Unique Course ID. If the user has not provided it, NEVER guess or hallucinate it; ask the user to clarify or search for the course first, confirming user intent before proceeding."
• Human-in-the-Loop Strategy: For high-stakes, irreversible, or contractual actions, the agent should not execute the mutation directly. Instead, the agent prepares the action and generates a secure confirmation link in the external system. The human user must review the pre-filled form and click the final "Submit" or "Approve" button themselves. This ensures accountability, prevents accidental automated changes, and guarantees compliance.

Even with meticulously engineered tool descriptions and robust permission enforcement, organizations must recognize that the underlying engine remains an LLM governed by probabilistic token prediction, as detailed in How LLM-based AI Works with Corporate Know-How. The risk of hallucinations and tool misuse can never be mathematically reduced to zero; robust architectural guardrails, Role-Based Access Control, and Human-in-the-Loop patterns are therefore essential requirements for enterprise deployment.

Example of AI Implementation with RAG in the iTutor Platform

Implementation of Classic RAG AI over Corporate Know-How in iTutor

iTutor is a comprehensive enterprise platform that, among other capabilities:

• Centralizes and organizes corporate know-how across all organizational levels.
• Governs which portions of know-how are accessible to which employees using competency and qualification matrices, training pathways, and a granular permission management system.
• Provides employees with timely information, targeted training, and guidance to effectively apply shared know-how in their day-to-day work.
• Enables automated assessments, 360° feedback, goal tracking (MBO), and analytical tools to verify that corporate know-how is properly retained, applied, and driving business results.

The iTutor platform offers extensive modular functionality—see the full iTutor platform overview. From the perspective of implementing AI over corporate know-how discussed in this article, this overview highlights the essential architecture.

Corporate know-how within iTutor is structured across two primary pillars:

• Centralized in iTutor Documents (DMS): A powerful document management system (iTutor Documents) housing text, graphic, audio, and video documentation. iTutor Documents is uniquely suited for managing enterprise know-how across multinational, multilingual environments because:
  • It features advanced review and approval workflows with granular access control across document versions and draft states.
  • It supports language sub-versions with dedicated translation and validation workflows, accommodating regional regulatory variances and localized guidelines.
  • It provides specialized business plugins, such as a graphical work instruction editor for creating modular, interactive assembly-line procedures and standard operating procedures (SOPs).
• Stored in iTutor LMS/LXP:
  • Conceptual know-how is embedded within courses and interactive e-learning modules. In iTutor, the integrated iTutor Publisher (LCMS) builds advanced e-courses and automatically parses their contents for AI ingestion. Interactive SCORM packages normally hide text inside complex JavaScript simulations; iTutor Publisher extracts and transforms this interactive pedagogical content into clean, structured textual transcripts suitable for vector embedding and retrieval.
  • Operational know-how exists as "live data" across relational databases—such as real-time course completions, certification expiry dates, scheduled training sessions, remaining seat capacities, and pending approval requests. For this live data, iTutor AI equips its Agentic RAG with dedicated callable Tools, as detailed in Agentic RAG.

Schematic Diagram of How Classic RAG Works in iTutor

The diagram below illustrates the ingestion, vectorization, and retrieval architecture of Classic RAG operating within the iTutor platform:

Agentic RAG in iTutor

In addition to Classic RAG, iTutor features an advanced Agentic RAG engine, surfaced to end users through the integrated AI Assistant available in learner and employee portals.

The AI Assistant is an autonomous agent equipped with a rich suite of enterprise Tools designed to solve complex multi-step tasks across the iTutor platform:

• Vector Knowledge Retrieval Tool: The AI Assistant utilizes Classic RAG as a foundational tool to query the semantic vector database across company policies, SOPs, and e-course content.
• Live Data Inspection Tools: The assistant is equipped with specialized tools to query real-time data from iTutor—including user training records, quiz attempts, qualification expiration dates, open webinar slots, instructor availability, and multi-level approval states.
• Action Execution Tools: Custom action tools can be configured to execute transactions in iTutor, such as enrolling users in courses or initiating approval requests, as described in Modifying the State of the "Surrounding World". These tools are tailored per client with strict safety configurations detailed in Security Requirements in Agentic RAG.
• RBAC for Tools (Role-Based Access Control): To ensure total data safety, iTutor AI applies a pioneering security architecture: dynamic tool filtering based on user permissions before tools are exposed to the LLM. Every tool is bound to specific system permissions. For example, if a tool retrieves group member lists, the user must hold the "ViewGroup" right or manage that specific team. If the logged-in user lacks this permission globally, iTutor AI completely excludes the tool from the LLM prompt. This delivers three critical benefits:
  • Enhanced Security: The LLM is completely unaware of unauthorized tools, eliminating any possibility of prompt injection attacks coercing the model into invoking forbidden actions.
  • Token Efficiency: Eliminates wasted context window tokens on irrelevant tool descriptions that the user cannot execute.
  • Higher Precision: Presenting fewer, highly relevant tools dramatically reduces tool hallucination and selection errors.
  For enterprise deployments with hundreds of tools, iTutor AI can first retrieve relevant candidate tools using vector search over tool metadata, and then filter the selection via RBAC.

The AI Assistant can autonomously plan, decompose, and execute multi-step inquiries, combining semantic knowledge extraction with live operational data retrieval from internal and connected external systems.

Examples of Use

Here are several practical examples demonstrating how iTutor AI and Agentic RAG operate in real-world corporate environments:

• Smart Work Instruction Assistant on Production Lines: A line worker asks how to mount a specific mechanical component. iTutor AI leverages its vector database to retrieve the relevant visual work instruction created in iTutor Documents. Simultaneously, via Agentic RAG tools connecting to SAP, it checks current parts availability in the warehouse and verifies whether the operator holds the required active qualification certificate to perform the installation. If the operator's qualification has expired, the AI provides the mounting instructions while alerting them that supervisory sign-off or recertification is required, linking directly to the source procedure.
• Contractual Compliance & Governance for Managers: A project manager inquires whether a proposed SLA adjustment for a specific client violates contractual terms and what penalties might apply. iTutor AI retrieves the client's master service agreement, current addenda stored in iTutor Documents, and past correspondence via Microsoft Graph API connector (Exchange). After verifying that the manager has authorized read access to client contracts, iTutor provides an exact contractual analysis with direct citations. If the user lacks access to confidential client files, the system falls back to general company guidelines on client communications and contracting principles, ensuring confidentiality is strictly preserved.
• Internal HR & Onboarding AI Chatbots: Organizations can deploy customized iTutor AI chatbots directly into corporate intranets via Single Sign-On (SSO). When an employee asks about vacation allowances, parental leave policies, or their specific employment contract clauses, the HR Chatbot retrieves personalized information based on the employee's verified identity and access rights. A coworker asking about a colleague's compensation or contract terms receives zero confidential information, ensuring complete GDPR compliance and internal data segregation.

In summary, iTutor is a comprehensive enterprise platform with fully implemented RAG Architecture and Agentic RAG capabilities, designed to manage, protect, and intelligently process enterprise know-how. Furthermore, iTutor AI can connect to external business systems (ERP, CRM, intranet portals) to execute structured, multi-step agentic workflows with live data integration. Deploying iTutor allows organizations to implement enterprise-grade, secure AI over their corporate know-how immediately and cost-effectively, while eliminating the substantial risks and expenses of custom AI model development.