Document RAG and chat configuration
Technical description
RAG (retrieval-augmented generation) rule sets for documents divide their texts into individual sections, expand these sections with further explanatory texts (e.g. "The following text is a description of the machine ...." or "The next section describes how the maintenance work must be carried out") and store these text sections in special RAG chunk embedding vector databases.
Instead of using keywords or categories to find specific documents - as was previously the case - you can now search for documents in a RAG database using entire sets of questions. The result of such a search is a list of the most suitable documents, each with a hit accuracy.
Furthermore, the text sections of the documents stored in the RAG database are also used to fine-tune an LLM (large language model). This makes it possible to chat with documents, i.e. to make the knowledge contained in the documents available via natural language.
The apps described here are part of the KLIO App Suite for NLP (natural language processing) DMS (document management system).
Functionality
- Edit document RAG
- List document RAG
- Select document RAG
- Default settings
- RAG database query
- Document Chat Inspector
Document RAG
This app can be used to create and test rules for RAG (retrieval-augmented generation) for documents and store them on documents.
These sets of rules are decisive for the quality of the "answers" of an NLP system, as it has been shown that the answers are always better if a lot of context information is given to each text section (chunk) to be stored.
You start by selecting an existing document reference, a single file, a file directory or a directly entered text as "Input Text". If the document reference refers to a directory or if you have selected a file directory, you can use the left/right buttons to display the individual documents/file contents (if a file preview is possible).
Depending on the type of input text(s) (e.g. plain text, HTML formatted text or text in .pdf files), it may be necessary to pre-process the text, which can be selected in the document preparation box.
The chunking of the input text(s) can be carried out automatically by the system or a procedure can be specified manually to generate individual text sections (chunks) and possibly expand these with further information (retrieval augmentation).
With the "Generate" button, the text currently displayed as input text is processed with the selected automatic or manual set of rules and the created sections are displayed in the output text chunks area. You can also use the left/right buttons to display the individual chunks.
In addition to displaying the generated text chunks, the documents from which the text chunks were created are listed in the metadata tab in the output. With the "Chunk structure" button, you can display the hierarchy of the text chunks within the document and edit them if necessary. The "Save chunks" button saves the chunks permanently in the RAG Rules document, i.e. they are no longer regenerated. This saving is the necessary preliminary stage for uploading the chunks to the RAG database.
If no document reference is specified, the input text can also be entered freely. This is intended for entering smaller texts to test the processing procedure.
If the set of rules for the referenced documents/files is considered functional and correct, you can link the set of rules directly to the document reference (see button in action group). If no document reference has been selected as input text at the beginning, but a file or a file directory, a new document reference is automatically created and linked to the set of rules.
| Menu item | Menu description |
|---|---|
| Edit | - |
| : Search terms | - |
| : : Insert | Another search term is added to the RAG document |
| : : Remove | The selected search term is removed. |
| : Notes | Display/maintain a single note. |
| : Create process note | - |
| : : Note | A process note can be created for the document RAG. |
| : Operations | Display of the process folder. |
| : Validity | Maintenance of the validity of the document RAG. |
| Close | The app is closed |
| Search | Calling up the list window |
| Default settings | Call up the default settings for this app. |
| Icon | Toolbar description |
|---|---|
 | Create new document RAG |
 | Save document RAG |
 | Delete document RAG |
 | Notes |
 | Processes |
 | Create new process document |
 | Validity |
| Field | Field description |
|---|---|
| General | |
| Data | |
| : : Number | Unique number of the document RAG set of rules. The format and assignment of the number can be controlled via counter objects. |
| : : Search terms | Search terms can be inserted or removed using the right mouse button. |
| : : Short name* | Enter a short name. |
| : : Classification | Assignment to a (classifying) classification term. |
| : : Description | Description text for the document RAG. |
| AI Tenant | Selection of the tenant in the AI Cloud. If this is not set, the entire app is blocked. The configuration of each tenant is completely separate from the configuration of other tenants, i.e. each tenant defines its own RAG database. |
| Input Text | The primary processing sequence goes anti-clockwise from text input via document preparation, chunk creation to chunk output. If a document reference has been fully processed, i.e. the text chunks have been uploaded to the RAG database, the entire left-hand side with the "Document preparation" and "Chunk creation" is blocked. |
| : : Document reference | Using the context menu on the "..." button, you can either select a document reference already stored in the database or on the document RAG or you can use the file explorer to select a file or a file directory from the system. If a directory has been selected, you can use the left/right buttons to display the individual documents from the directory. |
| Document preparation | |
| : : LLM Sherpa | The texts drawn via the document references are first read in and structured using LLM Sherpa and only then transferred to the chunk creation process. This selection represents the standard procedure for the preparation of documents. Permitted file extensions (file types) are: .docx (Word), .html, .pdf (PDF), .pptx (PowerPoint), .txt and .xml. |
| : : PDF to text | The .pdf files dragged via the document references are first converted into plain text using a simple procedure and only then transferred to the procedure for chunk creation. This selection is only recommended for very simple .pdf texts. |
| : : none (text) | The texts drawn via the document references are transferred directly to the chunk creation process. This applies to pure text files, which are then "chunked" using a manual procedure. If this selection is made in combination with automatic further processing (see below), the "TextChunking" procedure must be implemented on a project-specific basis. |
| Chunk creation | Selection of the procedure for the actual chunk creation, i.e. the splitting of the documents into individual text fragments (chunks). |
| : : Automatic | |
| : : : no editing | The creation of chunks is suspended, i.e. the result of the pure document preparation is output, e.g. after selecting the LLM Sherpa, the .pdf files prepared from it are output in JSON format. This mode can be used to check the quality of the preparation/structure of the text of .pdf files. |
| : : : no structuring | automatic structuring of the chunks - by recognising headers, for example - is suspended, i.e. all lines of text are written one after the other in a chunk. |
| : : Manual | Capturing the programme code of an anonymous procedure. Such a procedure expects the text to be processed as the first (top) call parameter, the chunks vector as the second call parameter (top-1), in which all created document chunks (JSON objects) are collected, and the document vector as the third call parameter (top-2), in which the document metadata is collected as JSON objects The anonymous procedure must not have a return value, as the result of the preparation must be entered in the chunks and the document vector. Use the button next to the text to open the code editor. |
| Generate | |
| : : automatically on text change | If this box is ticked, the selected set of rules is automatically applied each time the input text changes, i.e. the text sections are generated and displayed as text chunks. |
| : Output | |
| : : Text Chunks | If a document reference is selected, an input text is displayed and a procedure is defined, then after pressing the "Generate" button, the displayed input text is processed using the selected document formatting and chunk creation and the text sections generated and possibly extended with text are output as text chunks. The left/right buttons can be used to switch back and forth between the individual text sections. |
| : : Meta data | Output of the meta data of all generated (sub-)documents. The chunks of these sub-documents are all output together in the list of text chunks. |
| : : : Chunks structure | Editing the hierarchy of the text chunks created from the documents using the Structure Explorer. |
| : : : Save chunks | The text chunks that are initially only created transiently are saved and permanently linked to the set of rules. Saving the text chunks is the necessary preliminary stage for uploading the chunks to the RAG database. |
| : : : Initialise database | Indicates whether the RAG database should be initialised before uploading text chunks, i.e. any text chunks already in the database should be deleted. Whether and which documents already exist in the RAG database - of the correspondingly selected AI tenant - can be displayed in the status window of the RAG database query, for example. |
| : : : Upload chunks | The previously saved chunks are uploaded to the RAG database. |
| Characteristics | Input of characteristics to be used when executing the procedure. |
| Document references | |
| Documents | List of document references that are linked to the RAG Rules document that has just been called up - and are therefore to be processed. |
| Action | see buttons |
| Button | Button description |
|---|---|
| Action | |
| : Back | The app is closed |
| Delete | Delete document RAG |
| : New | Create new document RAG |
| Save | Save document RAG |
| Connect | The document RAG is linked to the currently selected document reference, i.e. the RAG rule system is assigned to the document reference for the creation of text chunks. If the document reference is a file/file directory selected via the file explorer, you are asked whether a (global) document reference stored in the database should be created automatically. The RAG rule system can only be assigned to the newly created document reference if this question is answered in the affirmative. |
| Input text | |
| < < | First file |
| < | Previous file |
| > | Next file |
| > > | Last file |
| Output Text Chunks | |
| < < | First chunk |
| < | Previous chunk |
| > | Next chunk |
| > > | Last chunk |
| Characteristics | |
| Create | Create attribute |
| Insert | Insert feature |
| Remove | Remove current feature |
| Procedure | |
| Generate | A currently displayed input text is processed with the rule system/procedure entered. The resulting text sections are displayed as output text chunks. The "Automatic on text change" checkbox displayed in front of the button controls whether the "Generate" should be executed automatically each time the input text changes. |
List document RAG
This app is used to list all saved document RAG objects.
| Menu item | Menu description |
|---|---|
| Edit | - |
| Reset | - |
| Options | - |
| Icon | Toolbar description |
|---|---|
 | Execute multiple search |
 | Reset list |
| Create new document RAG |
 | Edit document RAG |
| Delete document RAG |
| Notes |
| Processes |
| Validity |
 | Change report / Copy to clipboard |
| Field | Field description |
|---|---|
| Filter | |
| Number | |
| : Search term | |
| list | |
| : Standard list | |
| : Report-dependent list | |
| : Number of rows / selected | |
| Action |
| Button | Button description |
|---|---|
| Classification | Select class |
| Advanced search | - |
| Back to | The app is closed |
| Delete | Delete document RAG |
| New | Create new document RAG |
| Edit | Edit document RAG |
Select document RAG
This app is used to select a document RAG object.
| Menu item | Menu description |
|---|---|
| Edit | - |
| Reset | - |
| Options | - |
| Icon | Toolbar description |
|---|---|
| Execute multiple search |
| Reset list |
| Field | Field description |
|---|---|
| Filter | |
| Number | |
| : Search term | |
| list | |
| : Standard list | |
| : Report-dependent list | |
| : Number of rows / selected | |
| Action |
| Button | Button description |
|---|---|
| Classification | Select class |
| Advanced search | - |
| Back to | - |
| Select | Select document RAG |
Default settings
This app is used to maintain the settings of the document RAG editing app.
| Menu item | Menu description |
|---|---|
| Edit | - |
| : Set default settings | - |
| Field | Field description |
|---|---|
| List options | |
| : Search options | |
| : : Immediate transfer in case of a hit | |
| : : Empty list before new search | |
| : : If there are less than 100 objects, display directly. | |
| : : General search | |
| : : Advanced search | |
| : Validity | |
| : : | |
| : : : With exchange object | |
| : : : Without exchange object | |
| : : | |
| : : : Valid objects only | |
| : : : All objects | |
| : Number of objects | |
| : : All objects | |
| : : 100 objects | |
| : : Number of objects | |
| Default settings | |
| Global presets | |
| : User preferences | |
| Action |
| Button | Button description |
|---|---|
| Back button | Close app |
| OK | The default settings are set |
RAG database query
This app is used to query and thus test the quality of the RAG text chunk vector database. After entering a text/query and pressing the "Query" button, the database is queried and the result is displayed as a list of the most suitable texts/documents (chunks) for the query. If the list remains empty, no matching text chunks were found.
The scope of the list of chunks to be processed can be restricted using the threshold value and the maximum number of chunks.
| Field | Field description |
|---|---|
| RAG database query | |
| AI Tenant | Selection of the tenant in the AI Cloud. If this is not set, the entire app is blocked. The configuration of each tenant is completely separate from the configuration of other tenants, i.e. each tenant defines its own RAG database. |
| RAG database query | |
| : : Textbox | Text of the enquiry |
| : Documents | |
| : : List | List of chunks matching the enquiry. If a line is selected, the text of the chunk is displayed below it. If a line has a link, this link can be followed with the right mouse button. |
| : : Threshold value | Enter a value between 0 and 1 as the maximum permissible value for the error value of the chunks to be output. The smaller the error value, the higher the probability that the chunk matches the request. |
| : : maximum number of chunks | Maximum number of chunks to be processed. |
| status | |
| Database | |
| : : Embedding Model | Output of the name of the embedding model used to build the RAG database. |
| : : Last change | Output of the date and time of the last change in the RAG database. |
| : : Number of documents / chunks | Output of the number of documents and the text chunks created and saved from these documents in the RAG database. |
| : : Languages | Output of the languages of the documents. |
| : : Size database | Size of the RAG database in MBytes. |
| : Stored documents | Output of a list of the documents referenced by the text chunks. |
| Action | see buttons |
| Button | Button description |
|---|---|
| Queries | Query database |
| Action | |
| : Back | Close app |
Document Chat Inspector
This app is used to test the vector database built up with the documents and their sets of rules, as well as the LLM(Large Language Model) trained with it. It is therefore the central app for quality assurance for chatting with documents.
The individual building blocks of "chatting" are
- a basic LLM (e.g. Llama-2-13b-hf"
- a vector database built with the documents for RAG
- an adapter to the LLM generated with QLoRa
These three basic modules can be tested individually in this app, i.e. one and the same request can be passed in parallel to the basic LLM, the RAG database and the QLoRA adapter. This in turn can result in measures, e.g. using a different LLM, revising the text chunks for the RAG and/or the parameters of the QLoRA.
In addition to the individual selection of one of the three basic blocks, the enquiry can also be passed to the overall system (KLIO). The expectation is, of course, that all basic building blocks together as a consortium deliver the best result for the enquiry.
After entering a text/request and pressing the "Chat" button, the selected module or the entire system is activated. The result is the response from the block or the entire system as well as the list of the most suitable texts/documents (chunks) from the vector database for the enquiry. If the response box remains empty, the function block or the overall system was unable to provide a response.
The scope of the list of chunks to be processed can be limited by means of the threshold value and the maximum number of chunks.
| Field | Field description |
|---|---|
| AI Tenant | Selection of the tenant in the AI Cloud. If this is not set, the entire app is blocked. Each tenant is completely separate in its configuration from the configuration of other tenants, i.e. each tenant defines its own LLM, its own QLoRA adapter and its own RAG database. The tenant number, the short name and the administration (full) and public API key are displayed in the combo box. If the AI tenant has a public API key, the public chat can be called up directly to the right of the combo box. |
| chat | |
| Textbox | Text of the enquiry |
| Blocks/overall system | Selection of the basic modules LLM/RAG/QLora and/or the overall system KLIO |
| KLIO | Most comprehensive processing of a request (RAG database + QLoRA Adpater to LLM) |
| : : Answer | |
| : : : Textbox | Response text of the KLIO overall system. |
| : : Documents | |
: : : List | List of chunks matching the enquiry. If a line is selected, the text of the chunk is displayed below it. If a line has a link, this link can be followed with the right mouse button. |
| : : : Threshold value | Enter a value between 0 and 1 as the maximum permissible value for the error value of the chunks to be output. The smaller the error value, the higher the probability that the chunk matches the request. |
| : : : maximum number of chunks | Maximum number of chunks to be processed. |
| QLoRA | Processing of a request with QLoRA Adpater to the LLM |
| : : Response | |
| : : : Textbox | Response text of the QLoRA adapter with the LLM. |
| : : Documents | |
: : : List | List of chunks matching the enquiry. If a line is selected, the text of the chunk is displayed below it. If a line has a link, this link can be followed with the right mouse button. |
| : : : Threshold value | Enter a value between 0 and 1 as the maximum permissible value for the error value of the chunks to be output. The smaller the error value, the higher the probability that the chunk matches the request. |
| : : : maximum number of chunks | Maximum number of chunks to be processed. |
| RAG | Processing of a query per RAG database and LLM. For the query of the RAG database without LLM, see RAG database query |
| : : Response | |
| : : : Textbox | Response text of the RAG database and the LLM. |
| : : Documents | |
: : : List | List of chunks matching the enquiry. If a line is selected, the text of the chunk is displayed below it. If a line has a link, this link can be followed with the right mouse button. |
| : : : Threshold value | Enter a value between 0 and 1 as the maximum permissible value for the error value of the chunks to be output. The smaller the error value, the higher the probability that the chunk matches the request. |
| : : : maximum number of chunks | Maximum number of chunks to be processed. |
| : : status | Output of data on the status of the RAG database. |
| : : : Database | |
| : : : : Embedding model | Output of the embedding model used to build the RAG database... |
| : : : : Last change | Output of the last build/update of the RAG database. |
| : : : : Number of documents / chunks | Output of the number of documents and the text chunks split from these documents. |
| : : : : Languages | Output of the languages of the documents loaded in the RAG database. |
| : : : : Database size | Output of the size of the RAG database in MBytes. |
| : : : Stored documents | Output of the documents specified in the chunks. |
| LLM | Processing of a request per LLM configured for the selected tenant. |
| : : Response | |
| : : : Textbox | Response text of the LLM. |
| : : Configuration | Management of the configuration of the LLM. |
| : : : Model | Selection of the LLM. |
| : : : Prompt template | Specification of the prompt to be automatically added to a request. Placeholder for the request is {request}. The various LLMs have been trained for special prompts, e.g. for the "google/gemma-2-9b-it" LLM the prompt should be specified as follows:
|
| Action | see buttons |
| Button | Button description |
|---|---|
| status | Output of the development/deployment status of the selected chat element |
| Chat | Chat with the selected chat element |
| Back to | Close app |
Related topics
Technical Documentation
- Implementation
- Quality Assurance
- Document RAG Editing App
- Document RAG List App
- Document RAG Selection App
- Text RAG basic module
Implementation
classes
_
quality assurance
_
Document RAG Edit App
Module name
documentRagEdit.app
security
In addition to restricting access rights via the class and its data fields, the module can be restricted in its use via some of the received messages.
| Message | Parameters | Function | security |
|---|---|---|---|
| DELETE_DOCUMENT_RAG | |||
| EDIT_DOCUMENT_RAG | |||
| EXEC_DOCUMENT_RAG | |||
| LOAD_DOCUMENT_RAG |
| Message | Parameters | Function | Receive module |
|---|---|---|---|
| ADVANCED_SEARCH | |||
| CREATE_MODULE_SETTINGS | |||
| CREATE_TXN_NOTE | |||
| DELETE_MODULE_SETTINGS | |||
| DOCUMENT_INDEX_SELECTED | |||
| DOCUMENT_RAG_CHANGED | |||
| DOCUMENT_RAG_CREATED | |||
| DOCUMENT_RAG_DELETED | |||
| DOUBLE_CLICK | |||
| EDIT_COMMENT | |||
| EDIT_CONDITIONED_ATTRIBUTE | |||
| EDIT_DOCUMENT_INDEX | |||
| EDIT_DOCUMENT_RAG | |||
| EDIT_FORMULA_ATTRIBUTES | |||
| EDIT_MODULE_SETTINGS | |||
| EDIT_PROCEEDINGS | |||
| EDIT_SLOT_ATTRIBUTE | |||
| EDIT_VALIDITY_BY_DATE | |||
| INITIALIZE | |||
| LIST_DOCUMENT_RAG | |||
| PRINT_REPORT | |||
| SELECT | |||
| SELECT_ATTRIBUTE | |||
| SELECT_DOCUMENT_INDEX | |||
| SELECT_STRUCTURE | |||
| STRUCTURE_SELECTED |
Document RAG Listen-App
Module name
documentRagList.app
security
In addition to restricting access rights via the class and its data fields, the module can be restricted in its use via some of the received messages.
| Message | Parameters | Function | security |
|---|---|---|---|
| LIST_DOCUMENT_RAG |
| Message | Parameters | Function | Receive module |
|---|---|---|---|
| ADVANCED_SEARCH | |||
| CREATE_MODULE_SETTINGS | |||
| DELETE_MODULE_SETTINGS | |||
| DOCUMENT_RAG_DELETED | |||
| DOUBLE_CLICK | |||
| EDIT_COMMENT | |||
| EDIT_DOCUMENT_RAG | |||
| EDIT_MODULE_SETTINGS | |||
| EDIT_PROCEEDINGS | |||
| EDIT_VALIDITY_BY_DATE | |||
| INITIALIZE | |||
| PRINT_REPORT | |||
| SELECT | |||
| SELECT_STRUCTURE | |||
| STRUCTURE_SELECTED |
Document RAG selection app
Module name
documentRagSelect.app
security
In addition to restricting access rights via the class and its data fields, the module can be restricted in its use via some of the received messages.
| Message | Parameters | Function | security |
|---|---|---|---|
| SELECT_DOCUMENT_RAG |
| Message | Parameters | Function | Receive module |
|---|---|---|---|
| ADVANCED_SEARCH | |||
| CREATE_MODULE_SETTINGS | |||
| DELETE_MODULE_SETTINGS | |||
| DOCUMENT_RAG_DELETED | |||
| DOCUMENT_RAG_SELECTED | |||
| DOUBLE_CLICK | |||
| EDIT_COMMENT | |||
| EDIT_DOCUMENT_RAG | |||
| EDIT_MODULE_SETTINGS | |||
| EDIT_PROCEEDINGS | |||
| INITIALIZE | |||
| PRINT_REPORT | |||
| SELECT | |||
| SELECT_STRUCTURE | |||
| STRUCTURE_SELECTED |
Text RAG basic module
Module name
cxDocumentRag.mod
Security
In addition to restricting access rights via the class and its data fields, the module can be restricted in its use via some of the received messages.
| Message | Parameters | Function | Receive module |
|---|---|---|---|
| ADVANCED_SEARCH | |||
| CREATE_MODULE_SETTINGS | |||
| DELETE_MODULE_SETTINGS | |||
| DOCUMENT_RAG_DELETED | |||
| DOUBLE_CLICK | |||
| EDIT_COMMENT | |||
| EDIT_DOCUMENT_RAG | |||
| EDIT_MODULE_SETTINGS | |||
| EDIT_PROCEEDINGS | |||
| INITIALIZE | |||
| PRINT_REPORT | |||
| SELECT | |||
| SELECT_STRUCTURE | |||
| STRUCTURE_SELECTED |
;){kind=link}
;){kind=link}
;){kind=link}