Document Chat QLoRA Adapter
Technical description
This app is used to finetune an LLM for chatting with documents using a QLoRA adapter. The QLoRA adapter can either be set up using scored documents - with automatically generated questions and answers - or with manually created questions and answers to a manually entered text.
Functionality
QLoRA Adapter
This window is used to configure a QLoRA adapter for chatting with documents.
| Field | Field description |
|---|---|
| AI Tenant | Specification of the AI tenant for which the QLoRA adapter is to be edited. After selection, the document chunks and/or the manually maintained input/output example training data uploaded to this AI tenant are listed. |
| Documents | |
| Chunked documents | Output of the document chunks of the selected AI tenant. |
| Input/Output | |
| List | List of manually edited input/output examples for the fine tuning training of the LLM. |
| : Example | Maintenance of a training example. The individual fields are formatted as "input" and "output" fields for the training according to the so-called Alpaca format as follows: input {Text from Pre-Instruction field} + "### Instruction:" + {Text from Instruction field} + "Input:" + {Text from Input field} + "### Response:" + Example: "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\nWhat is Alice doing on the bank?\n\n### Input:\nAlice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do: once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, "and what is the use of a book," thought Alice "without pictures or conversations?"\n\n### Response:" Output {Text from Output field} Example: "Alice is sitting by her sister on the bench." |
| : : Pre-instruction | Pre-instruction text. As most LLMs are trained in English, only a monolingual text in English can/should be recorded here. |
| : : Instruction | Instruction text, i.e. here the actual question about the context, which is recorded as input (see below). |
| : : Input | Input text, i.e. here the text that is given as context for the instruction (question). |
| : : Output | Output Text, i.e. the intended answer to the question (instruction) and the context (input). |
| Configuration | |
| : Question prompt | Specification of the question prompt to automatically generate questions when training with documents. |
| : Answer prompt | Specification of the answer prompt to be able to automatically generate answers (to the automatically generated questions) when training with documents. |
| Pre-instruction | This text is pre-assigned to the Pre-instruction field (see above) when a new example training record is created. |
| QLoRA status | Output of data on the status of the QLoRA adapter. |
| Button | Button description |
|---|---|
| Back button | Close app |
| Fine tune | Depending on the selected tab window (Documents or Input/Output), the QLoRA adapter is set up with the scored documents or the manually entered question/answer examples. The setup of the QLoRa adapter or the fine tuning of the LLM can take up to 80 hours, whereby the status can be queried again and again in the meantime. |
Related topics
Technical documentation
Implementation
classes
_
Quality assurance
_
QLoRA Adapter App
Module name
documentChatQloraAdapterEdit.app
security
In addition to restricting access rights via the class and its data fields, the module can be restricted in its use via some of the received messages.
| Message | Parameters | Function | security |
|---|---|---|---|
| EDIT_DOCUMENT_CHAT_QLORA_ADAPTER |
;){kind=link}
;){kind=link}
;){kind=link}