Implementation of the project entitled “R&D work on a system based on LLM/GPT, enabling the use of the source language to obtain answers to questions arising from access to confidential organizational data.”
AUTOMADE Sp. z o. o. has been subsidized by the European Union to implement the project entitled “R&D work on a system based on LLM/GPT, enabling the use of the source language to obtain answers to questions arising from access to confidential organizational data.”
The aim of the project is to develop an architecture, implementation methodology and create a prototype of a system that will enable the implementation of solutions based on Large Language Models (LLM) / Generative Pretrained Transformers (GPT) in an On Premise model (not cloud-based) with the possibility of using systems and data available only within the organization, without making confidential data and internal systems publicly available (to the cloud).
The project is planned to include the following tasks:
1. Research and development work on key algorithms and system architecture.
This task will involve the following activities:
a) developing a system architecture that allows for the interconnection of RPA and API solutions as connectors to business systems and LLM as the inference engine that generates responses for the user;
b) evaluating and selecting an LLM solution appropriate for on-premise implementation and developing a proprietary solution that will enable the adaptation of the selected LLM model to address the key design challenge of meeting the user’s goal using data retrieved from external systems via connectors;
c) developing a proprietary algorithm that selects the appropriate connectors from a pool of pre-defined connectors based on the user’s goal (query);
d) developing a proprietary algorithm that invokes connectors with the appropriate parameters and retrieves data into the context of the chat conversation;
e) developing a proprietary algorithm that summarizes the information retrieved into the context of the conversation to formulate a precise response for the user.
2. Development of a prototype system and testing of the system in production-like conditions
This task will include the following activities:
a) Development of a functional prototype. Development of an implementation architecture and Dockerization of services;
b) Development of procedures for building, releasing, and deploying the system based on a CI/CD environment. The task includes developing solutions enabling the release of subsequent product versions along with automated solution deployment to customers based on CI/CD environments, Docker, Ansible, etc.;
c) Development of tools for system administration, model management, prompt management, and conversation flow management;
d) Development of administrative tools for managing and creating connectors to external systems;
e) Development of a regression testing and test automation environment. The task involves developing test scenarios for the prototype and the regression tests based on them, which will enable further development of the system without compromising quality;
f) Conducting production testing of the system in a client environment for at least two client environments;
g) Further improving the developed algorithms. Throughout this stage, ongoing work will be conducted iteratively to verify the quality of the system’s performance, particularly in the context of ongoing development work. A key step will be drawing and applying conclusions from system testing in the production environment of potential clients – it should be assumed that new conclusions may emerge that will need to be incorporated into the system’s algorithms;
h) Documenting the new product – developing a product concept and an implementation methodology for the final product. This is a summary task, which will result in the development of an implementation methodology encompassing general principles for implementing, configuring, and using the system, an analysis of the potential business benefits for clients, and instructions and other materials enabling further development and use of the product prototype.
Project Results:
As a result of the project, a product innovation will be implemented in the form of a new product: a system that enables the use of natural language to generate answers to questions requiring access to confidential data.
Target groups: The software planned for development will be offered to organizations that implement business processes involving personal/sensitive/confidential data, or other data that cannot be transferred to a third-party cloud, such as entities in the medical or financial sectors, or companies using legacy systems inaccessible to cloud solutions. The sales business model will be based on purchasing access (subscription) to use the tool for a predefined number of queries.
#EUFunds #EuropeanFunds
Project value: 6 562,771, 75 PLN
Contribution from European Funds: 4 999,999,99 PLN
