CHATBOT SUMMY – INTERACTIONWITH STRUCTURED DATA MADE EASY
Our SaaS product inpeek Summarix already provides us with reliable support in managing resumes, project references, skills, customers, and much more, enabling the seamless creation of documents for public tenders. The goal was to further simplify the work of our employees and sales team by making it easier to interact with the structured data in Summarix. This requirement gave rise to Summy, our AI-powered chatbot.
Summy is seamlessly integrated into Summarix and allows you to ask questions such as “Who has experience with Angular?” or “Which project is best suited to demonstrate our expertise with SAP BTP?” at any time.
THE CHALLENGES
inpeek Summarix is built entirely on the SAP Business Technology Platform (BTP) and stores data in the SAP HANA database. Developing and implementing the chatbot based on SAP technologies was therefore a requirement. The main challenge was to achieve high-quality responses even to complex questions. It was also important to us to be able to evaluate the introduction automatically and objectively after each iteration, so as not to be dependent on human assessment and to remain as open-minded as possible.
Another crucial point in the implementation was mapping terms to entities in the existing Summarix system. For example, a user may use “SAP BTP” as a term, but the system has “SAP Business Technology Platform (BTP)” recorded. This mapping must be reliable in order to search for the correct entities.
The third key aspect was context engineering (prompt engineering): All relevant information must be made available to the LLM in the prompt in order to formulate a correct SPARQL query (similar to SQL).
THE CHALLENGES
inpeek Summarix is built entirely on the SAP Business Technology Platform (BTP) and stores data in the SAP HANA database. Developing and implementing the chatbot based on SAP technologies was therefore a requirement. The main challenge was to achieve high-quality responses even to complex questions. It was also important to us to be able to evaluate the introduction automatically and objectively after each iteration, so as not to be dependent on human assessment and to remain as open-minded as possible.
Another crucial point in the implementation was mapping terms to entities in the existing Summarix system. For example, a user may use “SAP BTP” as a term, but the system has “SAP Business Technology Platform (BTP)” recorded. This mapping must be reliable in order to search for the correct entities.
The third key aspect was context engineering (prompt engineering): All relevant information must be made available to the LLM in the prompt in order to formulate a correct SPARQL query (similar to SQL).
OUR APPROACH
We implemented Summy as an agentic AI workflow. To do this, we used large language models (LLMs) provided via SAP Generative AI Hub to understand the respective question, extract entities, write SPARQL queries, and formulate the final answer. The process was implemented as a workflow using the LangGraph library, with certain decisions being made by LLMs (hence Agentic AI). We deliberately gave the system little leeway and only relied on autonomous decisions in specific cases.
We integrated the AI system directly into the existing SAP Cloud Application Programming Model (CAP) backend using the JS version of LangGraph and LangChain. The existing data is stored in the relational SAP HANA Cloud database. We decided to use a knowledge graph and map the information as such. Since SAP launched an Early Adopter Care (EAC) program for its new SAP HANA Knowledge Graph Engine at the same time, we were able to participate directly, test the new product, and have our implementation supported by the SAP team. To create the knowledge graph, we first extracted all data from the SAP HANA Cloud database, then converted it into the knowledge graph using Python, and imported it into the SAP HANA Knowledge Graph Engine. This process is now performed regularly as a job to keep the knowledge graph up to date.
To ensure optimal integration of the chatbot into the existing Fiori front end, we implemented our own chat interface with Angular and then integrated it into the SAP Work Zone as a shell plugin.
In addition to the technical implementation, we placed great emphasis on evaluating the performance of our system. For each change, we used a predefined test data set (questions + corresponding correct answers) to automatically check the change in system performance. The open source solution Langfuse was used for evaluation and monitoring. We also integrated a feature that allows users to mark incorrect answers. These answers are then stored in a dataset and can be analyzed by our AI engineers to fix weaknesses and continuously improve the system.
Our AI engineers explain more details about the technical implementation and architecture in a Medium Blog Post.
OUR APPROACH
We implemented Summy as an agentic AI workflow. To do this, we used large language models (LLMs) provided via SAP Generative AI Hub to understand the respective question, extract entities, write SPARQL queries, and formulate the final answer. The process was implemented as a workflow using the LangGraph library, with certain decisions being made by LLMs (hence Agentic AI). We deliberately gave the system little leeway and only relied on autonomous decisions in specific cases.
We integrated the AI system directly into the existing SAP Cloud Application Programming Model (CAP) backend using the JS version of LangGraph and LangChain. The existing data is stored in the relational SAP HANA Cloud database. We decided to use a knowledge graph and map the information as such. Since SAP launched an Early Adopter Care (EAC) program for its new SAP HANA Knowledge Graph Engine at the same time, we were able to participate directly, test the new product, and have our implementation supported by the SAP team. To create the knowledge graph, we first extracted all data from the SAP HANA Cloud database, then converted it into the knowledge graph using Python, and imported it into the SAP HANA Knowledge Graph Engine. This process is now performed regularly as a job to keep the knowledge graph up to date.
To ensure optimal integration of the chatbot into the existing Fiori front end, we implemented our own chat interface with Angular and then integrated it into the SAP Work Zone as a shell plugin.
In addition to the technical implementation, we placed great emphasis on evaluating the performance of our system. For each change, we used a predefined test data set (questions + corresponding correct answers) to automatically check the change in system performance. The open source solution Langfuse was used for evaluation and monitoring. We also integrated a feature that allows users to mark incorrect answers. These answers are then stored in a dataset and can be analyzed by our AI engineers to fix weaknesses and continuously improve the system.
Our AI engineers explain more details about the technical implementation and architecture in a Medium Blog Post.
THE RESULT
The chatbot is fully integrated into the existing architecture, can be accessed from anywhere in inpeek Summarix, and reliably answers queries from our employees, especially those in the sales team. Essential data for project profiles, employee references, or quotation templates can thus be accessed quickly and conveniently. The solution is continuously being improved and has been in productive operation at inpeek since August 2025.
Due to its innovative approach, the “Summarix + Summy” project was presented as a Success Story by SAP itself in October.
THE RESULT
The chatbot is fully integrated into the existing architecture, can be accessed from anywhere in inpeek Summarix, and reliably answers queries from our employees, especially those in the sales team. Essential data for project profiles, employee references, or quotation templates can thus be accessed quickly and conveniently. The solution is continuously being improved and has been in productive operation at inpeek since August 2025.
Due to its innovative approach, the “Summarix + Summy” project was presented as a Success Story by SAP itself in October.
ABOUT INPEEK
We are a young IT technology company specializing in consulting and development services in the SAP environment. Based on a strong technological foundation, we offer innovative, user-friendly, and practical solutions for a wide range of business areas and industries – and we do so with the utmost agility and passion.
