Programs

This chapter contains some useful programs that show the capabilities of the MBASE library. Their source code directory is given in their program identification section, so dont hesitate to use their implementation as a reference point for your specific implementations use case.

Here are the programs that are implemented:

Benchmark T2T: It is a program written to measure the performance of the given T2T LLM and its impact on your main application logic.

Embedding: An example program for generating the embeddings of the given prompt or prompts.

Openai Server: An Openai API compatible HTTP/HTTPS server for serving LLMs. This program provides chat completion API for TextToText models and embeddings API For embedder models.

Retrieval: An example for calculating the distance between the given query and multiple text files/documents and applying retrieval operation.

Simple Conversation: It is a simple executable program where you are having a dialogue with the LLM you provide. It is useful for examining the answer of the LLM since the system prompt and sampler values can be altered.

Typo Fixer: This is an applied example use case of the MBASE library. The program is reading a user-supplied text file and fixing the typos.