Inference Workflow in General

Important

This page is incomplete. Make sure to follow the examples such as About Single-Prompt Example, About Dialogue Example and Embeddings Example.

The inference workflow is same and simple for all types of models including the ones that are not implemented yet.

Here is the flow of operations user should employ to achieve non-blocking inference:

• Initialize the model object properly. See Model Object in Detail, and Initializing Model.

• Initialize the processor object properly. See Processor Object in Detail, and Initializing Processor.

• Initialize the client object properly. See Client Object in Detail and Initializing Client.

• Invoke their respective methods by employing the Signal-driven Parallel State Machine