Table of Contents

🛠️ What is Function Calling?


📄 TL;DR:
Function calling enables Large Language Models (LLMs) to execute predefined functions during inference. In LM-Kit.NET SDK, this feature allows models to interact with APIs, external services, or custom logic through code execution. While most models require specialized pretraining to support function calling, LM-Kit.NET introduces a groundbreaking sampling pipeline that allows any model—regardless of size, including smaller models—to effectively leverage function calling with state-of-the-art accuracy and speed. This eliminates reliance on commercial or large models for this capability.


📚 Function Calling

Definition:
In the LM-Kit.NET SDK, function calling refers to the model’s ability to invoke external or custom functions during inference based on a natural language input. Typically, models must be pre-trained specifically for function calling, a capability that is often limited to larger, commercially available models. However, LM-Kit.NET SDK resolves this limitation by implementing an advanced sampling pipeline, enabling any model to perform function calling with high precision and efficiency, regardless of its size.

For instance, the SingleFunctionCall class allows developers to dynamically invoke a function by analyzing user inputs, extracting relevant parameters, and triggering the appropriate method, whether interacting with APIs or executing custom logic. This approach provides enhanced flexibility and application reach without the constraints of relying on pre-trained, large-scale models.


🔍 The Role of Function Calling in LLMs:

  1. Model Pretraining Requirements:
    Traditionally, LLMs must undergo specific pretraining to support function calling, a feature found primarily in large, commercial models. This training enables the model to understand how and when to invoke external functions during text generation.

  2. LM-Kit’s Innovation:
    LM-Kit.NET overcomes the dependency on pre-trained models by implementing an innovative sampling pipeline. This pipeline allows function calling to be performed accurately with any model, from tiny to large-scale, without requiring the model to have prior specialized training. This is achieved by dynamically handling function selection and parameter extraction with superior performance.

  3. Dynamic Function Execution:
    By allowing models to invoke functions based on user input, function calling extends the LLM’s capabilities beyond text generation, making it suitable for handling real-time tasks such as data retrieval, API interaction, and executing complex logic.

  4. Enhanced Accessibility:
    Thanks to LM-Kit.NET’s flexible architecture, even smaller models, which typically lack function calling support, can now perform this task with state-of-the-art accuracy and speed, making advanced functionality accessible in resource-constrained environments.


⚙️ Practical Application in LM-Kit.NET SDK:

In LM-Kit.NET SDK, function calling is facilitated by several key classes and attributes that allow developers to register and invoke functions based on natural language prompts.

  1. SingleFunctionCall Class:
    The SingleFunctionCall class serves as the core interface for executing a single method based on user input. It allows dynamic function invocation, function registration, and parameter extraction through natural language understanding. The class supports both synchronous and asynchronous execution of functions and provides tools to register methods using the LMFunctionAttribute.

    • Function Registration:
      Functions marked with LMFunctionAttribute can be automatically registered via ImportFunctions(), making them accessible for dynamic invocation by the LLM during inference. This allows developers to expose specific methods for real-time execution based on user commands.

    • Real-Time Function Execution:
      The Submit() and SubmitAsync() methods allow developers to trigger functions either synchronously or asynchronously, enabling models to execute external logic or API calls dynamically as part of the conversation flow.

  2. LMFunctionAttribute:
    The LMFunctionAttribute class provides a declarative way to expose methods to the LLM for invocation during inference. This attribute marks methods that the language model can call, ensuring a seamless function integration within the model's response generation process.

  3. Advanced FC-Inference Pipeline:
    Unlike traditional approaches requiring specialized model pretraining for function calling, LM-Kit.NET’s FC-Inference pipeline intelligently processes user inputs and dynamically determines which functions to invoke, ensuring high-accuracy results. This method allows even smaller models to perform function calling tasks with remarkable precision.

  4. Function Call Result Handling:
    The FunctionCallResult class encapsulates the outcome of a function invocation, including the method called, the parameters passed, and the resulting output. This class allows developers to capture and handle the results of function calls effectively, ensuring the application responds appropriately to user interactions.


📖 Common Terms:

  • Function Pretraining: Typically, function calling requires specialized model training to teach the model how to invoke external functions, limiting its availability to large, commercial models.

  • Dynamic Invocation: The ability to call specific functions at runtime based on natural language prompts, including determining the correct parameters from user input.

  • Custom Functions: Functions defined by the developer, marked with LMFunctionAttribute, which can be invoked by the LLM during the function-calling process.

  • FC-Inference Pipeline: A sophisticated technology used in LM-Kit.NET that enables models of any size to perform function calling accurately, bypassing the need for specialized pretraining.


🔗 Related Concepts:

  • Edge Inference: All function calls in LM-Kit.NET occur on the edge (locally on the device), ensuring efficient performance and data security without reliance on cloud-based services.

  • Task Automation: Through function calling, LM-Kit.NET enables automation of tasks such as data retrieval, scheduling, and process control, making the LLM a powerful tool in interactive applications.

  • External Service Integration: The ability to connect LLMs with external APIs and services through function calls, allowing dynamic real-time data access and functionality.

  • Language Understanding: The model’s ability to parse natural language input to determine when a function call is needed, extracting parameters for proper function invocation.


📝 Summary:

Function calling in LM-Kit.NET SDK allows LLMs to perform dynamic actions such as API interactions, data retrieval, and executing custom logic. Unlike most models, which require specialized pretraining for function calling (available primarily in large, commercial offerings), LM-Kit.NET introduces a cutting-edge sampling pipeline that allows any model—regardless of size—to support function calling with state-of-the-art accuracy and speed. This flexibility empowers developers to integrate function calling into resource-constrained environments without sacrificing performance. The SDK’s powerful tools, including SingleFunctionCall and LMFunctionAttribute, facilitate dynamic method invocation, making function calling a versatile and accessible feature for LLM-driven applications.