Table of Contents

💬 Multi-Turn Chat with Custom Sampling Demo Overview


🎯 Purpose of the Sample

The Multi-Turn Chat with Custom Sampling Demo demonstrates how to use the LM-Kit.NET SDK to create an interactive chatbot with customized sampling strategies. This sample showcases the integration of large language models (LLMs) into a .NET application to facilitate multi-turn conversations with advanced sampling techniques to influence the chatbot's responses.


👥 Industry Target Audience

This sample is particularly beneficial for developers and organizations in the following sectors:

  • 🛠️ Customer Support: Provide tailored responses based on specific customer needs and preferences.
  • 📚 Education: Develop interactive learning assistants that can adapt to different educational contexts and styles.
  • 🩺 Healthcare: Create chatbots that can respond appropriately to patient inquiries by adjusting the language model's output dynamically.
  • 🛒 E-commerce: Enhance customer engagement by delivering personalized and context-aware responses.

🚀 Problem Solved

Generating responses that are contextually relevant and stylistically appropriate can be challenging, especially when dealing with diverse conversational contexts. The Multi-Turn Chat with Custom Sampling Demo addresses this problem by leveraging advanced sampling techniques to customize the chatbot's responses, ensuring that they are both relevant and engaging.


💻 Sample Application Description

The Multi-Turn Chat with Custom Sampling Demo is a console application that allows users to interact with a chatbot designed to adapt its responses using various sampling strategies and biases.

✨ Key Features

  • 📦 Model Selection: Users can choose from predefined models or provide a custom model URI.
  • 📊 Progress Tracking: The application displays download and loading progress for the selected model.
  • 🎲 Custom Sampling: The application uses advanced sampling strategies such as top-k, top-p, temperature, and others to influence the chatbot's responses.
  • ⚖️ Bias Adjustment: Adjusts the logit biases to encourage or discourage specific topics in the chatbot's responses.
  • 📈 Performance Metrics: Displays generated tokens, stop reasons, quality score, and speed of response for each interaction.
  • 🔧 Special Commands: Users can reset the conversation using the /reset command or regenerate responses with the /regenerate command.

🧠 Supported Models

The sample supports several state-of-the-art models:

  • Mistral Nemo 2407 12.2B
  • Meta Llama 3.1 8B
  • Google Gemma2 9B Medium
  • Microsoft Phi-3 3.82B Mini
  • Alibaba Qwen-2 7.6B

🛠️ Getting Started

📋 Prerequisites

  • .NET Framework 4.6.2 or .NET 6.0

📥 Download the Project

▶️ Running the Application

  1. 📂 Clone the repository:

    git clone https://github.com/LM-Kit/lm-kit-net-samples.git

  2. 📁 Navigate to the project directory:

    cd lm-kit-net-samples/console_framework_4.62/multi_turn_chat_with_custom_sampling

    or

    cd lm-kit-net-samples/console_net6/multi_turn_chat_with_custom_sampling

  3. 🔨 Build and run the application:

    dotnet build
dotnet run

  4. 📜 Follow the on-screen prompts to select a model and start the multi-turn chat.

💡 Example Usage

  1. Select a Model: Choose from the available models or enter a custom model URI.
  2. Initiate Conversation: Start by entering a prompt, such as "Hello!".
  3. Receive Custom Responses: The chatbot will generate responses influenced by the custom sampling strategies and biases.
  4. Continue Interaction: Enter another prompt or use the special commands (/reset or /regenerate) to manage the conversation.
  5. End the Chat: Submit an empty input to end the program.

By following these steps, developers can explore the functionalities of LM-Kit.NET and integrate advanced sampling strategies into their chatbot applications, enhancing the relevance and engagement of automated responses through a multi-turn conversational AI.