👉 Try the demo: https://github.com/LM-Kit/lm-kit-net-samples/tree/main/console_net/text-analysis/embeddings/multimodal_embeddings

Multimodal Embeddings for C# .NET Applications

🎯 Purpose of the Demo

An interactive console app that embeds text and images into one shared vector space with a single model, Google EmbeddingGemma 2 (embeddinggemma-2), and exposes three cross-modal workflows: cosine matrix, folder search by text, and image-to-tags ranking.

All processing runs on-device.

👥 Industry Target Audience

E-commerce / DAM: text-to-image search over product photos.
Photo apps: "find images matching this phrase" without per-query VLM calls.
Auto-tagging: rank candidate tags against an image by cosine similarity.
Multimodal RAG: align text and image chunks in one index.
Moderation: text-prompt screening against image content.

🚀 Problem Solved

Text-to-image search, image-to-text classification, and reverse image search all collapse to one operation: cosine similarity in a shared embedding space. A model that embeds both text and images makes that possible without any per-query VLM call. The demo wraps that single operation into three menu modes that cover the three most-asked workflows.

💻 Application Overview

Interactive menu (no command-line arguments) with three modes:

Mode	What it does
Matrix	Type captions and image paths, print the full cosine similarity matrix.
Search	Embed every image in a folder once; type repeated text queries; see top-K matches per query.
Tag	Embed one image, type candidate tags, see top-K tags ranked by similarity.
Quit	Exit.

✨ Key Features

One model, two modalities: embeddinggemma-2 embeds text and images into the same 768-dimensional space through one Embedder.
Query embedding for text: Embedder.GetQueryEmbeddings(string) applies the model's trained query prompt to captions, queries and tags, as the publisher pairs text with images.
Image embedding: Embedder.GetEmbeddings(ImageBuffer).
Cross-modal score: Embedder.GetCosineSimilarity(textVec, imageVec).
Reusable image index: search mode embeds the folder once and reuses across queries.

🧠 Models

embeddinggemma-2 (439 MB, text, code and images in one vector space; runs on CPU or GPU).

🛠️ Getting Started

📋 Prerequisites

.NET 8.0 or later

▶️ Running the Application

git clone https://github.com/LM-Kit/lm-kit-net-samples
cd lm-kit-net-samples/console_net/text-analysis/embeddings/multimodal_embeddings
dotnet run

The model loads once at startup. Pick a mode from the menu.

Table of Contents