Architecture

MCIP Adapters

Adapters (also called mappers) transform raw product data from any e-commerce platform into MCIP's unified schema. They're the foundation of the Machine Customer protocol — starting with product discovery today, and extending to cart, checkout, and order operations as the protocol evolves. Build one in 30 minutes, connect any store.

The Universal Translator for Commerce

Remember those universal translator devices from sci-fi movies? That's what MCIP adapters do — not just for product data, but for the entire commerce conversation between AI agents and stores.

Here's the challenge: Every e-commerce platform structures data differently. Vendure uses GraphQL with nested variants. Shopify has its own REST schema. WooCommerce does things the WordPress way. Your custom platform? It probably invented its own unique structure.

Without adapters, you'd need to write custom integrations for every platform — an exhausting, never-ending task.

Here's where adapters save the day: They transform raw product data from any source into MCIP's unified UnifiedProduct schema. You write an adapter once, and that platform's products become discoverable through MCIP's semantic search, agentic workflows, and eventually full commerce operations.

Think of It Like Power Adapters

Imagine traveling internationally with your laptop. UK outlets, Japanese outlets, European outlets — all different. But your laptop never changes. You just switch adapters.

That's how MCIP adapters work. The Machine Customer protocol always uses the same unified schema. The adapter handles transforming each platform's unique format. No rewriting your protocol for each store — just plug in the right adapter.

Key Insight: MCIP is a Machine Customer protocol, not just a search engine. Product discovery is the first module. Adapters are designed to grow with the protocol as cart management, checkout flows, and order tracking modules come online.

Where Adapters Sit in the Architecture

Adapters live in Layer 3 (Domain & Infrastructure) of MCIP's three-layer NestJS module architecture:

Layer 1 — Presentation & Protocol
  ├── Search Controller (HTTP Endpoints)
  ├── Hard Filtering Controller (Agentic Search)
  ├── MCP Tools (AI Agent Integration)
  └── Admin Controller (Management API)

Layer 2 — Application Services
  ├── Search Service (Simple Vector Search)
  ├── Hard Filtering Service (LangGraph Workflow)
  ├── Ingestion Service (Product Sync)
  └── Admin Service (System Management)

Layer 3 — Domain & Infrastructure  ← Adapters live here
  ├── Product Repository (Qdrant Client)
  ├── Vectorization Service (OpenAI Embeddings)
  ├── Product Mapper (Platform Adapters)  ← This is the adapter
  └── BullMQ Processor (Async Queue)

The Product Mapper works hand-in-hand with the BullMQ Processor — adapters transform the data, while BullMQ handles reliable async processing with retry logic and job persistence. This separation means adapters stay focused on one job: data transformation.

How Adapters Work Today

The Async Ingestion Pipeline

In the current implementation, adapters are used during product sync (ingestion) — an async, queue-based pipeline powered by BullMQ and Redis. Here's the full flow:

Admin Trigger → Fetch from Source → BullMQ Queue → Adapter.map() → Vectorization → Qdrant Storage

Step 1: Admin Triggers Sync

You call POST /admin/sync with your admin API key. The Ingestion Service (Layer 2) initiates the process.

Step 2: Products Fetched from Source

MCIP fetches raw product data from your configured SOURCE_URL using REST or GraphQL.

Step 3: Jobs Queued in BullMQ 🔄

Each raw product is added to the product-ingestion BullMQ queue backed by Redis. This gives you:

Retry logic: Failed jobs retry automatically (up to 3 attempts)
Job persistence: Queue survives restarts
Async processing: The sync endpoint returns immediately while products process in the background
Scalable throughput: Increase concurrency as needed

Step 4: Adapter Maps Data ✨

The BullMQ Processor picks up each job and runs the adapter's map() method. This is where the magic happens:

Platform field body_html becomes description (stripped of HTML)
Nested variants[0].price becomes normalized price
Custom attributes get extracted to attributes array
Keywords are generated for improved searchability
The result is validated against the Zod-powered UnifiedProductSchema

Step 5: Vectorization

Each normalized product gets converted to a 1536-dimensional vector using OpenAI's text-embedding-3-small model. The embedding captures the semantic meaning of the product's title, description, keywords, and attributes.

Step 6: Storage in Qdrant

Vectorized products are stored in Qdrant with payload indexes on price, category, and brand. These indexes are critical — they enable the hybrid search that makes MCIP powerful.

Step 7: Ready for Search 🔍

Products are now searchable through two complementary modes (see next section).

From Ingestion to Search: What Adapters Enable

Once your adapter has transformed and ingested products, MCIP offers two search modes that leverage the normalized data:

Mode 1: Simple Vector Search

Direct vector similarity search in Qdrant. Fast, low-latency for straightforward queries. No LLM calls — pure embedding comparison.

curl "http://localhost:8080/search?q=laptop"

Mode 2: Agentic Hard-Filtered Search (LangGraph Workflow)

The full 4-stage LangGraph state machine for complex natural language queries:

Parallel Filter Extraction — GPT-4o-mini extracts brand, category, and price constraints in parallel
Brand Validation — Qdrant facet search validates extracted brands against your actual catalog
Hybrid Search — Vector similarity + exact payload filtering (brand, category, price) in Qdrant
LLM Verification — Semantic verification that results match the user's intent

curl "http://localhost:8080/hard-filtering/search?q=nike+shoes+under+100"

Why this matters for adapters: The quality of your adapter's output directly impacts search quality. Well-mapped brand, category, and price fields enable precise payload filtering. Rich keywords and clean description fields improve vector similarity matching. Your adapter is the foundation that makes MCIP's intelligence possible.

The ProductMapper Interface

Every adapter implements the same simple interface:

// src/modules/ingestion/mapper/product-mapper.interface.ts

import { UnifiedProduct } from "../../../domain/product.schema";

export interface ProductMapper {
  map(raw: any): Promise<UnifiedProduct>;
}

That's it. One method. Transform raw data into a UnifiedProduct. The simplicity is intentional — it keeps adapters focused and easy to build.

The Target: UnifiedProduct Schema

Your adapter must produce this structure:

interface UnifiedProduct {
  // Identity
  externalId: string;          // Unique ID from source
  url: string;                 // Product page URL

  // Core Content
  title: string;               // Product name (min 3 chars)
  description: string;         // Plain text, no HTML

  // Categorization (optional but recommended)
  brand?: string;              // "Nike", "Apple", etc.
  category?: string;           // "Laptops", "Shoes", etc.

  // Commercial
  price: {
    amount: number;            // e.g., 99.99
    currency: "UAH" | "USD" | "EUR";
  };

  // Visuals
  mainImage: string;           // Primary image URL

  // Details
  attributes: Array<{
    name: string;
    value: string | number | boolean;
  }>;

  variants: Array<{
    sku: string;
    title: string;
    price: { amount: number; currency: string } | null;
    available: boolean;
  }>;

  // AI Metadata
  keywords: string[];          // 5-10 SEO keywords
}

Tip for hybrid search: The brand, category, and price fields are indexed as Qdrant payloads. The more accurately your adapter populates these, the better the agentic search performs during payload filtering.

Current Adapters

VendureMapper (Default)

The built-in adapter for Vendure e-commerce platform. Handles GraphQL responses with nested variants, facets, and collections.

Location: src/modules/ingestion/mapper/strategies/vendure/vendure.mapper.ts

Key transformations:

Extracts first variant's price (converting from cents)
Maps facetValues to attributes (used in Qdrant payload filtering)
Generates keywords from title, description, and facets (improves vector similarity)
Handles image URL transformation (internal → public URLs)
Validates output against UnifiedProductSchema (Zod)

CustomAiMapper (AI Fallback)

When you have unusual data formats, this adapter uses GPT-4 via LangChain to intelligently map any structure to UnifiedProduct.

Location: src/modules/ingestion/mapper/strategies/custom-ai.mapper.ts

How it works:

Sends raw product JSON to GPT-4 via LangChain's structured output
LangChain enforces Zod schema validation on the response
Infers missing fields (category from title, brand from description)
Returns validated UnifiedProduct

Pros: Works with any data format, LangChain handles retries and parsing

Cons: Slower (~500ms per product), uses API credits, less predictable

When to use it: The CustomAiMapper is great for prototyping and one-off imports. For production, we recommend writing a dedicated adapter — it's faster, cheaper, and more predictable.

Building Your Own Adapter

Step 1: Create the Mapper Class

// src/modules/ingestion/mapper/strategies/shopify/shopify.mapper.ts

import { Injectable, Logger } from "@nestjs/common";
import { ConfigService } from "@nestjs/config";
import { UnifiedProduct, UnifiedProductSchema } from "../../../../../domain/product.schema";
import { ProductMapper } from "../../product-mapper.interface";

@Injectable()
export class ShopifyMapper implements ProductMapper {
  private readonly logger = new Logger(ShopifyMapper.name);

  constructor(private readonly configService: ConfigService) {}

  async map(raw: any): Promise<UnifiedProduct> {
    // 1. Validate input
    if (!raw || typeof raw !== "object" || !raw.id) {
      throw new Error("Invalid Shopify product: missing ID");
    }

    this.logger.debug(`Mapping Shopify product: ${raw.title}`);

    // 2. Extract base price from first variant
    const basePrice = raw.variants[0]
      ? parseFloat(raw.variants[0].price)
      : 0;

    // 3. Build unified product
    const storefrontUrl = this.configService.get<string>("STOREFRONT_URL", "");
    
    const product: UnifiedProduct = {
      externalId: String(raw.id),
      url: `${storefrontUrl}/products/${raw.handle}`,
      title: raw.title,
      description: this.stripHtml(raw.body_html || ""),
      brand: raw.vendor || undefined,
      category: raw.product_type || undefined,
      price: {
        amount: basePrice,
        currency: this.getCurrency(),
      },
      mainImage: raw.images[0]?.src || "",
      attributes: this.extractAttributes(raw),
      variants: this.mapVariants(raw.variants, basePrice),
      keywords: this.generateKeywords(raw),
    };

    // 4. Validate with Zod schema
    return UnifiedProductSchema.parse(product);
  }

  private getCurrency(): "UAH" | "USD" | "EUR" {
    const currency = this.configService.get<string>("STORE_CURRENCY", "USD");
    if (["UAH", "USD", "EUR"].includes(currency)) {
      return currency as "UAH" | "USD" | "EUR";
    }
    return "USD";
  }

  private stripHtml(html: string): string {
    return html.replace(/<[^>]*>?/gm, "").trim();
  }

  // ... helper methods for extractAttributes, mapVariants, generateKeywords
}

Step 2: Register in the Module

Add your mapper to the ingestion module's factory:

// src/modules/ingestion/ingestion.module.ts

{
  provide: PRODUCT_MAPPER,
  useFactory: (
    configService: ConfigService,
    customAiMapper: CustomAiMapper,
    vendureMapper: VendureMapper,
    shopifyMapper: ShopifyMapper  // Add your mapper
  ) => {
    const storeProvider = configService.get<string>("STORE_PROVIDER", "VENDURE");
    
    switch (storeProvider.toLowerCase()) {
      case "vendure":
        return vendureMapper;
      case "shopify":           // Add case
        return shopifyMapper;
      default:
        return customAiMapper;
    }
  },
  inject: [ConfigService, CustomAiMapper, VendureMapper, ShopifyMapper],
}

Step 3: Configure Environment

Set STORE_PROVIDER to use your adapter:

STORE_PROVIDER=SHOPIFY
SOURCE_URL=https://your-store.myshopify.com/admin/api/2024-01/products.json
SOURCE_API_KEY=your-shopify-api-key
STORE_CURRENCY=USD
STOREFRONT_URL=https://your-store.myshopify.com

Step 4: Sync and Test

# Trigger async sync (jobs queued in BullMQ)
curl -X POST http://localhost:8080/admin/sync \
  -H "x-admin-api-key: your-secret-key"

# Wait for BullMQ to process jobs, then test simple search
curl "http://localhost:8080/search?q=laptop"

# Test agentic search with filter extraction
curl "http://localhost:8080/hard-filtering/search?q=nike+shoes+under+100"

Common Mapping Patterns

Price Normalization

Many platforms store prices in cents:

private normalizePrice(rawPrice: number): number {
  // Vendure stores 12999 = $129.99
  return rawPrice / 100;
}

HTML Stripping

Always clean HTML from descriptions:

private stripHtml(html: string): string {
  if (!html) return "";
  return html
    .replace(/<script\b[^<]*(?:(?!<\/script>)<[^<]*)*<\/script>/gi, "")
    .replace(/<style\b[^<]*(?:(?!<\/style>)<[^<]*)*<\/style>/gi, "")
    .replace(/<[^>]*>/g, "")
    .replace(/&nbsp;/g, " ")
    .trim();
}

Keyword Generation

Generate searchable keywords from product content:

private generateKeywords(product: RawProduct): string[] {
  const text = `${product.title} ${product.description}`.toLowerCase();
  const stopWords = ["the", "and", "for", "with", "this"];
  
  const words = text
    .replace(/[^\w\s]/g, "")
    .split(/\s+/)
    .filter((w) => w.length > 3)
    .filter((w) => !stopWords.includes(w));
  
  return [...new Set(words)].slice(0, 10);
}

Image URL Transformation

Handle internal vs public URLs:

private transformImageUrl(url: string): string {
  const internalHost = this.configService.get("VENDURE_INTERNAL_URL");
  const publicHost = this.configService.get("VENDURE_API_URL");
  
  if (internalHost && publicHost && url.includes(internalHost)) {
    return url.replace(internalHost, publicHost);
  }
  return url;
}

Adapter Directory Structure

src/modules/ingestion/mapper/
├── product-mapper.interface.ts    # The interface all adapters implement
└── strategies/
    ├── custom-ai.mapper.ts        # AI-powered fallback (LangChain + GPT-4)
    └── vendure/
        ├── types.ts               # Platform-specific types
        └── vendure.mapper.ts      # The mapper implementation

# When adding your adapter, follow this structure:
strategies/
└── shopify/
    ├── types.ts           # Shopify-specific interfaces
    └── shopify.mapper.ts  # Your mapper

Testing Your Adapter

// shopify.mapper.spec.ts

describe("ShopifyMapper", () => {
  let mapper: ShopifyMapper;

  beforeEach(async () => {
    const module = await Test.createTestingModule({
      providers: [
        ShopifyMapper,
        {
          provide: ConfigService,
          useValue: {
            get: jest.fn((key) => {
              const config = {
                STOREFRONT_URL: "https://example.com",
                STORE_CURRENCY: "USD",
              };
              return config[key];
            }),
          },
        },
      ],
    }).compile();

    mapper = module.get<ShopifyMapper>(ShopifyMapper);
  });

  it("should map a valid Shopify product", async () => {
    const rawProduct = {
      id: 123,
      title: "Cool T-Shirt",
      body_html: "<p>A very cool shirt</p>",
      vendor: "Acme",
      product_type: "Apparel",
      handle: "cool-t-shirt",
      images: [{ src: "https://cdn.shopify.com/image.jpg" }],
      variants: [
        { id: 1, sku: "TSHIRT-S", title: "Small", price: "29.99", available: true },
      ],
      tags: "summer,cotton",
    };

    const result = await mapper.map(rawProduct);

    expect(result.externalId).toBe("123");
    expect(result.title).toBe("Cool T-Shirt");
    expect(result.description).toBe("A very cool shirt"); // HTML stripped
    expect(result.brand).toBe("Acme");
    expect(result.price.amount).toBe(29.99);
    expect(result.price.currency).toBe("USD");
  });

  it("should throw on invalid input", async () => {
    await expect(mapper.map(null)).rejects.toThrow();
    await expect(mapper.map({})).rejects.toThrow();
  });
});

Future: From Mappers to Full Platform Connectors

MCIP is a Machine Customer protocol — product discovery is just the first step. As the protocol evolves to support cart management, checkout, and order tracking, adapters will grow from ingestion mappers into full platform connectors.

Current: ProductMapper (Ingestion)

What you've seen in this document — transforms raw product data for async ingestion via BullMQ.

interface ProductMapper {
  map(raw: any): Promise<UnifiedProduct>;
}

Next: IProductService (Real-Time Platform Queries)

The architecture is designed to support real-time translation where searches go directly to source platforms. This interface enables multi-store aggregation with always-fresh data:

interface IProductService {
  searchProducts(query: string, options?: SearchOptions): Promise<Product[]>;
  getProduct(id: string): Promise<Product>;
  getProductBySlug(slug: string): Promise<Product>;
}

Future: Full Commerce Lifecycle

As MCIP adds commerce modules, adapters will extend to handle:

Cart Operations — Add to cart, update quantities, view cart across platforms
Checkout Flows — Initiate checkout, apply coupons, handle payments
Order Tracking — Order status, shipping updates, returns
Capability Discovery — Each adapter reports what operations it supports

// Planned: Full platform connector interface
interface IPlatformConnector {
  // Product Discovery (current)
  searchProducts(query: string, options?: SearchOptions): Promise<Product[]>;
  getProduct(id: string): Promise<Product>;
  
  // Cart Management (planned)
  addToCart(productId: string, quantity: number): Promise<Cart>;
  getCart(): Promise<Cart>;
  
  // Checkout (planned)
  initiateCheckout(cart: Cart): Promise<CheckoutSession>;
  
  // Capabilities
  capability_descriptor(): CapabilityDescriptor;
}

The current ProductMapper interface will continue to work for ingestion scenarios. Building an adapter today positions your platform for the full protocol evolution.

Extension Points Summary

Extension Point	Current	Future Capability	Implementation Path
Platform Adapters	Vendure only	Any e-commerce API	Implement IProductService
Search Methods	Vector + Agentic	Hybrid search strategies	Pluggable search strategies
AI Models	OpenAI only	Multiple providers	Abstract embedding service
Cart Storage	Redis only	Multiple backends	Storage adapter pattern
Protocol	MCP only	GraphQL, REST APIs	Protocol adapters

Common Questions

Do I need to modify my existing platform?

No! Adapters work with your existing APIs. If your platform has an API, you can write an adapter for it.

What if my API changes?

Update your adapter. Products will be re-synced on the next POST /admin/sync call. BullMQ handles the async processing with retry logic.

Can I restrict what data gets indexed?

Absolutely. Your adapter controls exactly what data flows through. Filter products, hide prices, restrict categories — your adapter, your rules.

How often should I sync?

Depends on how often your catalog changes. Daily syncs work for most stores. For fast-changing inventory, consider more frequent syncs or webhook-triggered updates.

Does the adapter affect search quality?

Yes! Well-mapped brand, category, and price fields enable precise hybrid filtering in the agentic search. Rich keywords and clean description fields improve vector similarity. Your adapter is the foundation of search quality.

What happens if my adapter throws an error?

BullMQ retries the job automatically (up to 3 attempts). If it still fails, the job is marked as failed and logged. Other products continue processing — one failure doesn't block the pipeline.

Menu

Get started