Introduction to insert at the very top

I have been working on an enterprise project where speed is critical. That pushed me toward static generation, aggressive caching, and a solid cache invalidation strategy. Next.js is a great fit for this because you can cache both the full route and the data layer.

Because I’m using Payload CMS, the tricky part was the initial fetch for each statically generated page. After some iteration, I landed on a single initial fetch that gathers everything needed to build a page in one go. After that, Next.js can render, cache, and serve efficiently.

This approach worked well in production builds. The first real trouble showed up in Live Preview, where static generation is bypassed and pages fetch fresh data on each request. I started seeing lag, and after investigating I noticed the network response exceeded 10 MB. Another sign was that Next.js caching stopped working because the response was too large.

That led me to a more targeted approach: fetch lean data by default, then enrich only what each block needs, in batches, while keeping payloads small enough for caching to stay reliable.

---

How-To Guide: Data Enrichment Approach for Payload CMS

A guide to implementing targeted data enrichment to keep payloads under 2 MB, avoid N+1 queries, and preserve performance.

---

Table of Contents

1. Introduction & Problem Statement
2. Core Concepts & Mental Model
3. Implementation Patterns
4. Building Blocks
5. Performance Section
6. Testing & Troubleshooting
7. Quick Reference
8. Reference Documentation
9. Conclusion & Next Steps

---

Introduction & Problem Statement

The Payload depth dilemma

Payload CMS lets you fetch related data via a depth parameter. It looks like an easy solution:

// Fetch everything by setting a high depth
const pages = await payload.find({
  collection: 'pages',
  depth: 5,
  ...
});

The problem is payload size. In the client project, a single page query with depth: 3-5 produced 20 MB or more. When Next.js tries to cache this with unstable_cache, caching can fail because there is a hard limit of 2 MB per cached value. When that happens:

• Caching stops working
• Every page load requires a full database query
• Performance degrades
• Server costs go up

The hidden cost of deep fetching

Payload size is only part of it. Deep fetching also tends to introduce performance issues that are easy to miss at first.

1. N+1 query patterns

Deeply nested relationships, especially media, can lead to query fan-out:

Database Query 1: Get projects
  → queries to fetch media relationships individually
  → queries for industries
  → queries for division logos

In the Featured Projects gallery, this showed up as tab switching lag because each interaction triggered extra relationship work.

2. Memory and serialization overhead

Large responses use more memory during processing, increase bandwidth, and slow down serialization and deserialization. Even if the database is fast, the request can still feel slow.

3. Cache invalidation cascades

When you cache a page built from deeply nested data, small changes can invalidate large caches. You lose precision and you pay the cost more often than you need to.

Results from this project

These were the outcomes after switching to targeted enrichment:

What changed:

• Reduced global page depth from 3 to 5 down to 1
• Added block-specific enrichment for the relationships that actually matter
• Used batch fetching for relationships
• Kept Next.js caching within practical limits
• Reduced repeated relationship work during interactions

---

Core Concepts & Mental Model

The three-step enrichment pattern

Data enrichment is a consistent three-step process:

Step 1: Fetch lean
  - Page at depth 0 to 1 (minimal data)
  - Relationships are IDs
  - Typical payload: 50 to 200 KB

Step 2: Extract and deduplicate IDs
  - Collect ID references from a block
  - Remove duplicates with a Set
  - Prepare IDs for a batch fetch

Step 3: Batch enrich
  - One query for all missing data
  - Build an ID to object map for fast lookup
  - Replace ID references with full objects
  - Keep payload comfortably below 2 MB

ASCII data flow

┌─────────────────────────────────────────────────────────────┐
│ PAGE FETCH (depth: 1)                                       │
│                                                             │
│ ┌──────────────┐  ┌──────────────┐  ┌──────────────┐       │
│ │ Block 1      │  │ Block 2      │  │ Block 3      │       │
│ │ media: 42    │  │ cta_ids: [5] │  │ image: 12    │       │
│ │ logo: 55     │  │              │  │              │       │
│ └──────────────┘  └──────────────┘  └──────────────┘       │
└─────────────────────────────────────────────────────────────┘
                           │
                    EXTRACT AND COLLECT
                           │
        ┌──────────────────┼──────────────────┐
        │                  │                  │
    Media IDs          CTA IDs           Image IDs
    [42, 55, 12]       [5]               [12]
        │                  │                  │
        └──────────────────┼──────────────────┘
                           │
                  DEDUPLICATE AND BATCH
                           │
        ┌──────────────────┼──────────────────┐
        │                  │                  │
    getMediaImages     getCtasByIds         DEDUP
    [42, 55, 12]       [5]              [42, 55, 12, 5]
        │                  │                  │
        └──────────────────┴──────────────────┘
                           │
                    MAP BY ID (O(1))
                           │
        ┌──────────────────┼──────────────────┐
        │                  │                  │
    mediaMap           ctaMap
    42 → {...}         5 → {...}
    55 → {...}
    12 → {...}
        │                  │
        └──────────────────┼──────────────────┘
                           │
                 REPLACE REFERENCES
                           │
┌─────────────────────────────────────────────────────────────┐
│ ENRICHED BLOCKS (ready for components)                      │
│                                                             │
│ ┌──────────────┐  ┌──────────────┐  ┌──────────────┐       │
│ │ Block 1      │  │ Block 2      │  │ Block 3      │       │
│ │ media: {url} │  │ cta_ids: [{}]│  │ image: {url} │       │
│ │ logo: {url}  │  │              │  │              │       │
│ └──────────────┘  └──────────────┘  └──────────────┘       │
└─────────────────────────────────────────────────────────────┘

Decision tree: when to use enrichment

Do I need a deep relationship?

YES:
  Does it affect more than 3 blocks?
    NO: Enrich inside the block
    YES: Enrich in the fetcher layer

NO:
  Keep it as an ID (do not enrich)

What type of relationship?
  - Media: use Pattern 1
  - Direct ref: use Pattern 2 or 5 if combined
  - Polymorphic: use Pattern 3

Enrichment vs alternatives

---

Implementation Patterns

This section covers the patterns used across the codebase, with code examples and explanations.

Pattern 1: simple media enrichment

Use when a block has one or a few media fields per item, such as logo, image, or featuredImage.

Examples:

• Featured Clients: logo and logoLight
• Featured Articles: heroImage
• Featured Solutions: image

Featured Clients implementation

Block location: src/components/blocks/featured-clients/index.tsx (lines 47-73)

The page fetch returns logos as IDs:

// After page fetch (depth: 1)
selectedClients = [
  { id: 1, name: "Acme Corp", logo: 42, logoLight: 43 },
  { id: 2, name: "TechVision", logo: 55, logoLight: 56 },
];

The component needs full media objects. Here is the enrichment function:

async function enrichClientLogos(
  block: FeaturedClientsBlock,
): Promise<FeaturedClientsBlock> {
  const selectedClients = Array.isArray(block.selectedClients)
    ? block.selectedClients
    : [];

  const mediaIds = new Set<number>();

  selectedClients.forEach((client) => {
    if (!isObject(client)) return;
    const { logo, logoLight } = client;
    if (typeof logo === "number") mediaIds.add(logo);
    if (typeof logoLight === "number") mediaIds.add(logoLight);
  });

  if (mediaIds.size === 0) return block;

  const mediaResult = await getMediaImages(Array.from(mediaIds));

  const mediaById = new Map(mediaResult.docs.map((media) => [media.id, media]));

  const enrichedClients = selectedClients.map((client) => {
    if (!isObject(client)) return client;

    const logo =
      typeof client.logo === "number"
        ? (mediaById.get(client.logo) ?? client.logo)
        : client.logo;

    const logoLight =
      typeof client.logoLight === "number"
        ? (mediaById.get(client.logoLight) ?? client.logoLight)
        : client.logoLight;

    return { ...client, logo, logoLight };
  });

  return { ...block, selectedClients: enrichedClients };
}

Notes:

• Use a Set to deduplicate IDs.
• Use a Map for fast lookups.
• Fall back gracefully if an ID is missing.

---

Pattern 2: multi-field enrichment

Use when a block needs enrichment across multiple fields, such as media plus CTAs.

Examples:

• Featured Solutions: solution image plus block background image plus CTAs
• Featured Projects: featured image plus industries

Featured Solutions implementation

Block location: src/components/blocks/featured-solutions/index.tsx (lines 40-93)

This block enriches media first:

async function enrichSolutionMedia(
  solutions: any[] | undefined,
  blockBackgroundImageId: number | undefined,
  tenant: string | undefined,
): Promise<{ enrichedSolutions: any[]; enrichedBlockImage: any }> {
  if (!Array.isArray(solutions)) {
    return {
      enrichedSolutions: [],
      enrichedBlockImage: blockBackgroundImageId,
    };
  }

  const mediaIds = new Set<number>();

  solutions.forEach((solution) => {
    if (typeof solution.image === "number") {
      mediaIds.add(solution.image);
    }
  });

  if (typeof blockBackgroundImageId === "number") {
    mediaIds.add(blockBackgroundImageId);
  }

  if (mediaIds.size === 0) {
    return {
      enrichedSolutions: solutions,
      enrichedBlockImage: blockBackgroundImageId,
    };
  }

  const mediaResult = await getMediaImages(Array.from(mediaIds));
  const mediaById = new Map(mediaResult.docs.map((m) => [m.id, m]));

  const enrichedSolutions = solutions.map((solution) => {
    if (typeof solution.image === "number") {
      return {
        ...solution,
        image: mediaById.get(solution.image) ?? solution.image,
      };
    }
    return solution;
  });

  const enrichedBlockImage =
    typeof blockBackgroundImageId === "number"
      ? (mediaById.get(blockBackgroundImageId) ?? blockBackgroundImageId)
      : blockBackgroundImageId;

  return { enrichedSolutions, enrichedBlockImage };
}

Then it enriches CTAs:

const { enrichedSolutions, enrichedBlockImage } = await enrichSolutionMedia(
  block.selectedSolutions,
  (block as any).backgroundImage?.id,
  tenant,
);

const enrichedCtas = await enrichCtasWithPageMetadata(block.cta, tenant);

const enrichedBlock = {
  ...block,
  selectedSolutions: enrichedSolutions,
  backgroundImage: enrichedBlockImage,
  cta: enrichedCtas ?? block.cta,
};

Parallelization is often worth it when you have multiple independent fetches:

const [media, ctas] = await Promise.all([
  getMediaImages(mediaIds),
  getCtasByIds(ctaIds),
]);

---

Pattern 3: polymorphic relationship enrichment

Use when a relationship can point to multiple collections.

Example:

• Featured Products: items with relationTo that can reference products, pages, or solutions

Featured Products implementation

Block location: src/components/blocks/featured-products/index.tsx (lines 10-28)

Polymorphic values look like this:

selectedItems = [
  { relationTo: "product", value: { id: 42 } },
  { relationTo: "page", value: { id: 55 } },
];

Enrichment groups IDs by collection, fetches in parallel, then maps back:

async function enrichProducts(
  items: any[] | undefined,
  tenant: string | undefined,
): Promise<any[]> {
  if (!Array.isArray(items) || items.length === 0) {
    return items || [];
  }

  const productIds = new Set<number>();
  const pageIds = new Set<number>();

  items.forEach((item) => {
    if (!item.relationTo || !item.value) return;

    const id = typeof item.value === "object" ? item.value.id : item.value;

    if (item.relationTo === "product") {
      productIds.add(id);
    } else if (item.relationTo === "page") {
      pageIds.add(id);
    }
  });

  const [products, pages] = await Promise.all([
    productIds.size > 0
      ? getProductsByIds(Array.from(productIds), tenant)
      : Promise.resolve([]),
    pageIds.size > 0
      ? getPagesByIds(Array.from(pageIds), tenant)
      : Promise.resolve([]),
  ]);

  const productMap = new Map(products.map((p) => [p.id, p]));
  const pageMap = new Map(pages.map((p) => [p.id, p]));

  return items.map((item) => {
    if (!item.relationTo || !item.value) return item;

    const id = typeof item.value === "object" ? item.value.id : item.value;

    let enrichedValue;
    if (item.relationTo === "product") enrichedValue = productMap.get(id);
    if (item.relationTo === "page") enrichedValue = pageMap.get(id);

    return {
      ...item,
      value: enrichedValue ?? item.value,
    };
  });
}

---

Pattern 4: two-phase enrichment

Use when enrichment phase 1 unlocks what you need to do in phase 2.

Example:

• Featured Divisions: fetch full divisions, then enrich each division’s CTA

Featured Divisions implementation

Block location: src/components/blocks/featured-divisions/index.tsx (lines 11-61)

Phase 1: fetch divisions

async function enrichDivisions(
  divisions: any[] | undefined,
  tenant: string | undefined,
): Promise<Division[]> {
  if (!divisions || divisions.length === 0) return [];
  if (!tenant) return divisions as Division[];

  const divisionIds = divisions
    .map((div) => (typeof div === "number" ? div : div.id))
    .filter((id): id is number => typeof id === "number");

  if (divisionIds.length === 0) return divisions as Division[];

  const fullDivisions = await getDivisionsByIds(divisionIds, tenant);
  const divisionsMap = new Map(fullDivisions.map((div) => [div.id, div]));

  const orderedDivisions: Division[] = [];
  for (const id of divisionIds) {
    const division = divisionsMap.get(id);
    if (division) orderedDivisions.push(division);
  }

  return orderedDivisions;
}

Phase 2: enrich CTAs

async function enrichDivisionCtas(
  divisions: Division[],
  tenant: string | undefined,
): Promise<Division[]> {
  if (!divisions.length || !tenant) return divisions;

  const enriched = await Promise.all(
    divisions.map(async (division) => {
      if (!division.cta) return division;

      const cta = await enrichCtaWithPageMetadata(division.cta, tenant);

      return {
        ...division,
        cta: cta ?? division.cta,
      };
    }),
  );

  return enriched;
}

Usage:

const divisions = await enrichDivisions(block.selectedDivisions, tenant);
const enrichedDivisions = await enrichDivisionCtas(divisions, tenant);

return {
  ...block,
  selectedDivisions: enrichedDivisions,
};

---

Pattern 5: complex batch with multiple types

Use when a block needs multiple related datasets, such as media plus industries, and you want to fetch them efficiently.

Example:

• Featured Projects: featured image plus industries

Featured Projects implementation

Block location: src/components/blocks/featured-projects/index.tsx (lines 74-140)

async function enrichProjectData(
  selectedProjects: FeaturedProjectBlock["selectedProjects"],
  tenant: string,
): Promise<FeaturedProjectBlock["selectedProjects"]> {
  if (!Array.isArray(selectedProjects) || selectedProjects.length === 0) {
    return selectedProjects;
  }

  const mediaIds = new Set<number>();
  const industryIds = new Set<number>();

  selectedProjects.forEach((project) => {
    if (!isObject(project)) return;

    const featuredImage = project.featuredImage;
    if (typeof featuredImage === "number") {
      mediaIds.add(featuredImage);
    } else if (
      isObject(featuredImage) &&
      typeof featuredImage.id === "number"
    ) {
      mediaIds.add(featuredImage.id);
    }

    const industries = project.projectStats?.industry;
    if (Array.isArray(industries)) {
      industries.forEach((id) => {
        if (typeof id === "number") industryIds.add(id);
      });
    } else if (typeof industries === "number") {
      industryIds.add(industries);
    }
  });

  const [mediaResult, industryDocs] = await Promise.all([
    mediaIds.size > 0
      ? getMediaImages(Array.from(mediaIds))
      : Promise.resolve({ docs: [] }),
    industryIds.size > 0
      ? getIndustriesByIds(Array.from(industryIds), tenant)
      : Promise.resolve([]),
  ]);

  const mediaById = new Map(
    (mediaResult as any).docs.map((media: any) => [media.id, media]),
  );
  const industryById = new Map(
    industryDocs.map((industry: any) => [industry.id, industry]),
  );

  return selectedProjects.map((project) => {
    if (!isObject(project)) return project;

    const updatedProject = { ...project };

    const featuredImage = updatedProject.featuredImage;
    if (typeof featuredImage === "number") {
      updatedProject.featuredImage =
        (mediaById.get(featuredImage) as any) ?? featuredImage;
    } else if (
      isObject(featuredImage) &&
      typeof (featuredImage as any).id === "number"
    ) {
      updatedProject.featuredImage =
        (mediaById.get((featuredImage as any).id) as any) ?? featuredImage;
    }

    if (updatedProject.projectStats?.industry) {
      const currentIndustries = updatedProject.projectStats.industry;
      if (Array.isArray(currentIndustries)) {
        updatedProject.projectStats.industry = currentIndustries.map((id) =>
          typeof id === "number" ? ((industryById.get(id) as any) ?? id) : id,
        );
      } else if (typeof currentIndustries === "number") {
        updatedProject.projectStats.industry =
          (industryById.get(currentIndustries) as any) ?? currentIndustries;
      }
    }

    return updatedProject;
  });
}

---

Pattern 6: API route server-side enrichment

Use when building API routes for infinite scroll, search, or filters. The route returns lean docs, then enriches only what is needed for the response.

Example:

• Gallery API endpoints

Gallery Projects API implementation

API location: src/app/api/gallery/projects/route.ts (lines 1-195)

export async function GET(request: NextRequest) {
  const searchParams = request.nextUrl.searchParams;
  const tenant = searchParams.get("tenant") || "default-tenant";
  const cursor = searchParams.get("cursor");
  const limit = parseInt(searchParams.get("limit") || "12", 10);

  try {
    const startTime = performance.now();

    const projectQueryStart = performance.now();
    const projectsResult = await payload.find({
      collection: "project",
      where: {
        tenant: { equals: tenant },
        ...(cursor && { id: { less_than: cursor } }),
      },
      depth: 0,
      select: {
        id: true,
        title: true,
        slug: true,
        description: true,
        featuredImage: true,
      },
      limit: limit + 1,
    });
    const projectQueryTime = performance.now() - projectQueryStart;

    const mediaIds = new Set<number>();
    projectsResult.docs.forEach((project: any) => {
      if (typeof project.featuredImage === "number") {
        mediaIds.add(project.featuredImage);
      }
    });

    const mediaQueryStart = performance.now();
    const mediaResult =
      mediaIds.size > 0
        ? await getMediaImages(Array.from(mediaIds))
        : { docs: [] };
    const mediaQueryTime = performance.now() - mediaQueryStart;

    const mediaById = new Map(
      mediaResult.docs.map((media: any) => [media.id, media]),
    );

    const enrichStart = performance.now();
    const enrichedProjects = projectsResult.docs.map((project: any) => ({
      ...project,
      featuredImage:
        mediaById.get(project.featuredImage) ?? project.featuredImage,
    }));
    const enrichTime = performance.now() - enrichStart;

    console.log(`
=== GALLERY PROJECTS API - START ===
Params: { tenant: '${tenant}', cursor: ${cursor}, limit: ${limit} }
Projects query: ${projectQueryTime.toFixed(2)}ms (${projectsResult.docs.length} docs)
Media batch query: ${mediaQueryTime.toFixed(2)}ms (${mediaResult.docs?.length || 0} docs)
Enrichment: ${enrichTime.toFixed(2)}ms
Total time: ${(performance.now() - startTime).toFixed(2)}ms
=== GALLERY PROJECTS API - END ===
    `);

    return NextResponse.json({
      docs: enrichedProjects,
      hasMore: projectsResult.docs.length > limit,
    });
  } catch (error) {
    console.error("Gallery projects API error:", error);
    return NextResponse.json(
      { error: "Failed to fetch projects" },
      { status: 500 },
    );
  }
}

In the end, this approach is less about clever fetching and more about control. Keep the initial page query lean so your caching stays reliable, then enrich only what each block needs, in batches, at the point where you actually need it. That keeps payloads predictable, avoids accidental N+1 patterns, and makes performance issues easier to spot and fix as the project grows.

Thanks, Matija