Topic 03: Process Memory Accounting & Transparent HugePages (THP)

Target Duration: 2–4 minutes (~300–450 spoken words)
Focus: Pointwise verbal delivery covering $O(1)$ fast RSS counters vs $O(N)$ Maple tree VMA iteration, 2 MiB Transparent HugePage (THP) detection at the PMD level, and cursor stride optimizations.

🎙️ Pointwise Spoken Speech (Word-for-Word Delivery)

• Opening & Scope:
  "In 1/lkm4.c and 1/lkm5.c, I engineered process memory accounting subsystems to analyze virtual address allocations, resident physical memory (RSS), and 2 MiB Transparent HugePages (THP) under Linux 6.1."

• Step 1: O(1) Fast RSS vs O(N) Deep Page Walks:
  "First, I investigated how the kernel accounts for resident physical memory. The kernel provides an $O(1)$ fast lookup helper—get_mm_rss(mm) (1/lkm4.c:90)—which aggregates atomic counters (MM_FILEPAGES, MM_ANONPAGES, MM_SHMEMPAGES). This is what utilities like top and /proc/[pid]/statm read. However, to verify ground truth, I implemented an $O(N)$ deep traversal (get_mapped_size in 1/lkm4.c:19) that iterates through every Virtual Memory Area (VMA) and checks each individual page entry's presence."

• Step 2: Modern VMA Iteration with the Linux 6.1 Maple Tree API:
  "Next, in Linux 6.1, the historical red-black tree and linked list for VMAs were replaced by the Maple Tree data structure. To walk the VMAs safely and idiomatically, I used the Maple Tree iterator API: VMA_ITERATOR(vma_iter, mm, 0) & for_each_vma() (1/lkm4.c:107-112) (also implemented in 1/lkm5.c:108-110). For each VMA, I accumulated the virtual range (vma->vm_end - vma->vm_start) and walked the underlying page tables to verify physical residency."

• Step 3: Detecting 2 MiB Transparent HugePages (THP):
  "Then, in lkm5, I focused on Transparent HugePages. Modern x86_64 and arm64 architectures support 2 MiB huge pages, where the PMD points directly to a large physical frame rather than a sub-level PTE page table. To identify THPs, I walked the page tables down to the PMD stage and checked pmd_trans_huge(*pmd) (1/lkm5.c:72). If true, the memory region is backed by a 2 MiB compound huge page."

• Step 4: The 2 MiB Cursor Stride Optimization:
  "A critical algorithmic optimization I implemented was the stride skip. Normally, a page walk loop increments the virtual address by PAGE_SIZE (4 KiB). When encountering a 2 MiB THP, continuing with a 4 KiB step would erroneously attempt 512 redundant lookups on non-existent PTE tables, causing corrupt statistics and redundant CPU cycles. Instead, once pmd_trans_huge returned true, I incremented the virtual cursor directly by HPAGE_PMD_SIZE (2 MiB) (1/lkm5.c:76-78), guaranteeing exact $O(1)$ processing for the huge block."

• Step 5: Testing with madvise and MADV_HUGEPAGE:
  "Finally, to validate the module under controlled conditions, I wrote a user-space test harness (1/test2.c:8). It allocated aligned memory buffers and issued madvise(addr, size, MADV_HUGEPAGE) to instruct the kernel's khugepaged daemon to back the allocation with 2 MiB pages. Running lkm5 against this process accurately reported the exact number of 2 MiB THP blocks allocated."

📋 Step-by-Step Summary (What, How & Why)

❓ Anticipated Interview Questions & Crisp Answers

Q1: Why did you build an $O(N)$ VMA and page table walk when the kernel already provides fast atomic RSS counters (get_mm_rss())?

Answer: get_mm_rss() (which powers ps, top, and /proc/[pid]/statm) is an $O(1)$ read of atomic counters (MM_FILEPAGES, MM_ANONPAGES, MM_SHMEMPAGES) updated lazily during page fault handling and unmapping. In multi-threaded systems or copy-on-write sharing, atomic counters can diverge or over-count shared pages. Furthermore, atomic counters cannot distinguish which specific VMA or memory segment (heap, stack, anonymous mmap) owns the physical frames. Our $O(N)$ VMA traversal inspects present PTE flags directly, providing the ground-truth physical footprint broken down by address region.

Q2: What performance bug or accounting error occurred when traversing Transparent HugePages (THPs)?

Answer: Standard page table traversals advance virtual addresses in 4 KiB increments (PAGE_SIZE). But a THP is mapped at the PMD level as a single contiguous 2 MiB leaf entry (pmd_trans_huge(*pmd) is true). If a walker treats a huge PMD as a pointer to a PTE page or increments by 4 KiB, two things break: 1) calling pte_offset_kernel() corrupts kernel pointers because the PMD holds raw data, not a pointer table; and 2) stepping 4 KiB causes 512 redundant lookups of the exact same huge page, artificially inflating physical residency by 512x. Upon detecting pmd_trans_huge(), our loop immediately accounts for 2 MiB and strides the address forward by HPAGE_PMD_SIZE (2 MiB), avoiding 511 redundant iterations.

Q3: How did you create a reproducible test workload in user space to verify your THP detection logic?

Answer: The Linux kernel's background khugepaged daemon collapses pages opportunistically and non-deterministically. To reliably trigger THP allocation for our test harness, we allocated an aligned 10 MiB buffer with posix_memalign(), called madvise(buf, size, MADV_HUGEPAGE) to explicitly register the VMA for huge pages, and touched every 4 KiB page. Our test harness then compared our kernel module's reported huge page count against /proc/self/smaps (AnonHugePages: 10240 kB), verifying 100% detection accuracy.