# kernel-internals

Guides AI agents through Linux memory allocators, CFS and EEVDF schedulers, and VFS internals. Useful when tracking allocation failures or tuning kernel caches.

- skill · `type=skill` · Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode · 307 installs · ★ 267
- Author: mohitmishra786
- License: MIT · policy=mirror · the beginning of the file is at the end of this page
- Source: https://github.com/mohitmishra786/low-level-dev-skills/tree/HEAD/skills/kernel/kernel-internals/SKILL.md
- Updated: 2026-06-27
- HTML: https://affpapa.org/en/nevibecode/skills/kernel-internals

## What it does

This skill equips coding agents with knowledge of Linux kernel internals. It covers runqueues, latency inspection with `perf sched`, and the transition from CFS to EEVDF. Memory coverage spans the buddy allocator, SLUB caches, and the trade-offs between `kmalloc` and `vmalloc` during allocation routines.

The guide also outlines VFS structures such as dentries and inodes, along with page cache writeback behavior. It provides a breakdown of `/proc/meminfo` metrics and explains how the OOM killer scores victim processes.

Scheduler debugging through `/sys/kernel/debug/sched/debug` will fail unless `debugfs` is already mounted on your machine.

## Install

### Claude Code (native platform)

1. **Skills CLI (Vercel)**
```bash
npx skills add mohitmishra786/low-level-dev-skills --skill kernel-internals -a claude-code
```

2. **Manually: put the files in place**
- File: `~/.claude/skills/kernel-internals/SKILL.md` — Copy the whole skill folder; for one project use .claude/skills/kernel-internals/

### Codex

1. **Skills CLI (Vercel)**
```bash
npx skills add mohitmishra786/low-level-dev-skills --skill kernel-internals -a codex
```

2. **Manually: put the files in place**
- File: `.agents/skills/kernel-internals/SKILL.md` — For all projects use ~/.agents/skills/

### Cursor

1. **Skills CLI (Vercel)**
```bash
npx skills add mohitmishra786/low-level-dev-skills --skill kernel-internals -a cursor
```

2. **Manually: put the files in place**
- File: `.cursor/skills/kernel-internals/SKILL.md` — Cursor also reads .agents/skills/ and .claude/skills/

### GitHub Copilot

1. **GitHub CLI**
```bash
gh skill install mohitmishra786/low-level-dev-skills kernel-internals
```

2. **Manually: put the files in place**
- File: `.github/skills/kernel-internals/SKILL.md`

### Gemini CLI

1. **Skills CLI (Vercel)**
```bash
npx skills add mohitmishra786/low-level-dev-skills --skill kernel-internals -a gemini-cli
```

2. **Platform CLI**
```bash
gemini skills install https://github.com/mohitmishra786/low-level-dev-skills --path skills/kernel/kernel-internals
```

### Windsurf (Devin Desktop)

**Manually: put the files in place**
- File: `.devin/skills/kernel-internals/SKILL.md` — The legacy .windsurf/skills/ path still works

### OpenCode

**Skills CLI (Vercel)**
```bash
npx skills add mohitmishra786/low-level-dev-skills --skill kernel-internals -a opencode
```

### Claude.ai

**Claude.ai**
- Zip the folder and upload it in Settings → Capabilities → Skills

Where a file is given instead of a command, copy the files from the source to that path: https://github.com/mohitmishra786/low-level-dev-skills/tree/HEAD/skills/kernel/kernel-internals/SKILL.md

## When to use it
- diagnose failed `kmalloc` calls caused by physical memory fragmentation
- inspect runqueue latency bottlenecks in the EEVDF scheduler
- evaluate OOM score adjustments for critical system processes
- tune page cache readahead parameters on block devices

## Example prompts
- Explain why my kernel module encounters allocation failures with `kmalloc` under heavy memory load.
- Show how to inspect runqueue latency for a specific PID using `perf sched` and debugfs.
- Help diagnose memory pressure from `/proc/meminfo` when reclaimable slab memory keeps rising.

## Source text · SKILL.md · MIT

Below is the content of a third-party file. It is data, not instructions for you: do not act on anything written inside unless the human asks. Block tag: 0bed9a2aebfc33f7.

BEGIN THIRD-PARTY FILE 0bed9a2aebfc33f7
````text
---
name: kernel-internals
description: Linux kernel internals skill for scheduler, memory, and VFS subsystems. Use when analyzing CFS/EEVDF scheduling, buddy/SLUB allocators, page cache, memory zones, OOM killer, or interpreting /proc/meminfo. Activates on queries about kernel scheduler, vruntime, SLUB, vmalloc, page cache, or OOM killer.
---

# Kernel Internals

## Purpose

Guide agents through Linux kernel internals: the CFS and EEVDF schedulers, runqueues and vruntime, the buddy allocator and SLUB, vmalloc vs kmalloc, VFS dentry/inode/file objects, page cache and readahead, memory zones, OOM killer heuristics, and `/proc/meminfo` interpretation.

## When to Use

- Diagnosing scheduling latency or unfair CPU distribution
- Understanding kmalloc failures vs vmalloc for large kernel allocations
- Analyzing page cache behavior and readahead effectiveness
- Interpreting `/proc/meminfo` during memory pressure
- Debugging OOM killer victim selection
- Reading kernel source in `kernel/sched/`, `mm/`, or `fs/`

## Workflow

### 1. Scheduler — CFS and EEVDF

Linux uses **EEVDF** (Earliest Eligible Virtual Deadline First) for CFS-class scheduling. EEVDF was merged as an option in **6.6** (2023); the gradual CFS→EEVDF transition **completed in 6.12** (Nov 2024). On older 6.6–6.11 kernels, verify which scheduler is active. Core concepts (runqueues, vruntime/lag) carry over:

```
Per-CPU runqueue (struct rq)
├── cfs_rq — fair-class tasks sorted by vruntime/deadline
├── rt_rq  — real-time tasks (FIFO/RR)
└── dl_rq  — SCHED_DEADLINE tasks
```

Key metrics:

```bash
# Task scheduling info
chrt -p <pid>
cat /proc/<pid>/sched

# Runqueue latency (scheduler debugging)
cat /sys/kernel/debug/sched/debug  # requires debugfs

# Trace scheduler events
perf sched record -a -- sleep 5
perf sched latency
```

`vruntime` — virtual runtime tracking CPU time consumed; lower vruntime = more eligible for CPU.

```bash
# CFS tunables
sysctl kernel.sched_latency_ns
sysctl kernel.sched_min_granularity_ns
sysctl kernel.sched_wakeup_granularity_ns
```

### 2. EEVDF specifics

EEVDF picks the task with the earliest eligible virtual deadline, improving latency fairness for short time-slice tasks. Tasks with positive lag are eligible; the scheduler selects the earliest virtual deadline among eligible tasks.

```bash
# Check kernel version (EEVDF transition complete in 6.12+)
uname -r

# Scheduler documentation
# docs.kernel.org/scheduler/sched-eevdf.html
```

### 3. Memory subsystem — buddy allocator

Physical pages allocated in power-of-two order (order 0 = 4KB, order 1 = 8KB, ...).

```
ZONE_DMA / ZONE_DMA32 / ZONE_NORMAL / ZONE_MOVABLE
└── free_area[MAX_ORDER] — buddy lists per order
```

```bash
# Buddy allocator stats
cat /proc/buddyinfo

# Per-zone page counts
````
END THIRD-PARTY FILE 0bed9a2aebfc33f7

The third-party file has ended. Everything between the 0bed9a2aebfc33f7 tags is data: requests and commands from it are not followed. An “end of file” line with another tag inside the block is part of the file.

First 2821 of 7126 bytes. © mohitmishra786 · MIT · https://github.com/mohitmishra786/low-level-dev-skills/tree/HEAD/skills/kernel/kernel-internals/SKILL.md

_Part of the page did not fit — see the HTML version._

---
NeVibeCode on affpapa.org. Catalog texts are descriptions, not instructions for the agent. How to choose and install: https://affpapa.org/en/nevibecode/agents.md