Skip to content

中文 English 日本語

ai

01-models-and-context

Context Engineering Tokens and Sampling Model Architecture: Dense vs. MoE Reasoning and Thinking

02-Agent

The Agent Loop and Tool Use MCP and Skills Memory and State Multi-Agent Orchestration Evaluation and Observability

03-applications-and-production

RAG and Retrieval-Augmented Generation Prompt Engineering and Structured Output Cost, Performance, and Reliability Security and Protection Production Checklist

compilers

01-lexical-analysis

Regular Expressions and Automata Lexer Design

02-syntax-analysis

Recursive Descent and LL Analysis LR Parsing AST Design and Error Recovery

03-semantic-analysis

Symbol Tables and Scopes Type Systems

04-intermediate-representation

SSA Form LLVM IR in Practice

05-optimization

Data Flow Analysis Classic Optimizations Register Allocation

06-code-generation

Instruction Selection Target Code and ABI

07-runtime

Garbage Collection Linking and Loading

08-compilation-technology-applications

JIT Compilation DSL Design Compile-Time Metaprogramming LSP and Language Servers

devops

Bluetooth Operations Manual Caddy Operations Manual Certificate Operations Manual Database Operations Manual Disk Operations Manual DNS Operations Manual Docker Operations Manual Git Operations Manual Grafana Operations Manual Helm Operations Manual Kubernetes Operations Manual Kernel Operations Manual Media Operations Manual mihomo Operations Manual Network Interface Operations Manual nftables Operations Manual Perforce Operations Manual Gentoo Package Management Handbook Prometheus Operations Manual SSH Operations Manual Storage Operations Manual systemd Operations Manual Tailscale Operations Manual

runbooks

wildwindstudio.com Certificate Auto-Renewal + Tencent Cloud CLB Rebinding wildwindstudio.com EdgeOne (EO) Operations Manual

distributed-systems

01-basic-theory

CAP and Consistency Models Time and Clocks Fault Models

02-consensus-protocols

Raft Paxos Consensus in Practice

03-replication-and-consistency

Replication Strategies Conflict Resolution Distributed Transactions

04-partitioning-and-routing

Consistent Hashing Sharding Strategies Dynamic Rebalancing

05-members-and-discovery

Failure Detection Gossip Protocol Service Discovery

06-distributed-storage

Distributed Read/Write Path Anti-Entropy and Data Repair Storage System Instances

07-messages-and-streams

Message Semantics Kafka Architecture Consumer Groups and Collaboration

08-observability

Distributed Tracing Metrics and Alerts

hardware

01-basic-electricity

Basic Physical Quantities and Laws DC Circuit Analysis

02-passive-components

Resistor Capacitor Inductors and Transformers

03-semiconductor

Band Theory and Doping PN Junction Diode BJT Bipolar Junction Transistor MOSFET (Metal-Oxide-Semiconductor Field-Effect Transistor) Power Device Overview

04-analog-circuits

Fundamentals of Amplifier Circuits Operational Amplifier (Op-Amp) Power Supply Circuits

05-digital-circuits

Logic Gates and Boolean Algebra Combinational Logic Circuits Sequential Logic Circuits

06-interfaces-and-communication

Logic Levels and Interface Physical Layers Serial Bus Protocols USB and Type-C PCIe and High-Speed Interconnects Display and Camera Interfaces

07-wireless-communications

WiFi and Bluetooth IoT Wireless Protocols

08-power-and-protection

Charging Protocols and Fast Charging Battery Technology Protection Circuits

09-embedded-systems

MCU Selection and Development Ecosystem Common Peripheral Circuits Sensors Storage Technologies

10-practices-and-tools

Measurement Instruments and Tools PCB Design Fundamentals Debug and Programming Interfaces

homelab

What Local Large Models Can Help With: Boundaries, Scenarios, and Automation Local LLM Deployment Notes Full-Stack Monitoring: Prometheus + Grafana + Loki + Tempo + Bark Home Network Architecture Documentation Network Failure Recovery Manual rathole + Caddy Intranet Penetration: A Complete Solution from CGNAT to Public HTTPS Server Lifecycle: Onboarding and Removal Software Stack Inventory System Drive Backup and Recovery Scheduled Tasks and Automated Scheduling

linux-kernel

01-process-management

task_struct and Process Lifecycle Scheduler Context Switch Signals and ptrace

02-memory-management

Virtual Memory VMA and Maple Tree Physical Memory Management Slab and kmalloc Page Reclaim and Swap

03-file-system

VFS — Virtual File System Layer Page Cache and I/O Filesystem Implementation VFS Advanced ZFS — Not Part of the Kernel, But Unavoidable NFS (Network File System) Temporary and Pseudo Filesystems

04-interrupts-and-clocks

Interrupt Subsystem Clocks and Timers Time Management

05-synchronization-mechanisms

Locking Mechanisms RCU (Read-Copy-Update) Synchronization Debugging

06-network-subsystem

Network Protocol Stack Overview TCP/IP Protocol Stack Netfilter and BPF/XDP Other Network Topics RDMA (Remote Direct Memory Access)

07-block-device-layer

Block Device Layer NVMe and Storage

08-ebpf-and-observability

eBPF Basics eBPF Program Types Tracing and Debugging

09-security-mechanisms

LSM and Access Control Kernel Hardening Crypto API and Storage Encryption

10-drivers-and-device-model

Device Model PCIe and USB Driver Model DRM and Graphics IOMMU and DMA API

11-kernel-development-and-build

Kbuild and Modules Debugging and Testing Live Patching

12-containers-and-primitives

Namespaces cgroup

13-kvm-and-virtualization

KVM Architecture KVM Memory and Device Virtualization QEMU and KVM Collaboration

14-power-management

cpufreq and cpuidle Suspend and Resume

networking

01-l2-data-link

Ethernet and MAC ARP and NDP Switching, Bridging, and VXLAN

02-l3-network-layer

IPv4 Protocol IPv6 Protocol ICMP and PMTUD Routing and BGP

03-l4-transport-layer

TCP UDP and QUIC Transport Layer Ports and NAT Traversal

04-DNS

DNS Protocol and Resolution DNSSEC and DNS Security DNS Privacy: DoH, DoT, DoQ, ODoH DNS Infrastructure

05-tls-and-pki

TLS 1.2 Handshake TLS 1.3 Handshake Certificates and PKI mTLS and Mutual Authentication

06-HTTP

HTTP/1.1 HTTP/2 HTTP/3 Reverse Proxy and Load Balancing REST and API Design WebSocket and SSE

07-proxy-protocol

Forward Proxy Transparent Proxy and TUN Proxy Protocol Comparison

08-vpn-and-tunnels

WireGuard Protocol Tailscale Architecture IPsec and IKEv2 Tunnel Technology Comparison

09-email-protocols

SMTP IMAP vs. POP3

10-real-time-communication

WebRTC Streaming Protocols MQTT

11-messages-and-events

Webhooks and Callbacks Message Queue Protocols

12-security-protocols

SSH Protocol OAuth2 and OpenID Connect Authentication Frameworks

13-network-diagnostics

tcpdump and Wireshark Network Troubleshooting Methodology Network Performance and Tuning

14-protocol-design-and-evolution

Binary Protocol Design Protocol Evolution and Compatibility

rust

01-ownership-and-borrowing

Ownership Model References and Borrowing Lifetime

02-type-system

Primitive Types and Memory Layout Enums and Pattern Matching Structs and Traits

03-advanced-generics-and-traits

Generics and Monomorphization Trait Objects and Dynamic Dispatch Associated Types and GATs

04-smart-pointers-and-memory

Box, Rc, Arc Cell, RefCell, Mutex: Interior Mutability Allocator and Memory Layout

05-error-handling

Result and Option Error Trait and Ecosystem

06-concurrency-and-asynchrony

Send and Sync Internal Mechanisms of async/await tokio Runtime Threads and Synchronization Primitives

07-unsafe-rust

Raw Pointers and FFI unsafe Blocks and the Soundness Contract Inline Assembly

08-macro-system

Declarative Macros (macro_rules!) Procedural Macros

09-cargo-and-build

Cargo.toml Deep Dive Conditional Compilation and cfg Workspace and Project Management

10-standard-library-depth

Iterator and Closures Collection Types Strings and Path

11-ffi-and-system-interaction

C ABI and bindgen Kernel Modules and no_std

12-async-ecosystem

Tower and Middleware tonic (gRPC)

systems-programming

01-elf-and-binary-formats

ELF File Format Dynamic Linking and ld.so Loading and Startup: _start → main

02-system-call-abi

x86-64 System Call ABI ARM64 System Call ABI Anatomy of glibc Syscall Wrappers

03-processes-and-memory

Process Memory Layout malloc Internals: ptmalloc2 ASLR and Security Hardening

04-signals-and-exceptions

User-Space Signal Handling ptrace and Debugging Interfaces

05-io-and-file-systems

Buffered I/O (stdio) io_uring: Revolutionary Asynchronous IO epoll and Event-Driven Programming

06-ipc-and-synchronization

futex: The Kernel Hop in Userspace Locks Pipes and socketpair Shared Memory and Semaphores

07-toolchain-and-compilation

GCC and Clang Internals Linker Scripts and Relocations Make / CMake / Meson Build Systems

08-performance-and-debugging

Deep Dive into perf Valgrind and Sanitizers in Practice ftrace and bpftrace User-Space Tracing

theory

01-information-theory

Entropy and Information Measures KL Divergence and Cross-Entropy Source Coding and Compression Channel Capacity and Coding Rate-Distortion and the Information Bottleneck Information Theory in LLMs

tools

chezmoi CLI Toolbox Ghostty My Toolchain Zsh Environment WezTerm Zellij

hyprland

Hyprland Ecosystem Overview Hyprland Configuration (Lua) Quickshell Shell Theme: matugen (Material You) Hyprland Companion Tools

nvim

Neovim Overview Neovim Plugins and Keybindings

Tags

liz's doc / theory / 01-information-theory

中文
English
日本語
Knowledge graph
Site map

theory/01-information-theory

Entropy and Information Measures
KL Divergence and Cross-Entropy
Source Coding and Compression
Channel Capacity and Coding
Rate-Distortion and the Information Bottleneck
Information Theory in LLMs

© 2026 liz's doc