LLM's Fatal Flaw: Amnesia Tests Architecture Skills

🧠 AI's Biggest Lie: I Remember Everything

The most terrifying thing about large language models isn't their intelligence issues, but their memory problems. They're like geniuses with "digital amnesia" - eloquent one moment, forgetting what they said the next.

During Tidepool Notes development, I discovered that context pollution and memory decay are the most hidden traps in AI programming. Today, I'll teach you how to combat AI's amnesia through architectural design.

🎯 Problem 1: Memory Fragmentation Under Token Limits

AI's Memory Principles: Cruel Physical Constraints

Large Model Memory Mechanism:

Short-term memory: Current conversation context, usually only a few thousand to tens of thousands of tokens
Long-term memory: Knowledge from training data, but cannot remember specific conversation content
Memory decay: As conversation length increases, early content is gradually "forgotten"

How cruel is reality?

Assume a 2000-line code file:
- Average 50 characters per line = 100,000 characters
- Chinese 1 character ≈ 1.5 tokens, code 1 character ≈ 0.7 tokens
- Total tokens: about 80,000-120,000 tokens
- GPT-4 context window: 128,000 tokens
- Conclusion: One 2000-line file almost fills the entire context!

Specific Manifestations of Amnesia

1. Progressive Forgetting

// Global configuration defined at file beginning
const API_CONFIG = {
  baseUrl: "https://api.example.com",
  timeout: 5000,
  retries: 3
}

// 500 lines later, AI might forget this configuration exists
async function callAPI(endpoint: string) {
  // AI might redefine similar configuration instead of using existing one
  const config = {
    url: "https://api.example.com" + endpoint, // Duplicate definition!
    timeout: 3000 // Inconsistent configuration!
  }
}

2. Context Pollution

// Wrong example at file beginning
function BadExample() {
  const user = { name: "John", age: 25 }
  return user.name
}

// 1000 lines later, AI might be influenced by this wrong example
function createUser() {
  // AI might imitate wrong structure instead of best practices
  const newUser = { name: "Alice", age: 30 }
  // Forgets type definitions, validation, error handling, etc.
  return newUser
}

3. Loss of Constraint Conditions

// Constraint conditions at project beginning
/*
Project Principles:
- All functions must have error handling
- Use TypeScript strict mode
- Single function not exceeding 50 lines
- Prohibit using any type
*/

// 800 lines later, AI-generated code might violate these constraints
function processUserData(data: any) { // Violates: prohibit any
  // No error handling - violates error handling principle
  // Function too long - violates 50-line limit
  // Data processing logic complex but no type safety
}

🏗️ Solution 1: Architectural Art of File Size Control

Why 1000 Lines? Scientific Basis

Cognitive Load Theory:

7±2 Rule: Human working memory can handle 7±2 information chunks
Cognitive chunks: Functions, classes, modules in code are all information chunks
Context switching cost: Larger files mean higher cost to find relevant information

AI Context Efficiency:

Attention mechanism: AI needs to "focus" on all relevant content
Information density: 1000-line file's information density is just right for AI processing
Retrieval efficiency: AI can find relevant information within reasonable range

Design Principles for 1000-line Files

1. Single Responsibility Files

// ✅ User-related logic - users.service.ts
import { userRepository } from '@/repositories'
import { logger } from '@/utils'
import type { User, CreateUserInput } from '@/types'

export class UserService {
  async create(userData: CreateUserInput): Promise<User> {
    try {
      const user = await userRepository.create(userData)
      logger.info(`User created: ${user.id}`)
      return user
    } catch (error) {
      logger.exception('Failed to create user', error)
      throw error
    }
  }

  async findById(id: string): Promise<User> {
    const user = await userRepository.findById(id)
    if (!user) {
      throw new Error(`User not found: ${id}`)
    }
    return user
  }

  // Total: about 150 lines
}

// ❌ Avoid: Multi-function mixed files
// user-manager.service.ts - 2000 lines
// - User management (500 lines)
// - Permission control (400 lines)
// - Session management (600 lines)
// - Data statistics (500 lines)

2. Modular Layered Design

// ✅ Layered architecture, controllable file sizes
backend/src/
├── routes/          # Each file 100-200 lines
│   ├── users.routes.ts
│   ├── sessions.routes.ts
│   └── messages.routes.ts
├── services/        # Each file 200-400 lines
│   ├── users.service.ts
│   ├── sessions.service.ts
│   └── llm.service.ts
├── repositories/    # Each file 100-300 lines
│   ├── users.repository.ts
│   └── sessions.repository.ts
└── types/          # Each file 50-150 lines
    ├── users.types.ts
    └── sessions.types.ts

3. Interface Definition Separation

// ✅ types/user.types.ts - Dedicated type definition file
export interface User {
  id: string
  name: string
  email: string
  createdAt: Date
  updatedAt: Date
}

export interface CreateUserInput {
  name: string
  email: string
}

export interface UserQuery {
  page?: number
  limit?: number
  search?: string
}

// This way, files in services folder won't be occupied too much by type definitions

🧩 Solution 2: Protection Strategies Against Context Pollution

Identification and Isolation of Pollution Sources

1. Isolation of Wrong Examples

// ❌ Wrong approach: Mix wrong examples in normal code
export class UserService {
  // Don't write wrong example code here!
  // badExample() { ... } // Pollutes AI's learning

  async createUser(userData: CreateUserInput): Promise<User> {
    // Correct implementation
  }
}

// ✅ Correct approach: Independent example files
docs/examples/common-mistakes.ts
export const badExamples = {
  // Wrong examples concentrated here
  badUserCreation: `function createUser(data: any) { ... }`
}

2. Optimization of Comments and Documentation

// ❌ Overly detailed comments cause pollution
/*
This function is used to create users, its principle is...
Historical background: Initial design was to solve...
In version 1.2 we fixed a certain bug...
Future expansion plans include...
Possible alternatives...
Performance considerations...
Security considerations...
*/
async function createUser(data: CreateUserInput): Promise<User> {
  // Actual code
}

// ✅ Simple and effective comments
// Create new user, input validation completed in service layer
async function createUser(data: CreateUserInput): Promise<User> {
  // Implementation
}

3. Cleanup of Version History

// ❌ Keeping deprecated code causes pollution
// async function createUserV1(data: any): Promise<any> { ... } // Deprecated
// async function createUserV2(data: UserInput): Promise<UserData> { ... } // Deprecated

// ✅ Delete deprecated code immediately
async function createUser(data: CreateUserInput): Promise<User> {
  // Current version
}

Context Optimization Techniques

1. Code Density Optimization

// ❌ Low-density code, occupies too much context
export class UserService {
  // Lots of empty lines
  // Overly detailed comments


  async createUser(userData: CreateUserInput): Promise<User> {
    // Verbose variable names
    const theUserThatWeWantToCreateInDatabase = userData

    // Overly split simple logic
    const isValid = this.validateUserData(theUserThatWeWantToCreateInDatabase)
    if (isValid) {
      const createdUser = await userRepository.create(theUserThatWeWantToCreateInDatabase)
      return createdUser
    }
    return null
  }
}

// ✅ High-density code, saves context
export class UserService {
  async createUser(userData: CreateUserInput): Promise<User> {
    this.validate(userData)
    return await userRepository.create(userData)
  }

  private validate(userData: CreateUserInput): void {
    if (!userData.email) throw new Error('Email required')
  }
}

2. Import Optimization

// ❌ Over-importing causes context pollution
import {
  UserService,
  MessageService,
  AuthService,
  UserServiceV1, // Deprecated but not deleted
  UserTypes,
  MessageTypes,
  HelperFunctions1,
  HelperFunctions2,
  // 20 other imports...
} from '@/services'

// ✅ Precise imports
import { UserService } from '@/services/users.service'
import { type CreateUserInput } from '@/types/user.types'

🔧 Solution 3: Engineering Countermeasures Against Amnesia

Project-Level Memory Enhancement

1. Structured Claude.md Documentation

# Tidepool Notes - AI Programming Project Specification

## Core Architecture Principles (Must Follow)
1. **File Size Limit**: Single file not exceeding 1000 lines
2. **Type Safety**: Strict TypeScript usage, prohibit any
3. **Error Handling**: Use logger.exception, Let it Crash principle
4. **Dependency Injection**: Use Repository pattern

## Technology Stack Constraints
- Frontend: Taro + React + TypeScript
- Backend: Fastify + TypeScript + Prisma
- Database: PostgreSQL
- AI Services: OpenAI API

## Code Templates (For Reference)
```typescript
// Service template
export class ExampleService {
  async create(data: CreateData): Promise<Example> {
    try {
      const result = await exampleRepository.create(data)
      logger.info(`Example created: ${result.id}`)
      return result
    } catch (error) {
      logger.exception('Failed to create example', error)
      throw error
    }
  }
}

Prohibited Patterns (Strictly Avoid)

Don't use try-catch to mask errors
Don't keep deprecated code in normal code
Don't use any types
Don't create files exceeding 1000 lines


**2. Automated Checking Tools**
```typescript
// scripts/file-size-check.js
const fs = require('fs')
const path = require('path')

function checkFileSize(filePath) {
  const content = fs.readFileSync(filePath, 'utf8')
  const lines = content.split('\n').length

  if (lines > 1000) {
    console.error(`❌ ${filePath}: ${lines} lines (exceeds 1000)`)
    return false
  }

  console.log(`✅ ${filePath}: ${lines} lines`)
  return true
}

// Check all TypeScript files
const srcDir = './src'
const files = getAllTSFiles(srcDir)
let allValid = true

files.forEach(file => {
  if (!checkFileSize(file)) {
    allValid = false
  }
})

process.exit(allValid ? 0 : 1)

3. Layered Development Strategy

// First layer: Interface definitions (always visible)
types/
├── user.types.ts      # 100 lines
├── session.types.ts   # 80 lines
└── message.types.ts   # 60 lines

// Second layer: Data access (stable, few changes)
repositories/
├── user.repository.ts     # 200 lines
├── session.repository.ts  # 180 lines
└── message.repository.ts  # 150 lines

// Third layer: Business logic (most changes, but keep modular)
services/
├── users.service.ts      # 400 lines
├── sessions.service.ts   # 350 lines
└── llm.service.ts        # 500 lines

// Each file is within AI's optimal memory range

Development Process Amnesia Protection

1. Frequent Context Reset

// Before each new feature begins, reload key context
/*
# New Feature Development: User Feedback System

Related files:
- types/feedback.types.ts (Define data structure)
- repositories/feedback.repository.ts (Data access)
- services/feedback.service.ts (Business logic)
- routes/feedback.routes.ts (API interface)

Constraint conditions:
- Single file not exceeding 1000 lines
- Strict TypeScript type checking
- Complete error handling
- Follow Repository pattern

Current task: Implement CRUD operations for feedback data
*/

2. Modular Development Rhythm

// Development rhythm: Each module developed and tested independently
// First phase: User module (500 lines of code)
// Second phase: Session module (400 lines of code)
// Third phase: Message module (600 lines of code)
// Fourth phase: AI analysis module (800 lines of code)

// Instead of: One big file containing all features (3000 lines of code)

3. Continuous Code Refactoring

// Regularly check file sizes, split timely
if (file.lines > 800) {
  // Start considering splitting
  const newFile = extractRelatedFunctions(file)
  refactorImports(file, newFile)
  updateTests(file, newFile)
}

🎯 Practical Cases: Amnesia Protection in Tidepool Notes

Case 1: Message System Refactoring

Original Problem:

// messages.service.ts - 1800 lines, containing too many features
export class MessageService {
  // Message CRUD (400 lines)
  // Message search (300 lines)
  // Message statistics (200 lines)
  // Message analysis (400 lines)
  // Message export (200 lines)
  // AI message processing (300 lines)
}

Refactoring Solution:

// messages.service.ts - 400 lines
export class MessageService {
  // Core CRUD operations
}

// message-search.service.ts - 300 lines
export class MessageSearchService {
  // Search-related logic
}

// message-analytics.service.ts - 400 lines
export class MessageAnalyticsService {
  // Statistical analysis logic
}

// message-export.service.ts - 200 lines
export class MessageExportService {
  // Export functionality
}

Case 2: Modular AI Integration

Amnesia Challenges Faced by AI Services:

LLM prompt management
Conversation history maintenance
Multi-turn conversation state
Error handling and retry

Solution:

// llm/
├── prompt-manager.ts      # 200 lines - Prompt management
├── conversation-manager.ts # 300 lines - Conversation state
├── response-parser.ts     # 150 lines - Response parsing
└── error-handler.ts       # 100 lines - Error handling

// llm.service.ts - 400 lines, combines use of above modules
export class LLMService {
  constructor(
    private promptManager: PromptManager,
    private conversationManager: ConversationManager,
    private responseParser: ResponseParser,
    private errorHandler: ErrorHandler
  ) {}
}

Case 3: Context Pollution Cleanup

Problem Identification:

// Pollution found in problem files
// 1. Deprecated functions not deleted
// 2. Wrong example code mixed in
// 3. Overly detailed comments
// 4. Duplicate code snippets

Cleanup Strategy:

// 1. Delete deprecated code immediately
git log --name-status # Find change records
git blame file.ts     # Find pollution source

// 2. Move wrong examples to documentation directory
mv examples/bad-examples.ts docs/common-mistakes.md

// 3. Simplify comments
// Delete historical background, future plans, etc.

// 4. Extract duplicate code
const common = extractCommonCode(file.ts)
createUtilityFile(common)

🌟 Summary: Architecture in the Age of Amnesia

Essence of LLM Amnesia

Core Problems:

Physical Limitations: Hard constraints of token context windows
Attention Decay: Distant information gets less attention
Pattern Interference: Wrong or deprecated code pollutes AI's learning

Architectural Thinking for Solutions

1. Splitting Thinking

Split files by responsibility, not by technical layers
Keep single responsibility for each file, control within 1000 lines
Avoid large files, and especially super-large files

2. Cleanup Thinking

Delete deprecated code promptly, don't keep it for "possible future use"
Move wrong examples to avoid polluting AI's learning
Simplify comments, delete irrelevant information

3. Structured Thinking

Use clear file structure to guide AI understanding
Use type definitions to constrain AI behavior
Use documentation templates to standardize AI output

Practical Checklist

File Health Check:

File lines not exceeding 1000
No deprecated functions and classes
No wrong example code
Precise imports, no redundancy
Simple and effective comments, no historical information

Project Architecture Check:

Each module has single responsibility
Clear and independent type definitions
Consistent Repository pattern application
Unified error handling standards
Complete and updated Claude.md documentation

Development Process Check:

Each new feature has clear context scope
Regular code refactoring and cleanup
Use automated tools to check file sizes
Maintain modular development rhythm

Remember: AI programming is not about making AI adapt to your code, but designing code architecture that AI can efficiently understand. In an era where amnesia becomes the norm, good architecture is the best weapon against forgetting.

Next Article Preview: Architecture as Code: How to Tame AI with Systematic Thinking

Search "四哥还会聊AI" on WeChat to see how these architecture principles are applied in actual projects

Fourth Brother's AI Journey

article-5-llms-fatal-flaw-amnesia-testing-architecture-skills