# Context Engineering Part 4: Teaching AI to Take Smart Notes

This is the promise of summarization-based context management.

## The Key Idea: Compress Without Losing Meaning

![](https://cdn.hashnode.com/uploads/covers/618f409be61d1c383da7b550/5bc4bc59-9bba-460d-811a-2ddff4318bbe.png align="center")

Token savings: 90% reduction Context preserved: Key facts retained

## The Summarization Flow

![](https://cdn.hashnode.com/uploads/covers/618f409be61d1c383da7b550/5c9ff63e-24a5-4c1f-a978-25a306af4dfb.png align="center")

### Implementation

```python
class SummarizingContextManager:
    def __init__(self, max_tokens=4000, threshold=0.8):
        self.max_tokens = max_tokens
        self.threshold = threshold  # Summarize at 80% capacity
        self.messages = []
        self.summaries = []

    def add_message(self, role, content):
        self.messages.append({"role": role, "content": content})

        if self._usage_ratio() > self.threshold:
            self._summarize_older_messages()

    def _summarize_older_messages(self):
        # Keep the 3 most recent messages
        recent = self.messages[-3:]
        older  = self.messages[:-3]

        if not older:
            return

        # Create summary using the LLM
        summary = self._call_llm_for_summary(older)
        self.summaries.append({
            "role": "system",
            "content": f"Previous conversation summary: {summary}"
        })

        # Replace old messages with summary
        self.messages = recent

    def get_full_context(self):
        return self.summaries + self.messages

    def _call_llm_for_summary(self, messages):
        conversation = "\n".join(
            f"{m['role']}: {m['content']}" for m in messages
        )
        prompt = f"""Summarize this conversation concisely. Preserve:
        - User's goals and requirements
        - Technical decisions made
        - Problems and solutions discussed
        - Constraints and preferences

        Conversation:
        {conversation}"""

        return llm_api_call(prompt)
```

## Summarization Strategies

### 1\. Incremental Summarization

Summarize in small chunks rather than all at once:

![](https://cdn.hashnode.com/uploads/covers/618f409be61d1c383da7b550/ecdc8405-3602-4f6d-89ee-e67819d7b18b.png align="center")

### 2\. Structured Summarization

Use a structured template instead of free-form summaries:

```python
def create_structured_summary(messages):
    prompt = """Create a structured summary with these sections:

    GOALS:       What the user wants to accomplish
    TECH_STACK:  Technologies and tools mentioned
    DECISIONS:   Important decisions made
    PROBLEMS:    Issues discussed and their status
    CONSTRAINTS: Limitations or requirements

    Conversation:
    {conversation}"""

    return llm_api_call(prompt.format(conversation=format(messages)))
```

Output:

```plaintext
SUMMARY:
  GOALS:       Build inventory management system with real-time tracking
  TECH_STACK:  React, TypeScript, Django, PostgreSQL, Redis, AWS ECS
  DECISIONS:   Using WebSocket for real-time, Redis for caching
  PROBLEMS:    WebSocket connections dropping — investigating nginx config
  CONSTRAINTS: Must handle 10K concurrent users, deploy by Q3
```

### 3\. Hierarchical Summarization

Multiple levels of detail for different needs:

![](https://cdn.hashnode.com/uploads/covers/618f409be61d1c383da7b550/dd9b682a-09d1-4e89-b865-0b3aa0db65e4.png align="center")

## The Trade-offs of Summarization

![](https://cdn.hashnode.com/uploads/covers/618f409be61d1c383da7b550/3c54e0cf-5dd0-4823-983a-7fb7df6bd8bb.png align="center")

## What Gets Lost in Summarization?

![](https://cdn.hashnode.com/uploads/covers/618f409be61d1c383da7b550/3b5632af-f2bd-4052-89e5-086583527414.png align="center")

## The Deepest Problem: Decision-Rationale Separation

There's a specific failure mode of summarization that's worse than general information loss:

This is **decision-rationale separation**—when compression preserves *what* was decided but loses *why* it was decided. The AI then feels free to override the decision because it doesn't see the constraint behind it.

This is the most dangerous failure in AI-assisted software development because:

*   The user doesn't know the rationale was lost (it's invisible)
    
*   The AI sounds confident (it doesn't know it's contradicting anything)
    
*   The result is wrong code, architectural inconsistencies, or compliance violations
    

## Beyond Summarization

Summarization is a massive improvement over simple dropping, but it still has an inherent ceiling: you're **compressing lossy**. Some information will always be lost.

What if, instead of trying to fit everything into the window, we could **reach outside it**?

This brings us to modern context engineering: RAG, tool use, and memory systems.

* * *

*Read Part 5: RAG, Tools, and the Context Engineering Stack to learn how modern approaches extend beyond the context window.*

#AI #Summarization #ContextEngineering #LLM