domus-semper-palingenesis

Universal File Organization Strategy

Solving the “Messy System” Problem with Contextual Awareness

Problem: Files scattered between /, ~/, external drives, with no clear organization or context.

Solution: A layered approach combining XDG compliance, intelligent categorization, and external drive integration.

---

🎯 Goals

1. Contextual Awareness - System “knows” what files are and where they belong
2. XDG Compliance - Follow modern Linux/macOS standards
3. External Drive Integration - Seamless access to files on external storage
4. Deduplication - Identify and remove duplicate files
5. Discoverability - Find files by purpose, not just by path
6. Portability - Works across machines and drive connections

---

📐 Architecture: The Three-Layer System

Layer 1: Local System (Fast, Always Available)

Purpose: Active working files, configs, and caches

~/
├── .config/              # XDG: Application configs
├── .local/
│   ├── bin/             # User executables
│   ├── share/           # Application data
│   └── state/           # Logs, history, state
├── .cache/              # Temporary/regenerable data
│
├── Documents/           # Active documents
├── Projects/            # Active development work
├── Downloads/           # Temporary incoming files
├── Desktop/             # Working surface (keep minimal)
│
└── Archive/             # Local archive (symlink to external)

Layer 2: External Storage (Large, Periodic Access)

Purpose: Archives, media libraries, backups

/Volumes/4444-iivii/     # or ~/External (symlinked)
├── Archive/
│   ├── Documents/       # Old documents by year
│   ├── Projects/        # Completed projects
│   ├── Media/           # Photos, videos, music
│   └── System/          # Old system backups
│
├── Workspace/           # Development environment
│   ├── toolchains/
│   └── repos/
│
├── Library/             # Reference materials
│   ├── Books/
│   ├── Courses/
│   └── Documentation/
│
└── Backups/             # Time Machine, system backups

Layer 3: Metadata/Context (Knowledge Layer)

Purpose: Tag, categorize, and find files by purpose

~/.local/share/file-context/
├── manifests/           # File inventories
│   ├── archive.yaml
│   ├── projects.yaml
│   └── media.yaml
│
├── tags/                # Tag database
│   └── tags.db          # SQLite tags database
│
└── links/               # Symlink farm by category
    ├── by-type/
    │   ├── code/
    │   ├── docs/
    │   └── media/
    ├── by-project/
    │   └── [project-name]/
    └── by-year/
        └── [2024]/

---

🗂️ XDG Base Directory Specification

Current State Analysis

Already XDG-Compliant:

• ✅ ~/.config/ - Configurations (87MB)
• ✅ ~/.local/share/chezmoi/ - Dotfiles source
• ✅ ~/.cache/ - Cached data

Needs Migration:

• ❌ Root-level files: AGENTS.md, README.md, firebase-debug.log, package-lock.json
• ❌ ~/System/ (1.9GB) - Should be in ~/.local/share/ or external
• ❌ ~/Workspace/ (7.8GB) - Should be ~/Projects/ or ~/Documents/Projects/

Proposed XDG Structure

# Configs (already good)
~/.config/                    # App configurations
  ├── chezmoi/
  ├── gh/
  ├── kitty/
  └── ...

# Data (consolidate here)
~/.local/share/               # Application data
  ├── chezmoi/               # Dotfiles (current)
  ├── file-context/          # NEW: File organization metadata
  ├── projects/              # NEW: Symlinks to active projects
  └── ...

# State (logs, history)
~/.local/state/               # Persistent state
  ├── command-history/
  ├── logs/
  └── ...

# Cache (regenerable)
~/.cache/                     # Temporary cached data
  ├── pip/
  ├── npm/
  └── ...

# User directories (visible, active)
~/Documents/                  # Active documents
~/Projects/                   # Active development (was Workspace)
~/Downloads/                  # Temporary incoming
~/Desktop/                    # Working surface
~/Pictures/                   # Active photos
~/Music/                      # Active music

---

🔗 External Drive Integration Strategy

Problem: Hardcoded Paths Break

Current issue in your .zshrc:

export PATH=$PATH:/Volumes/4444-iivii/ivi374forivi3ivi3/toolchains/bin

What breaks:

• Drive not mounted → PATH broken
• Different machine → Drive might have different name
• Drive mounted at different path → Still broken

Solution 1: Conditional Path with Fallback

# In .zshrc.tmpl
# External toolchains (if available)
if [ -d "/Volumes/4444-iivii/ivi374forivi3ivi3/toolchains/bin" ]; then
  export PATH="$PATH:/Volumes/4444-iivii/ivi374forivi3ivi3/toolchains/bin"
fi

Solution 2: Symlink Strategy (Recommended)

Create stable symlinks in your home directory:

# One-time setup
ln -s /Volumes/4444-iivii ~/External
ln -s ~/External/ivi374forivi3ivi3/workspace ~/Projects/External
ln -s ~/External/ivi374forivi3ivi3/toolchains ~/.local/share/toolchains

# In .zshrc.tmpl
if [ -d "$HOME/.local/share/toolchains/bin" ]; then
  export PATH="$PATH:$HOME/.local/share/toolchains/bin"
fi

Benefits:

• Stable paths that don’t break
• Works even when drive unmounted (just broken symlink)
• Can be managed by chezmoi
• Easy to update when drive changes

Solution 3: Environment Variable Approach

# In ~/.config/environment (sourced by .zshrc)
export EXTERNAL_DRIVE="/Volumes/4444-iivii"
export WORKSPACE="$EXTERNAL_DRIVE/ivi374forivi3ivi3/workspace"
export TOOLCHAINS="$EXTERNAL_DRIVE/ivi374forivi3ivi3/toolchains"

# Then use variables
[ -d "$TOOLCHAINS/bin" ] && export PATH="$PATH:$TOOLCHAINS/bin"

---

🧠 Contextual Awareness System

The “What is This?” Problem

Files need context:

• What is it? (type, purpose)
• Why is it here? (project, reason)
• When was it used? (active, archived)
• Where else is it? (duplicates, backups)

Solution: File Manifest + Tag System

1. Create File Manifests

# ~/.local/share/file-context/manifests/projects.yaml
projects:
  active:
    - name: "domus-semper-palingenesis"
      path: "~/Documents/domus-semper-palingenesis"
      external_backup: "~/External/Archive/domus-semper-palingenesis"
      type: "configuration"
      tags: ["system", "critical", "synced"]

    - name: "ivi374forivi3ivi3"
      path: "~/External/ivi374forivi3ivi3/workspace"
      symlink: "~/Projects/ivi374"
      type: "development"
      tags: ["work", "external", "large"]

  archived:
    - name: "old-project-2023"
      path: "~/External/Archive/Projects/2023/old-project"
      archived_date: "2023-12-01"
      tags: ["archived", "reference"]

2. Tag Database (tmsu or custom)

Install tmsu (tag manager for Unix):

brew install tmsu

# Initialize
tmsu init

# Tag files
tmsu tag ~/Documents/important-doc.pdf important work active
tmsu tag ~/Pictures/vacation-2024/ personal photos 2024

# Query by tags
tmsu files important work
tmsu files photos 2024

3. Smart Symlink Farm

Auto-organize files by creating categorized symlinks:

# ~/.local/share/file-context/links/by-type/
~/. local/share/file-context/links/
├── by-type/
│   ├── code/
│   │   ├── domus-semper-palingenesis -> ~/Documents/domus-semper-palingenesis
│   │   └── ivi374 -> ~/External/workspace
│   ├── docs/
│   │   └── important -> ~/Documents/Work/Important/
│   └── media/
│       └── photos-2024 -> ~/Pictures/2024/
├── by-project/
│   └── ivi374/
│       ├── workspace -> ~/External/workspace
│       └── toolchains -> ~/External/toolchains
└── by-status/
    ├── active/
    └── archived/

---

🛠️ Implementation Tools

Tool 1: organize-cli (Python)

pip install organize-tool

# Create rules in ~/.config/organize/config.yaml
rules:
  - name: "Clean Downloads"
    locations:
      - ~/Downloads/
    filters:
      - extension: [pdf, epub]
      - created: {days: 7}
    actions:
      - move: ~/Documents/Reading/{created.year}/

  - name: "Archive Old Projects"
    locations:
      - ~/Projects/
    filters:
      - lastmodified: {days: 90}
    actions:
      - echo: "Consider archiving {path}"

Tool 2: rmlint (Deduplication)

brew install rmlint

# Find duplicates
rmlint ~/Documents ~/External/Archive

# Remove duplicates (careful!)
# rmlint creates a shell script you can review first
sh rmlint.sh -d  # dry run

Tool 3: Custom Context Manager Script

# ~/.local/bin/file-context
#!/bin/bash

case "$1" in
  index)
    # Index all files and create manifest
    ;;
  find)
    # Find files by tag or purpose
    ;;
  archive)
    # Move file to archive with metadata
    ;;
  link)
    # Create symlinks in categorized folders
    ;;
esac

---

📋 Chezmoi Integration

Managing Symlinks with Chezmoi

# Add symlinks to chezmoi
chezmoi add --follow=false ~/.local/share/toolchains

# This creates a .chezmoi.symlink file
# ~/.local/share/chezmoi/private_dot_local/share/symlink_toolchains.tmpl
/Volumes/4444-iivii/ivi374forivi3ivi3/toolchains

Directory Structure Template

# ~/.local/share/chezmoi/.chezmoiscripts/run_once_setup_structure.sh
#!/bin/bash

# Create XDG directories
mkdir -p ~/.config
mkdir -p ~/.local/{bin,share,state}
mkdir -p ~/.cache

# Create project structure
mkdir -p ~/Projects/{Active,Archive}
mkdir -p ~/Documents/{Work,Personal,Archive}

# Create context system
mkdir -p ~/.local/share/file-context/{manifests,tags,links}
mkdir -p ~/.local/share/file-context/links/{by-type,by-project,by-year}

# Create symlinks if external drive mounted
if [ -d "/Volumes/4444-iivii" ]; then
  ln -sf /Volumes/4444-iivii ~/External
  ln -sf ~/External/ivi374forivi3ivi3/workspace ~/Projects/ivi374
  ln -sf ~/External/ivi374forivi3ivi3/toolchains ~/.local/share/toolchains
fi

---

🎬 Action Plan: Clean Up Your System

Phase 1: Inventory (Do First)

# 1. List all root-level files in home
ls -la ~/ | grep -v "^d" > ~/file-inventory.txt

# 2. Identify large directories
du -sh ~/* | sort -hr > ~/directory-sizes.txt

# 3. Find duplicates
rmlint ~/ --size 1M > ~/duplicates.txt

Phase 2: Categorize

Move scattered files to proper locations:

# Root-level files
AGENTS.md → ~/Documents/Notes/AGENTS.md
README.md → ~/Projects/[project]/README.md (or delete if from chezmoi)
firebase-debug.log → ~/.local/state/firebase/debug.log (or delete)
package-lock.json → ~/Projects/[project]/package-lock.json

# System folder
~/System/ → ~/External/Archive/System/ (or ~/.local/share/system/)

# Workspace
~/Workspace/ → ~/Projects/ (rename for clarity)

Phase 3: Create Structure

# Create the new organization
mkdir -p ~/Projects/{Active,Archive}
mkdir -p ~/Documents/{Work,Personal,Archive,Notes}
mkdir -p ~/.local/share/file-context/{manifests,links}

# Move active projects
mv ~/Workspace/active-project ~/Projects/Active/

# Archive old projects
mv ~/Workspace/old-project ~/External/Archive/Projects/2023/

Phase 4: Setup Symlinks

# Create stable external drive symlink
ln -s /Volumes/4444-iivii ~/External

# Link active external projects to local
ln -s ~/External/ivi374forivi3ivi3/workspace ~/Projects/ivi374

# Link toolchains
ln -s ~/External/ivi374forivi3ivi3/toolchains ~/.local/share/toolchains

Phase 5: Add to Chezmoi

# Add directory creation script
chezmoi add --template ~/.zshrc  # Already done
chezmoi add --follow=false ~/.local/share/toolchains  # Add symlink

# Create setup script
cat > ~/.local/share/chezmoi/.chezmoiscripts/run_once_setup_structure.sh <<'EOF'
#!/bin/bash
# Setup directory structure
mkdir -p ~/Projects/{Active,Archive}
mkdir -p ~/.local/share/file-context
# ... etc
EOF

chmod +x ~/.local/share/chezmoi/.chezmoiscripts/run_once_setup_structure.sh
chezmoi add ~/.chezmoiscripts/run_once_setup_structure.sh

---

🎪 Example: Complete Transformation

Before (Messy)

~/
├── AGENTS.md                    # ← What is this?
├── Workspace/                   # ← Mixed active/old projects
├── System/                      # ← 1.9GB of what?
├── package-lock.json            # ← Why in root?
└── /Volumes/4444-iivii/...      # ← Hardcoded paths break

After (Organized)

~/
├── .config/                     # ✅ Configurations
├── .local/
│   ├── bin/                     # ✅ User executables
│   ├── share/
│   │   ├── file-context/        # ✅ Organization metadata
│   │   └── toolchains/ →        # ✅ Symlink to external
│   └── state/                   # ✅ Logs and state
│
├── Projects/
│   ├── Active/                  # ✅ Current work
│   │   └── my-app/
│   ├── ivi374 →                 # ✅ Symlink to external
│   └── Archive/                 # ✅ Old projects
│
├── Documents/
│   ├── Work/
│   ├── Personal/
│   ├── Notes/
│   │   └── AGENTS.md            # ✅ Found it!
│   └── Archive/
│
└── External →                   # ✅ Stable symlink
    └── /Volumes/4444-iivii/

---

🚀 Next Steps

1. Start Small - Don’t reorganize everything at once
2. Inventory First - Know what you have
3. One Category at a Time - Documents, then Projects, then Media
4. Test Symlinks - Make sure they work before deleting originals
5. Add to Chezmoi - Make it reproducible
6. Automate Maintenance - Scripts to keep it organized

---

📚 Additional Resources

• XDG Base Directory Spec: https://specifications.freedesktop.org/basedir-spec/
• tmsu (tagging): https://tmsu.org/
• organize-tool: https://organize.readthedocs.io/
• rmlint (dedup): https://rmlint.readthedocs.io/

---

Remember: Perfect organization is the enemy of good organization. Start with the critical paths (external drives, PATH variables), then gradually improve the rest.

This site is open source. Improve this page.