AtoM Heratio — AI Condition Assessment

Product: AtoM Heratio Framework v2.8.2 Component: ahgConditionPlugin + ConditionAIService Date: 16 March 2026 Author: The Archive and Heritage Group (Pty) Ltd

Overview

AI Condition Assessment uses a local vision model (LLaVA) to analyze photographs of cultural heritage objects and generate structured condition reports. The system runs entirely on-premises — no cloud services, no API fees, no data leaves your network.

How It Works

┌──────────────┐     ┌──────────────┐     ┌──────────────────────┐
│  Condition    │     │   Ollama     │     │  Database             │
│  Photo        │────>│   LLaVA:7b   │────>│  spectrum_condition_  │
│  (uploaded)   │     │  (local LLM) │     │  check + condition_   │
│               │     │              │     │  damage               │
└──────────────┘     └──────────────┘     └──────────────────────┘
       │                    │                       │
       │              Conservation-            Structured:
       │              specific prompt          • Rating
       │              with material            • Severity
       │              type hint                • Damage types
       │                                       • Description
       │                                       • Recommendations
       ▼
┌──────────────────────────────────────────────────────────┐
│  UI: Rating badge, Severity badge, Damage list,          │
│       Recommendations, link to full condition check      │
└──────────────────────────────────────────────────────────┘

User Workflow

1. Navigate to a record with a condition assessment
2. Go to Condition Photos
3. Click the robot icon (AI Scan) on any photo
4. Wait 2-30 seconds (GPU vs CPU)
5. Review the AI assessment: rating, damage types, severity, recommendations
6. Assessment is automatically saved to the database

Damage Types Detected

15 Spectrum 5.1 damage types with 30+ synonym mappings:

Rating Scale

Severity Levels

Technology Stack

No cloud dependency. No API keys. No usage fees. No data leaves your network.

Performance

To point to 115: AHG Settings > Voice & AI > Local LLM URL → http://192.168.0.115:11434

Current Accuracy

Overall: 60-70% accuracy with prompt engineering alone. Suitable for first-pass triage. Expert review still required for formal condition reports.

Improving Accuracy — Fine-Tuning

LoRA Fine-Tuning Path

Fine-tuning LLaVA with conservation images improves accuracy to 85-90%. The fine-tuned model still runs locally with no external dependencies.

Requirements:

• 200-500 annotated condition photos (more = better)
• GPU with 10GB+ VRAM (RTX 3080 on server 115)
• 4-8 hours training time
• Result: conservation-specific model weights (~100MB LoRA adapter)

Training Data Sources

Practical Path

1. Start with your own data — you already have condition photos with expert damage annotations in spectrum_condition_photo.annotations (JSON with damage categories, coordinates, severity)
2. Export 200-500 annotated photo/label pairs from your DB
3. LoRA fine-tune LLaVA on 115 (RTX 3080 can fine-tune 7B model with LoRA in ~4-8 hours on 500 images)
4. Result: A conservation-specific vision model that runs locally, knows your terminology, and improves with every assessment your staff does

Scaling Up

Recommendation: LoRA fine-tuning the 7B model gives the best accuracy-to-speed ratio for your hardware.

Database Integration

Tables Written

API Endpoint

POST /condition/ai-assess
Content-Type: application/x-www-form-urlencoded

Parameters:
  photo_id      — spectrum_condition_photo ID
  object_id     — information_object ID
  material_type — paper, textile, metal, etc. (optional, improves accuracy)

Response:
{
  "success": true,
  "overall_rating": "fair",
  "severity": "moderate",
  "damage_types": [
    {"type": "tear", "severity": "moderate", "location": "overall"},
    {"type": "stain", "severity": "minor", "location": "overall"}
  ],
  "description": "The document shows moderate wear with a visible tear along the left edge and light staining in the upper right quadrant.",
  "recommendations": "Repair the tear with Japanese tissue and wheat starch paste. Surface clean the stained area with a soft brush.",
  "condition_check_id": 42,
  "raw_response": "..."
}

Files

AtoM Heratio Framework v2.8.2 — The Archive and Heritage Group (Pty) Ltd