Files

T

WesandClaude Sonnet 4.5 27afedb32e Complete rewrite: SnapRAID SMART logger with BackBlaze algorithm

Fetches SMART data from Scrutiny's InfluxDB and calculates failure
probabilities using the exact BackBlaze tables from SnapRAID source.

Key features:
- Calculates both v12 (with attr 193) and v13 (without attr 193) algorithms
- v12 matches SnapRAID pre-v13.0 (includes Load Cycle Count)
- v13 matches modern SnapRAID v13.0+ (excludes Load Cycle Count)
- Stores both values for trending and comparison
- Correctly applies bit masks (16-bit for 187/188, 32-bit for others)
- Annualizes monthly rates and applies Poisson distribution
- Logs to PostgreSQL with device metadata

Solved the mystery: SnapRAID v12 uses attribute 193 which was removed
in v13.0. Load cycle count has massive impact on failure predictions
for some drives (80% vs 4% difference).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

2025-12-07 06:55:39 -05:00

3.2 KiB

Raw Blame History

Scrutiny InfluxDB Schema Notes

Based on analysis of your explore_output.txt, here's what I discovered:

InfluxDB Structure

Measurements

smart - SMART attribute data
temp - Temperature data (simpler format)

Tags (Device Identifiers)

device_wwn - World Wide Name, unique identifier (e.g., 0x5000c500744487c5)
device_protocol - Protocol type (ATA, NVME, etc.)

Fields in `smart` Measurement

Basic Metrics:

temp - Temperature in Celsius
power_on_hours - Total power-on hours
power_cycle_count - Number of power cycles

SMART Attributes (pattern: attr.{id}.{property}):

For each SMART attribute ID (e.g., 5, 187, 188, 194, 197, 198), Scrutiny stores:

attr.{id}.attribute_id - The attribute ID itself
attr.{id}.raw_value - Raw value (what we need for calculations!)
attr.{id}.raw_string - String representation
attr.{id}.transformed_value - Normalized value (0-100)
attr.{id}.value - Current value
attr.{id}.thresh - Threshold
attr.{id}.worst - Worst value seen
attr.{id}.status - Status indicator
attr.{id}.status_reason - Status explanation
attr.{id}.failure_rate - Scrutiny's own failure rate calculation
attr.{id}.when_failed - Failure timestamp (if applicable)

Key SMART Attributes for Failure Prediction

BackBlaze analysis (used by SnapRAID) focuses on these attributes:

ID	Name	Description
5	Reallocated Sectors Count	Sectors remapped due to errors
187	Reported Uncorrectable Errors	Errors that couldn't be corrected
188	Command Timeout	Commands that timed out
197	Current Pending Sector Count	Sectors waiting for remapping
198	Offline Uncorrectable Sector Count	Errors found during offline scan

Your Devices (from explore_output.txt)

Found 5 devices with these WWNs:

0x5000c500744487c5
0x5000c500c455c45e
0x5000c500c46b0832
0x5000c500c570663e
0x5000c500e584b9c8

Device Metadata Location

Important: Serial number, model, device path (/dev/sdX) are NOT stored in InfluxDB!

They are stored in:

Scrutiny's SQLite database (/opt/scrutiny/config/scrutiny.db)
Scrutiny's REST API endpoints:
- /api/summary - All devices with metadata
- /api/device/{wwn}/smart - Individual device data

Solution Used

The smart_logger_v2.py script:

Fetches device metadata from Scrutiny's /api/summary endpoint
Fetches SMART data from InfluxDB (faster than API for historical queries)
Calculates failure probability using BackBlaze tables
Exports to PostgreSQL with full metadata

Example InfluxDB Query

from(bucket: "metrics")
    |> range(start: -1h)
    |> filter(fn: (r) => r._measurement == "smart")
    |> filter(fn: (r) =>
        r._field == "temp" or
        r._field == "power_on_hours" or
        r._field == "attr.5.raw_value" or
        r._field == "attr.187.raw_value" or
        r._field == "attr.188.raw_value" or
        r._field == "attr.197.raw_value" or
        r._field == "attr.198.raw_value"
    )
    |> last()
    |> pivot(rowKey:["device_wwn"], columnKey: ["_field"], valueColumn: "_value")

This groups all fields by device WWN, making it easy to get all SMART data per device.

3.2 KiB Raw Blame History