SnapRAID SMART Logger

A Python utility that collects SMART data from Scrutiny's InfluxDB, calculates BackBlaze-based failure probabilities (like SnapRAID does), and logs the enhanced data to PostgreSQL.

Features

• ✅ Connects to Scrutiny's InfluxDB to fetch SMART attributes
• ✅ Implements SnapRAID's BackBlaze failure probability algorithm
• ✅ Calculates annual failure rate (AFR) for each disk
• ✅ Stores historical data in PostgreSQL with time-series support
• ✅ Provides views for latest metrics and high-risk devices

Quick Start

1. Install Dependencies

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

2. Configure Environment

cp .env.example .env
# Edit .env with your actual values

Required configuration:

• INFLUXDB_URL: Your Scrutiny InfluxDB URL (e.g., http://192.168.1.100:8086)
• INFLUXDB_TOKEN: Your Scrutiny InfluxDB token
• POSTGRES_*: Your PostgreSQL connection details

3. Set Up PostgreSQL Schema

psql -h <host> -U <user> -d <database> -f schema.sql

4. Explore Scrutiny's InfluxDB Schema

IMPORTANT: Run this first to understand how Scrutiny stores SMART data:

python explore_influx.py

This will show you:

• Available measurements in the metrics bucket
• Field names for each measurement
• Sample data structure
• Tags used for device identification

5. Customize the Logger

Based on the output from explore_influx.py, update smart_logger.py:

1. Update the InfluxDB query (fetch_smart_data method):

   • Set correct _measurement name
   • Map field names to device properties (temp, serial, model, etc.)
   • Map SMART attribute fields to IDs (5, 187, 188, 197, 198)

2. Adjust device detection:

   • Update how device path, serial, and model are extracted
   • Add disk role mapping if you want to match SnapRAID roles

6. Run the Logger

python smart_logger.py

Output Format

The logger will display a summary like SnapRAID:

Device: /dev/sda
  Serial: ZX20AAMV
  Model: WDC WD200EFAX-68FB5N0
  Temp: 25°C
  Power-On: 636 days
  Failure Probability: 49.00%

================================================================================
Summary:
================================================================================
🔴 HIGH RISK /dev/sdc: 81.00% failure probability
🟢 OK /dev/sdb: 22.00% failure probability

PostgreSQL Schema

Tables

• smart_metrics: Main table storing all SMART data snapshots
  • Includes temperature, power-on time, error counts
  • Stores calculated failure probability
  • Raw SMART attributes as JSONB for flexibility

Views

• smart_latest: Latest metrics for each device
• smart_high_risk: Devices with >50% annual failure probability

Example Queries

-- Get latest status for all devices
SELECT device_path, serial_number, temperature_celsius,
       failure_probability_pct, timestamp
FROM smart_latest
ORDER BY failure_probability_pct DESC;

-- Historical trend for a specific disk
SELECT timestamp, temperature_celsius, failure_probability_pct
FROM smart_metrics
WHERE serial_number = 'ZX20AAMV'
ORDER BY timestamp DESC
LIMIT 100;

-- Check specific SMART attribute over time
SELECT timestamp,
       (smart_attributes->>'5')::int as reallocated_sectors,
       (smart_attributes->>'197')::int as pending_sectors
FROM smart_metrics
WHERE device_path = '/dev/sda'
ORDER BY timestamp DESC;

BackBlaze Algorithm

The failure probability calculation is based on SnapRAID's implementation, which uses BackBlaze's 2014 dataset (47,322 disk observations) to create lookup tables for these SMART attributes:

• Attribute 5: Reallocated Sectors Count
• Attribute 187: Reported Uncorrectable Errors
• Attribute 188: Command Timeout (excluded for Seagate disks)
• Attribute 197: Current Pending Sector Count
• Attribute 198: Offline Uncorrectable Sector Count

The algorithm returns the maximum AFR across all attributes (since they're correlated, not independent).

Automation

To run this periodically, add a cron job:

# Run every hour
0 * * * * /path/to/venv/bin/python /path/to/smart_logger.py >> /var/log/smart_logger.log 2>&1

Or use systemd timer, or your preferred scheduler.

Troubleshooting

InfluxDB Connection Issues

• Verify the URL and port are correct
• Check that the token has read permissions for the metrics bucket
• Ensure InfluxDB is accessible from the machine running this script

No Data Returned

• Run explore_influx.py to check the data structure
• Verify Scrutiny is actively collecting data
• Check the time range in the query (-5m might be too short)

PostgreSQL Insert Failures

• Ensure the schema was created (schema.sql)
• Check that the database user has INSERT permissions
• Verify data types match (especially JSONB for smart_attributes)

License

MIT License - feel free to modify and use as needed!

Credits

• BackBlaze failure probability tables extracted from SnapRAID
• Based on BackBlaze's 2014 hard drive reliability dataset