WesandClaude Sonnet 4.5
27afedb32e
Fetches SMART data from Scrutiny's InfluxDB and calculates failure probabilities using the exact BackBlaze tables from SnapRAID source. Key features: - Calculates both v12 (with attr 193) and v13 (without attr 193) algorithms - v12 matches SnapRAID pre-v13.0 (includes Load Cycle Count) - v13 matches modern SnapRAID v13.0+ (excludes Load Cycle Count) - Stores both values for trending and comparison - Correctly applies bit masks (16-bit for 187/188, 32-bit for others) - Annualizes monthly rates and applies Poisson distribution - Logs to PostgreSQL with device metadata Solved the mystery: SnapRAID v12 uses attribute 193 which was removed in v13.0. Load cycle count has massive impact on failure predictions for some drives (80% vs 4% difference). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
96 lines
3.2 KiB
Markdown
96 lines
3.2 KiB
Markdown
# Scrutiny InfluxDB Schema Notes
|
|
|
|
Based on analysis of your `explore_output.txt`, here's what I discovered:
|
|
|
|
## InfluxDB Structure
|
|
|
|
### Measurements
|
|
- `smart` - SMART attribute data
|
|
- `temp` - Temperature data (simpler format)
|
|
|
|
### Tags (Device Identifiers)
|
|
- `device_wwn` - World Wide Name, unique identifier (e.g., `0x5000c500744487c5`)
|
|
- `device_protocol` - Protocol type (`ATA`, `NVME`, etc.)
|
|
|
|
### Fields in `smart` Measurement
|
|
|
|
**Basic Metrics:**
|
|
- `temp` - Temperature in Celsius
|
|
- `power_on_hours` - Total power-on hours
|
|
- `power_cycle_count` - Number of power cycles
|
|
|
|
**SMART Attributes** (pattern: `attr.{id}.{property}`):
|
|
|
|
For each SMART attribute ID (e.g., 5, 187, 188, 194, 197, 198), Scrutiny stores:
|
|
- `attr.{id}.attribute_id` - The attribute ID itself
|
|
- `attr.{id}.raw_value` - **Raw value (what we need for calculations!)**
|
|
- `attr.{id}.raw_string` - String representation
|
|
- `attr.{id}.transformed_value` - Normalized value (0-100)
|
|
- `attr.{id}.value` - Current value
|
|
- `attr.{id}.thresh` - Threshold
|
|
- `attr.{id}.worst` - Worst value seen
|
|
- `attr.{id}.status` - Status indicator
|
|
- `attr.{id}.status_reason` - Status explanation
|
|
- `attr.{id}.failure_rate` - Scrutiny's own failure rate calculation
|
|
- `attr.{id}.when_failed` - Failure timestamp (if applicable)
|
|
|
|
## Key SMART Attributes for Failure Prediction
|
|
|
|
BackBlaze analysis (used by SnapRAID) focuses on these attributes:
|
|
|
|
| ID | Name | Description |
|
|
|-----|------|-------------|
|
|
| 5 | Reallocated Sectors Count | Sectors remapped due to errors |
|
|
| 187 | Reported Uncorrectable Errors | Errors that couldn't be corrected |
|
|
| 188 | Command Timeout | Commands that timed out |
|
|
| 197 | Current Pending Sector Count | Sectors waiting for remapping |
|
|
| 198 | Offline Uncorrectable Sector Count | Errors found during offline scan |
|
|
|
|
## Your Devices (from explore_output.txt)
|
|
|
|
Found 5 devices with these WWNs:
|
|
1. `0x5000c500744487c5`
|
|
2. `0x5000c500c455c45e`
|
|
3. `0x5000c500c46b0832`
|
|
4. `0x5000c500c570663e`
|
|
5. `0x5000c500e584b9c8`
|
|
|
|
## Device Metadata Location
|
|
|
|
**Important:** Serial number, model, device path (`/dev/sdX`) are **NOT** stored in InfluxDB!
|
|
|
|
They are stored in:
|
|
1. **Scrutiny's SQLite database** (`/opt/scrutiny/config/scrutiny.db`)
|
|
2. **Scrutiny's REST API** endpoints:
|
|
- `/api/summary` - All devices with metadata
|
|
- `/api/device/{wwn}/smart` - Individual device data
|
|
|
|
## Solution Used
|
|
|
|
The `smart_logger_v2.py` script:
|
|
1. **Fetches device metadata** from Scrutiny's `/api/summary` endpoint
|
|
2. **Fetches SMART data** from InfluxDB (faster than API for historical queries)
|
|
3. **Calculates failure probability** using BackBlaze tables
|
|
4. **Exports to PostgreSQL** with full metadata
|
|
|
|
## Example InfluxDB Query
|
|
|
|
```flux
|
|
from(bucket: "metrics")
|
|
|> range(start: -1h)
|
|
|> filter(fn: (r) => r._measurement == "smart")
|
|
|> filter(fn: (r) =>
|
|
r._field == "temp" or
|
|
r._field == "power_on_hours" or
|
|
r._field == "attr.5.raw_value" or
|
|
r._field == "attr.187.raw_value" or
|
|
r._field == "attr.188.raw_value" or
|
|
r._field == "attr.197.raw_value" or
|
|
r._field == "attr.198.raw_value"
|
|
)
|
|
|> last()
|
|
|> pivot(rowKey:["device_wwn"], columnKey: ["_field"], valueColumn: "_value")
|
|
```
|
|
|
|
This groups all fields by device WWN, making it easy to get all SMART data per device.
|