Skip to content

Glaber ValueCache User Guide

Overview

The ValueCache is an advanced in-memory caching system in Glaber that stores historical monitoring data for items. It provides high-performance access to recent and historical data while intelligently managing memory usage based on actual usage patterns.

Key Features

1. Two-Tier Cache Architecture

The ValueCache uses a two-tier system for optimal performance and memory efficiency:

  • Raw Cache: Stores recent, high-resolution data points as they arrive from monitored systems
  • Downsampled Cache: Stores older data in aggregated form, reducing memory footprint while preserving historical trends

2. Intelligent Demand-Based Management

The system learns usage patterns and automatically adjusts cache sizes based on actual data requirements:

  • Tracks how far back in time users query (timeshift)
  • Monitors how many values are typically requested (count)
  • Adapts cache allocation to match real usage patterns

3. Automatic Database Fetching

When requested data is not in cache, the system automatically: - Fetches missing data from the database - Populates both raw and downsampled caches - Ensures subsequent requests are served from memory

4. Persistence Support

Cache contents can be dumped to disk and restored, allowing: - Fast system restarts without losing cached data - Maintenance operations without data loss - State preservation across updates

Configuration Parameters

Configure these parameters in zabbix_server.conf or zabbix_proxy.conf:

ValueCacheSize

ValueCacheSize=256M
Default: 256 MB
Range: 128 KB to 64 GB
Description: Total shared memory allocated for the ValueCache. This memory is shared across all monitored items.

ValueCacheDefaultElements

ValueCacheDefaultElements=10
Default: 10
Range: 1 to 1024
Description: Initial number of elements allocated for each item's raw cache. The cache automatically grows based on demand.

ValueCacheMaxDuration

ValueCacheMaxDuration=172800
Default: 172800 seconds (2 days)
Range: Any positive value in seconds
Description: Maximum time period for which raw (high-resolution) data is kept in cache. Older data is moved to the downsampled cache.

  • Values within this duration: returned from raw cache (full resolution)
  • Values older than this: returned from downsampled cache (aggregated)

ValueCacheMinCleanupValues

ValueCacheMinCleanupValues=128
Default: 128
Range: 2 to any reasonable number
Description: Minimum number of values to keep in cache during cleanup operations. Prevents excessive cache clearing.

How It Works

Data Flow

1. New Value Arrives
   ↓
2. Added to Raw Cache (recent, full-resolution data)
   ↓
3. When sufficient old data accumulates
   ↓
4. Older data aggregated and moved to Downsampled Cache
   ↓
5. Cleanup removes obsolete data based on demand

Query Processing

When you request historical data (e.g., in dashboards, triggers, or API calls):

  1. Cache Check: System checks if data is available in cache
  2. Time-Based Decision:
  3. Recent data (within ValueCacheMaxDuration): served from raw cache
  4. Older data: served from downsampled cache
  5. Database Fetch: If data is missing, automatically fetched from DB and cached
  6. Demand Update: System records your query pattern to optimize future caching

Demand Tracking

The system tracks three types of demand for each item:

  • Count Demand: How many values are typically requested
  • Period Demand: How far back in time queries go
  • Timeshift Demand: If queries are offset from current time

The cache automatically grows or shrinks based on these patterns: - Frequently accessed items get more cache space - Rarely accessed items use minimal memory - Cache adapts over a 24-hour learning period

Supported Data Types

The ValueCache supports the following Zabbix item types:

Type Storage Downsampling
Numeric (uint64) Full support Yes (averaged)
Float Full support Yes (averaged)
String Raw cache only No
Text Raw cache only No
Log Raw cache only No

Note: Text-based types (string, text, log) are not downsampled because averaging doesn't make sense for textual data.

Cache Operations

Viewing Cache Statistics

You can view ValueCache statistics through the Glaber internal API or diagnostic tools.

Key Metrics: - Total items cached - Memory usage per item - Cache hit/miss ratios - Demand patterns

Cache Dump and Restore

Manual Dump: The cache can be dumped to JSON format for backup or analysis. Location configured via VCDumpLocation parameter.

Format: Each item's cache contains: - Metadata (value type, last access time) - Demand information (usage patterns) - Raw values (recent data) - Downsampled values (historical aggregates)

Automatic Restore: On startup, if dump files exist, they are automatically loaded to warm up the cache.

Cleanup Behavior

The cache automatically cleans up old data:

  1. Time-Based Cleanup:
  2. Raw cache: removes data older than ValueCacheMaxDuration

  3. Downsampled cache: removes data no longer needed based on demand

  4. Demand-Based Cleanup:

  5. If demand is met, oldest values are removed

  6. If demand is not met, cache grows (within global memory limits)

  7. Minimum Retention:

  8. At least ValueCacheMinCleanupValues are kept per item
  9. Ensures some data always available even for rarely-queried items

Historical Functions and Downsampled Cache

Important: All historical functions in Glaber (such as avg(), min(), max(), last(), etc.) automatically benefit from the downsampled cache when querying long time periods. This provides significant performance improvements for historical data queries without any configuration changes.

How Historical Functions Work

When you use any historical function in triggers, calculated items, or API queries:

  1. Short periods (within ValueCacheMaxDuration): Data served from raw cache with full resolution
  2. Long periods (beyond ValueCacheMaxDuration): Automatically combines:
  3. Recent data from raw cache (full resolution)
  4. Historical data from downsampled cache (aggregated)

Example: 

avg(/host/item.key, 7d)  // Automatically uses downsampled cache for older data

Feature Regular Historical Functions Trends Functions
Time Periods Any arbitrary period (e.g., 12345 seconds, 3h 27m) Only complete periods (1h, 1d, 1w)
Period Alignment Can start/end at any timestamp Must align to period boundaries
Use Cases Real-time monitoring, flexible queries Reporting, fixed-interval analysis
Current Period Includes incomplete current data Excludes incomplete current period

Examples: 

// Regular functions - flexible time periods
avg(/host/cpu, 10m:now-1d)                  // Average over 10 minutes, 1 day ago
min(/host/memory, 2h:now-30m)               // Minimum over 2 hours, 30 minutes ago
last(/host/disk, #5:now-1h)                 // 5th last value from 1 hour ago
avg(/host/temp, 12345s:now-2h)              // Average over 12345 seconds, 2 hours ago

// Trends functions - complete periods only  
trendavg(/host/cpu, 1h:now/h-1d)            // Hourly average for last 24 hours
trendmax(/host/memory, 1d:now/d-1w)         // Daily maximum for last week
trendsum(/host/traffic, 1w:now/w-1M)        // Weekly sum for last month
trendmin(/host/temp, 1M:now/M-1y)           // Monthly minimum for last year

Performance Benefits

Both regular historical functions and trends functions benefit from the downsampled cache:

  • Reduced database load: Historical data served from memory
  • Faster query response: No need to scan millions of database rows
  • Consistent performance: Query time doesn't grow linearly with period length

The downsampled cache makes long-term historical queries practical and performant, whether you're using flexible time periods with regular functions or fixed intervals with trends functions.

Best Practices

Memory Sizing

Rule of Thumb: Allocate ValueCacheSize based on: 

Memory = (Number of Active Items) × (Average Values per Item) × (Bytes per Value)

Example Calculation: - 10,000 items - Average 1000 values cached per item - Float values (~16 bytes each) - Required: ~152 MB minimum

Recommendation: Start with 256 MB and monitor usage.

Optimal Configuration

  1. High-Frequency Monitoring (1-second intervals):
  2. Increase ValueCacheMaxDuration (e.g., 7 days)

  3. Increase ValueCacheDefaultElements (e.g., 50-100)

  4. Standard Monitoring (30-60 second intervals):

  5. Default settings usually sufficient
  6. ValueCacheMaxDuration = 2 days

  7. ValueCacheDefaultElements = 10

  8. Low-Frequency Monitoring (5+ minutes):

  9. Can reduce ValueCacheDefaultElements to 5
  10. ValueCacheMaxDuration can be longer (e.g., 7 days)

Performance Optimization

  1. Cache Hit Rate: Monitor cache statistics to ensure high hit rates (>90%)
  2. Demand Patterns: Allow 24 hours for the system to learn usage patterns
  3. Memory Monitoring: Watch for cache evictions indicating insufficient memory

Troubleshooting

Problem: High Memory Usage

Symptoms: Server memory consumption grows continuously

Solutions: 1. Reduce ValueCacheMaxDuration to keep less raw data 2. Increase cleanup frequency 3. Review items with abnormally high demand

Problem: Frequent Database Queries

Symptoms: High DB load, slow dashboard performance

Solutions: 1. Increase ValueCacheSize to cache more items 2. Increase ValueCacheDefaultElements for frequently-queried items 3. Verify demand tracking is working (allow 24h learning period)

Problem: Missing Historical Data

Symptoms: Queries return incomplete data sets

Solutions: 1. Check ValueCacheMaxDuration - may be too short 2. Verify database contains the requested data 3. Check cache dump/restore if after restart

Monitoring the Cache

Key Indicators

Watch these metrics for optimal performance:

  1. Cache Hit Ratio: Should be >90% for good performance
  2. Memory Usage: Should stay under ValueCacheSize limit
  3. Demand Satisfaction: Items should meet their demand requirements
  4. Database Fetch Frequency: Should be minimal for active items

Log Messages

Enable debug logging for specific items: 

DebugItem=123456

Look for log entries like: - "Fetching X seconds from raw cache" - "Cache demand IS MET/IS NOT MET" - "Downsampled cache added X values"

Comparison with Standard Zabbix

Feature Standard Zabbix Glaber ValueCache
Cache Architecture Single-tier Two-tier (raw + downsampled)
Memory Management Fixed allocation Demand-based, adaptive
Historical Data Database only Cached with downsampling
Long-term Queries Always hit DB Served from downsampled cache
Persistence None Dump/restore support
Learning None 24-hour demand learning

Summary

The Glaber ValueCache provides intelligent, high-performance caching for monitoring data with minimal configuration. Its demand-based approach ensures optimal memory usage while maintaining fast query performance. For most use cases, default settings work well, with automatic adaptation to your specific usage patterns over time.