🔄 Incremental Sync

Incremental synchronization allows you to efficiently update your local data by retrieving only records that have changed since your last sync. This approach minimizes API calls, reduces bandwidth usage, and improves performance.

🎯 Why Use Incremental Sync?

Benefits

When to Use

📊 Supported Endpoints

Endpoint Modified Date Parameter Typical Update Frequency Notes
GET /products fromLastModifiedDateUTC Low (hours/days) Product catalog changes
GET /customer/customers fromLastModifiedDateUTC Medium (minutes/hours) Customer profile updates
GET /reporting/inventory None (current snapshot only) Medium (minutes/hours) Current inventory levels
GET /reporting/register-transactions fromLastModifiedDateUTC/toLastModifiedDateUTC High (real-time) Transaction history by date range

🕒 Date-Based Filtering

Most endpoints support date-based filtering using ISO 8601 formatted timestamps in UTC.

Common Parameters

Example Request:

GET /products?fromLastModifiedDateUTC=2024-01-15T10:30:00Z

Date Format:

💡 Timezone Note: All timestamps in the API are in UTC. Convert your local timestamps to UTC before making requests.

🔧 Implementation Strategy

1Initial Full Sync

Start with a complete data sync to establish your baseline:

// Initial sync - get all products
GET /products

// Store the current timestamp for next sync
const lastSyncTime = new Date().toISOString();

2Store Last Sync Timestamp

Track when your last successful sync completed:

3Incremental Updates

Use the stored timestamp for subsequent syncs:

// Incremental sync
GET /products?fromLastModifiedDateUTC=2024-01-15T10:30:00Z

// Process updates, then update timestamp
const newSyncTime = new Date().toISOString();

4Handle Edge Cases

📝 Implementation Examples

JavaScript Implementation

class IncrementalSync {
  constructor(apiClient, storage) {
    this.api = apiClient;
    this.storage = storage;
  }
  
  async syncProducts() {
    const lastSync = await this.storage.getLastSync('products');
    const syncStartTime = new Date().toISOString();
    
    try {
      // Build request with fromLastModifiedDateUTC if we have a last sync time
      const params = lastSync 
        ? { fromLastModifiedDateUTC: lastSync }
        : {};
      
      const response = await this.api.get('/products', { params });
      
      // Process the updates
      await this.processProductUpdates(response.data);
      
      // Update last sync time only on success
      await this.storage.setLastSync('products', syncStartTime);
      
      return {
        success: true,
        updatedCount: response.data.length,
        isIncremental: !!lastSync
      };
    } catch (error) {
      // Don't update lastSync timestamp on failure
      throw error;
    }
  }
  
  async processProductUpdates(products) {
    for (const product of products) {
      await this.storage.upsertProduct(product);
    }
  }
}

Sync Scheduler

// Set up different sync frequencies for different data types
const syncSchedule = {
  products: 3600000,        // 1 hour (low change frequency)
  customers: 300000,        // 5 minutes (medium change frequency)
  inventory: 60000,         // 1 minute (high change frequency)
  transactions: 30000       // 30 seconds (real-time needs)
};

Object.entries(syncSchedule).forEach(([endpoint, interval]) => {
  setInterval(async () => {
    try {
      await syncManager.sync(endpoint);
      console.log(`${endpoint} sync completed`);
    } catch (error) {
      console.error(`${endpoint} sync failed:`, error);
    }
  }, interval);
});

⚠️ Important Considerations

Clock Synchronization

Server and client clocks may not be perfectly synchronized:

⚠️ Clock Skew Example: If your clock is 30 seconds ahead of the server, you might miss updates that occur in that 30-second window.

Handling Deletions

The API typically doesn't expose deleted records in incremental syncs:

Error Recovery

🎯 Best Practices

Sync Frequency Guidelines

Performance Optimization

Data Integrity

🎯 Pro Tip: Start with longer sync intervals and gradually decrease them based on your actual business needs and rate limit capacity.

🔍 Monitoring and Troubleshooting

Key Metrics to Track

Common Issues