RAG Index Maintenance
Index maintenance, incremental sync, knowledge freshness management, and monitoring for the RAG system.
Knowledge Freshness
Update Cadence
| Content Type | Update Trigger | Max Staleness |
|---|---|---|
| Menu items | On change | 5 minutes |
| Pricing | On change | Immediate |
| FAQs | Weekly review | 7 days |
| Runbooks | On change | 1 day |
| Analytics | Daily refresh | 24 hours |
Incremental Sync
async function incrementalSync(indexName: string, since: Date) {
// Get changed documents
const changes = await getChangedDocs(indexName, since);
for (const doc of changes) {
if (doc.deleted) {
await index.deleteByIds([doc.id]);
} else {
const chunks = await processDocument(doc);
const embeddings = await embedChunks(chunks);
await index.upsert(embeddings);
}
}
// Update sync timestamp
await setSyncTimestamp(indexName, new Date());
}
Monitoring & Observability
Key Metrics
| Metric | Target | Alert Threshold |
|---|---|---|
| Query latency p50 | Under 50ms | Over 100ms |
| Query latency p99 | Under 200ms | Over 500ms |
| Result quality score | Over 0.75 | Under 0.6 |
| Index freshness | Under 1 hour | Over 24 hours |
| Vector count | Under 5M | Over 4.5M |
Logging
// Log every RAG query for analysis
logger.info('rag_query', {
query: sanitize(query),
agentType,
topK,
resultCount: results.length,
topScore: results[0]?.score,
latencyMs: Date.now() - startTime,
tenantId
});