life-echo/api/.agents/skills/redis-development/rules/rqe-skip-initial-scan.md

title, impact, impactDescription, tags, description, alwaysApply

title	impact	impactDescription	tags	description	alwaysApply
Use SKIPINITIALSCAN for New Data Only Indexes	MEDIUM	Faster index creation, avoids indexing existing data	index, skipinitialscan, rqe, search	Use SKIPINITIALSCAN for New Data Only Indexes	true

Use SKIPINITIALSCAN for New Data Only Indexes

Enable the SKIPINITIALSCAN option when creating an index if you only want to include items that are added after the index is created. This makes index creation faster and avoids indexing existing data that you don't need to search.

Correct: Use SKIPINITIALSCAN when you only need to index new data.

Python (redis-py):

import redis
from redis.commands.search.field import TextField, TagField
from redis.commands.search.indexDefinition import IndexDefinition, IndexType

client = redis.Redis(host='localhost', port=6379)

# Create index that only indexes new documents
schema = (
    TextField("name"),
    TagField("categories")
)

definition = IndexDefinition(
    prefix=["person:"],
    index_type=IndexType.HASH
)

# SKIPINITIALSCAN - only index documents added after creation
client.ft("idx").create_index(
    schema,
    definition=definition,
    skip_initial_scan=True
)

Java (Jedis):

import redis.clients.jedis.UnifiedJedis;
import redis.clients.jedis.search.FTCreateParams;
import redis.clients.jedis.search.IndexDataType;
import redis.clients.jedis.search.schemafields.SchemaField;
import redis.clients.jedis.search.schemafields.TagField;
import redis.clients.jedis.search.schemafields.TextField;

try (UnifiedJedis jedis = new UnifiedJedis("redis://localhost:6379")) {
    FTCreateParams params = new FTCreateParams()
        .on(IndexDataType.HASH)
        .skipInitialScan();  // Only index new documents

    jedis.ftCreate(
        "idx",
        params,
        new SchemaField[]{
            new TextField("name"),
            new TagField("categories")
        }
    );
}

When to use SKIPINITIALSCAN:

• Creating an index for a new feature where existing data is irrelevant
• Setting up indexes in advance before data arrives
• When existing data would be too large to scan during index creation
• Event-driven architectures where you only care about new events

When NOT to use (default behavior is correct):

• You need to search existing data immediately after index creation
• Migrating to a new index schema and need all data indexed
• Most typical use cases where historical data matters

Note: The default behavior (without SKIPINITIALSCAN) indexes all existing matching keys, which is usually what you want.

Reference: FT.CREATE SKIPINITIALSCAN