API Docs

Index syncing module.

class invenio_index_migrator.api.Job(name, migration, config)

Index migration job.

Initialize a migration job.

Parameters:

• name – job’s name.
• migration – an invenio_index_migrator.api.migration.Migration object.
• config – job’s configuration.

cancel()

Cancel the job.

create_index(index)

Create indexes needed for the job.

document_name

Get the document name for the job.

rollover_actions()

Rollover actions.

run()

Run the job.

src_es_client

Get the source ES client.

status()

Return the status of the job.

class invenio_index_migrator.api.Migration(name, **config)

Index migration base class.

cancel()

Cancel migration and all its jobs.

classmethod create_from_config(recipe_name, **recipe_config)

Create Migration instance from config.

classmethod create_from_state(recipe_name, **recipe_config)

Create Migration instance from ES state.

create_index()

Create Elasticsearch index for the migration.

init(dry_run=False)

Initialize the index with recipe and jobs documents.

load_jobs_from_config()

Load jobs from config.

notify()

Notify when rollover is possible.

Override this to notify the user whenever the threshold is reached and a rollover is possible.

rollover(force=False)

Perform a rollover action.

run()

Run the index sync job.

status()

Get status for index sync job.

strategy

Return migration strategy.

class invenio_index_migrator.api.MultiIndicesReindexJob(name, migration, config)

Reindex job that uses Elasticsearch’s reindex API.

Initialize a migration job.

Parameters:

• name – job’s name.
• migration – an invenio_index_migrator.api.migration.Migration object.
• config – job’s configuration.

create_index(index)

Create templates.

initial_state(dry_run=False)

Build job’s initial state.

rollover_actions()

Rollover actions.

run()

Fetch source index using ES Reindex API.

class invenio_index_migrator.api.ReindexJob(name, migration, config)

Reindex job that uses Elasticsearch’s reindex API.

Initialize a migration job.

Parameters:

• name – job’s name.
• migration – an invenio_index_migrator.api.migration.Migration object.
• config – job’s configuration.

cancel()

Cancel reindexing job.

initial_state(dry_run=False)

Build job’s initial state.

run()

Fetch source index using ES Reindex API.

class invenio_index_migrator.api.ReindexAndSyncJob(name, migration, config)

Job that both reindexes with ES reindex API and syncs with the DB.

The first run will use the reindex API and the subsequent runs will fetch from the database and sync the data.

Initialize a migration job.

Parameters:

• name – job’s name.
• migration – an invenio_index_migrator.api.migration.Migration object.
• config – job’s configuration.

cancel()

Cancel reinding and syncing job.

iter_indexer_ops(start_date=None, end_date=None)

Iterate over documents that need to be reindexed.

run()

Run reindexing and syncing job.

run_delta_job()

Calculate delta from DB changes since the last update.

Utilities

Utility functions for index migration.

class invenio_index_migrator.utils.ESClient(es_config)

ES clinet for sync jobs.

.

client

Return ES client.

reindex_auth

Return username and password for reindex HTTP authentication.

reindex_remote

Return ES client reindex API host.

class invenio_index_migrator.utils.State(index, document_id, client=None)

Migration ES state.

The state is stored in ElasticSearch and can be accessed similarly to a python dictionary.

Synchronization job state in ElasticSearch.

commit(state)

Save the state to ElasticSearch.

create(initial_state, force=False)

Create state document.

read()

Fetch the current state from Elasticsearch.

invenio_index_migrator.utils.extract_doctype_from_mapping(mapping_fp)

Extract the doc_type from mapping filepath.

invenio_index_migrator.utils.get_queue_size(queue)

Get the queue size.

invenio_index_migrator.utils.obj_or_import_string(value, default=None)

Import string or return object.

Params value:Import path or class object to instantiate. Params default:Default object to return if the import fails. Returns:The imported object.