Client

The client module provides a client interface for interacting with OAI-PMH services.

This module defines the Scythe class, which facilitates the harvesting of records, identifiers, and sets from OAI-PMH compliant repositories. It handles various OAI-PMH requests, manages pagination with resumption tokens, and supports customizable error handling and retry logic.

`Scythe`

A client for interacting with OAI-PMH interfaces, facilitating the harvesting of records, identifiers, and sets.

The Scythe class is designed to simplify the process of making OAI-PMH requests and processing the responses. It supports various OAI-PMH verbs and handles pagination through resumption tokens, error handling, and retry logic.

Attributes:

Name	Type	Description
`endpoint`		The base URL of the OAI-PMH service.
`http_config`		A HTTPConfig instance controlling how HTTP requests are made.
`retry_config`		A RetryConfig instance controlling the retry behavior for failed requests.
`http_method`		(Deprecated, will be removed in version 0.19.0) Use `http_config.http_method` instead.
`iterator`		The iterator class to be used for iterating over responses.
`max_retries`		(Deprecated, will be removed in version 0.19.0) Use `retry_config.max_retries` instead.
`retry_status_codes`		(Deprecated, will be removed in version 0.19.0) Use `retry_config.retry_status_codes` instead.
`default_retry_after`		(Deprecated, will be removed in version 0.19.0) Use `retry_config.default_retry_after` instead.
`class_mapping`		A mapping from OAI verbs to classes representing OAI items.
`encoding`		(Deprecated, will be removed in version 0.19.0) Use `http_config.encoding` instead.
`auth`		(Deprecated, will be removed in version 0.19.0) Use `http_config.auth` instead.
`timeout`		(Deprecated, will be removed in version 0.19.0) Use `http_config.timeout` instead.

Examples:

>>> with Scythe("https://zenodo.org/oai2d") as scythe:
>>>     records = scythe.list_records()
>>>     for record in records:
>>>         print(record)

`client` `property`

Provide a reusable HTTP client instance for making requests.

This property ensures that an httpx2.Client instance is created and maintained for the lifecycle of the Scythe instance. It handles the creation of the client and ensures that a new client is created if the existing one is closed.

Returns:

Type	Description
`Client`	A reusable HTTP client instance for making HTTP requests.

`close()`

Close the internal HTTP client if it exists and is open.

This method is responsible for explicitly closing the httpx2.Client instance used by the Scythe class. It should be called when the client is no longer needed, to ensure proper cleanup and release of resources.

Note

It's recommended to call this method at the end of operations or when the Scythe instance is no longer in use, especially if it's not being used as a context manager.

`get_record(identifier, metadata_prefix='oai_dc')`

Issue a GetRecord request to the OAI server.

Send a request to the OAI server to retrieve a specific record. The request is constructed with the provided identifier and metadata prefix. The method then processes and returns the relevant OAIResponse or Record object using an iterator.

Ref: https://openarchives.org/OAI/openarchivesprotocol.html#GetRecord

Parameters:

Name	Type	Description	Default
`identifier`	`str`	A unique identifier for the record to be retrieved from the OAI server.	required
`metadata_prefix`	`str`	The metadata format to be returned for the record. Defaults to "oai_dc".	`'oai_dc'`

Returns:

Type	Description
`OAIResponse \| Record`	An OAIResponse or Record object representing the requested record.

Raises:

Type	Description
`CannotDisseminateFormat`	If the specified metadata_prefix is not supported by the OAI server for the requested record.
`IdDoesNotExist`	If the specified identifier does not correspond to any record in the OAI server.

`get_retry_after(http_response)` `deprecated`

Deprecated

Scythe.get_retry_after() is not part of the public API. There is no public replacement. To customize retries, use the retry_config argument instead. This method will be removed in version 0.18.0.

See _get_retry_after.

`harvest(query)`

Perform an HTTP request to the OAI server with the given parameters.

Send an OAI-PMH request to the server using the specified parameters. Handle retry logic for failed requests based on the configured retry settings and response status codes.

Parameters:

Name	Type	Description	Default
`query`	`dict[str, str]`	A dictionary containing the request parameters.	required

Returns:

Type	Description
`OAIResponse`	An OAIResponse object encapsulating the server's response.

Raises:

Type	Description
`OAIPMHException`	If the response contains an OAI-PMH element, regardless of the HTTP status code. The exception type corresponds to the error code (e.g. IdDoesNotExist).
`HTTPError`	If the HTTP request fails after the maximum number of retries.

`identify()`

Issue an Identify request to the OAI server.

Send a request to identify the OAI server and retrieve its information. This includes details such as the repository name, the base URL, the protocol version, and other relevant data about the OAI server. It's useful for understanding the capabilities and configuration of the server.

Ref: https://openarchives.org/OAI/openarchivesprotocol.html#Identify

Returns:

Type	Description
`Identify`	An object encapsulating the server's identify response, which contains various pieces of information about the OAI server.

`list_identifiers(from_=None, until=None, metadata_prefix='oai_dc', set_=None, resumption_token=None, ignore_deleted=False)`

Issue a ListIdentifiers request to the OAI server.

Send a request to list record identifiers from the OAI server. This method allows filtering records based on date range, set membership, and metadata format. It also supports pagination through resumption tokens and has an option to ignore deleted records.

Ref: https://openarchives.org/OAI/openarchivesprotocol.html#ListIdentifiers

Parameters:

Name	Type	Description	Default
`from_`	`str \| None`	An optional date string specifying the start of a date range for harvesting records.	`None`
`until`	`str \| None`	An optional date string specifying the end of a date range for harvesting records.	`None`
`metadata_prefix`	`str`	The metadata format for the records to be harvested. Defaults to "oai_dc".	`'oai_dc'`
`set_`	`str \| None`	An optional set identifier to restrict the harvest to records within a specific set.	`None`
`resumption_token`	`str \| None`	An optional token for pagination, used to continue a request for the next page of identifiers.	`None`
`ignore_deleted`	`bool`	If True, skip records flagged as deleted in the response.	`False`

Yields:

Type	Description
`OAIResponse \| Header`	An iterator over OAIResponse or Header objects, each representing an individual record identifier or response from the server.

Raises:

Type	Description
`BadResumptionToken`	If the provided resumption token is invalid or expired.
`CannotDisseminateFormat`	If the specified metadata_prefix is not supported by the OAI server.
`NoRecordsMatch`	If no records match the provided criteria.
`NoSetHierarchy`	If set-based harvesting is requested but the OAI server does not support sets.

`list_metadata_formats(identifier=None)`

Issue a ListMetadataFormats request to the OAI server.

Send a request to list the metadata formats available from the OAI server. This can be done for the entire repository or for a specific record if an identifier is provided. The method constructs a query and yields an iterator over OAIResponse or MetadataFormat objects, each representing a different metadata format or response from the server.

Ref: https://openarchives.org/OAI/openarchivesprotocol.html#ListMetadataFormats

Parameters:

Name	Type	Description	Default
`identifier`	`str \| None`	An optional unique identifier for a specific record to query available metadata formats. If None, all metadata formats available in the repository are listed.	`None`

Yields:

Type	Description
`OAIResponse \| MetadataFormat`	An iterator over OAIResponse or MetadataFormat objects, each representing an individual metadata format or response from the server.

Raises:

Type	Description
`IdDoesNotExist`	If the specified identifier does not correspond to any record in the OAI server.
`NoMetadataFormats`	If there are no metadata formats available for the requested record or repository.

`list_records(from_=None, until=None, metadata_prefix='oai_dc', set_=None, resumption_token=None, ignore_deleted=False)`

Issue a ListRecords request to the OAI server.

Send a request to list records from the OAI server, allowing for selective harvesting based on date range, set membership, and metadata format. This method supports pagination via resumption tokens and can optionally ignore records marked as deleted.

Ref: https://openarchives.org/OAI/openarchivesprotocol.html#ListRecords

Parameters:

Name	Type	Description	Default
`from_`	`str \| None`	An optional date string specifying the start of a date range for harvesting records.	`None`
`until`	`str \| None`	An optional date string specifying the end of a date range for harvesting records.	`None`
`metadata_prefix`	`str`	The metadata format for the records to be harvested. Defaults to "oai_dc".	`'oai_dc'`
`set_`	`str \| None`	An optional set identifier to restrict the harvest to records within a specific set.	`None`
`resumption_token`	`str \| None`	An optional token for pagination, used to continue a request for the next page of records.	`None`
`ignore_deleted`	`bool`	If True, skip records flagged as deleted in the response.	`False`

Yields:

Type	Description
`OAIResponse \| Record`	An iterator over OAIResponse or Record objects, each representing an individual record or response from the server.

Raises:

Type	Description
`BadArgument`	If the arguments provided do not conform to the expectations of the OAI server.
`BadResumptionToken`	If the provided resumption token is invalid or expired.
`CannotDisseminateFormat`	If the specified metadata_prefix is not supported by the OAI server.
`NoRecordsMatch`	If no records match the provided criteria.
`NoSetHierarchy`	If set-based harvesting is requested but the OAI server does not support sets.

`list_sets(resumption_token=None)`

Issue a ListSets request to the OAI server.

Send a request to list all sets defined in the OAI server. Sets are used to categorize records in the OAI repository. This method allows for the retrieval of these sets, optionally using a resumption token to handle pagination.

Ref: https://openarchives.org/OAI/openarchivesprotocol.html#ListSets

Parameters:

Name	Type	Description	Default
`resumption_token`	`str \| None`	An optional token for pagination, used to continue a request for the next batch of sets.	`None`

Yields:

Type	Description
`OAIResponse \| Set`	An iterator over OAIResponse or Set objects, representing an individual set or response from the server.

Raises:

Type	Description
`BadResumptionToken`	If the provided resumption token is invalid or expired.
`NoSetHierarchy`	If the OAI server does not support sets or has no set hierarchy available.

Client

Scythe

client property

close()

get_record(identifier, metadata_prefix='oai_dc')

get_retry_after(http_response) deprecated

harvest(query)

identify()

list_identifiers(from_=None, until=None, metadata_prefix='oai_dc', set_=None, resumption_token=None, ignore_deleted=False)

list_metadata_formats(identifier=None)

list_records(from_=None, until=None, metadata_prefix='oai_dc', set_=None, resumption_token=None, ignore_deleted=False)

list_sets(resumption_token=None)

`Scythe`

`client` `property`

`close()`

`get_record(identifier, metadata_prefix='oai_dc')`

`get_retry_after(http_response)` `deprecated`

`harvest(query)`

`identify()`

`list_identifiers(from_=None, until=None, metadata_prefix='oai_dc', set_=None, resumption_token=None, ignore_deleted=False)`

`list_metadata_formats(identifier=None)`

`list_records(from_=None, until=None, metadata_prefix='oai_dc', set_=None, resumption_token=None, ignore_deleted=False)`

`list_sets(resumption_token=None)`