inspirehep.utils package

Submodules

inspirehep.utils.citations module

inspirehep.utils.citations.get_and_format_citations(record)

Deprecated since version 2018-08-23.

inspirehep.utils.conferences module

inspirehep.utils.conferences.conferences_contributions_from_es(cnum)

Query ES for conferences in the same series.

inspirehep.utils.conferences.conferences_in_the_same_series_from_es(seriesname)

Query ES for conferences in the same series.

inspirehep.utils.conferences.render_conferences(recid, conferences)

Render a list of conferences to HTML.

inspirehep.utils.conferences.render_conferences_contributions(cnum)

Conference export for single record in datatables format. :returns: list List of lists where every item represents a datatables row. A row consists of [conference_name, conference_location, contributions, date]

inspirehep.utils.conferences.render_conferences_in_the_same_series(recid, seriesname)

Conference export for single record in datatables format. :returns: list List of lists where every item represents a datatables row. A row consists of [conference_name, conference_location, contributions, date]

inspirehep.utils.conferences.render_contributions(hits)

Render a list of conferences to HTML.

inspirehep.utils.experiments module

inspirehep.utils.experiments.experiment_contributions_from_es(experiment_name)

Query ES for conferences in the same series.

inspirehep.utils.experiments.experiment_people_from_es(experiment_name)

Query ES for conferences in the same series.

inspirehep.utils.experiments.render_contributions(hits)

Render a list of conferences to HTML.

inspirehep.utils.experiments.render_experiment_contributions(experiment_name)

Conference export for single record in datatables format. :returns: list List of lists where every item represents a datatables row. A row consists of [conference_name, conference_location, contributions, date]

inspirehep.utils.experiments.render_experiment_people(experiment_name)

Conference export for single record in datatables format. :returns: list List of lists where every item represents a datatables row. A row consists of [conference_name, conference_location, contributions, date]

inspirehep.utils.experiments.render_people(hits)

Render a list of conferences to HTML.

inspirehep.utils.export module

class inspirehep.utils.export.Export(record, *args, **kwargs)

Bases: object

Base class used for export formats.

arxiv_field

Return arXiv field if exists

exception inspirehep.utils.export.MissingRequiredFieldError(field)

Bases: exceptions.LookupError

Base class for exceptions in this module. The exception should be raised when the specific, required field doesn’t exist in the record.

inspirehep.utils.ext module

Utils extension.

class inspirehep.utils.ext.INSPIREUtils(app=None)

Bases: object

Utils extension.

configure_appmetrics(app)

create_rt_instance(app)

Make a RT instance and return it.

init_app(app)

Initialize the application.

inspirehep.utils.ext.exception_hook(response, exception, metric, func_args, func_kwargs)

@time_execution hook to collect info about the raised exception.

inspirehep.utils.jinja2 module

inspirehep.utils.jinja2.render_template_to_string(input, _from_string=False, **context)

Render a template from the template folder with the given context. Code based on https://github.com/mitsuhiko/flask/blob/master/flask/templating.py :param input: the string template, or name of the template to be rendered, or an iterable with template names the first one existing will be rendered :param context: the variables that should be available in the context of the template. :return: a string

inspirehep.utils.latex module

inspirehep.utils.latex.decode_latex(latex_text)

Decode latex text.

Parameters:latex_text (str) – a latex text. Returns:the latex text decoded. Return type:str

inspirehep.utils.lock module

Locking.

exception inspirehep.utils.lock.DistributedLockError

Bases: exceptions.Exception

inspirehep.utils.lock.distributed_lock(*args, **kwds)

Context manager to acquire a lock visible by all processes.

This lock is implemented through Redis in order to be globally visible.

Parameters:

• lock_name (str) – name of the lock to be acquired.
• expire (int) – duration in seconds after which the lock is released if not renewed in the meantime.
• auto_renewal (bool) – if True, the lock is automatically renewed as long as the context manager is still active.
• blocking (bool) – if True, wait for the lock to be released. If False, return immediately, raising DistributedLockError.

It is recommended to set expire to a small value and auto_renewal=True, which ensures the lock gets released quickly in case the process is killed without limiting the time that can be spent holding the lock.

Raises:DistributedLockError – when blocking is set to False and the lock is already acquired.

inspirehep.utils.normalizers module

inspirehep.utils.normalizers.normalize_journal_title(journal_title)

inspirehep.utils.proxies module

Utils proxies.

inspirehep.utils.proxies.rt_instance = <LocalProxy unbound>

Helper proxy to access the state object.

inspirehep.utils.record module

inspirehep.utils.record.create_index_op(record, version_type='external_gte')

inspirehep.utils.record.get_abstract(record)

Return the first abstract of a record.

Parameters:record (InspireRecord) – a record. Returns:the first abstract of the record. Return type:str

Examples

>>> record = {
...     'abstracts': [
...         {
...             'source': 'arXiv',
...             'value': 'Probably not.',
...         },
...     ],
... }
>>> get_abstract(record)
'Probably not.'

inspirehep.utils.record.get_arxiv_categories(record)

Return all the arXiv categories of a record.

Parameters:record (InspireRecord) – a record. Returns:all the arXiv categories of the record. Return type:list(str)

Examples

>>> record = {
...     'arxiv_eprints': [
...         {
...             'categories': [
...                 'hep-th',
...                 'hep-ph',
...             ],
...             'value': '1612.08928',
...         },
...     ],
... }
>>> get_arxiv_categories(record)
['hep-th', 'hep-ph']

inspirehep.utils.record.get_arxiv_id(record)

Return the first arXiv identifier of a record.

Parameters:record (InspireRecord) – a record. Returns:the first arXiv identifier of the record. Return type:str

Examples

>>> record = {
...     'arxiv_eprints': [
...         {
...             'categories': [
...                 'hep-th',
...                 'hep-ph',
...             ],
...             'value': '1612.08928',
...         },
...     ],
... }
>>> get_arxiv_id(record)
'1612.08928'

inspirehep.utils.record.get_collaborations(record)

Return the collaborations associated with a record.

Parameters:record (InspireRecord) – a record. Returns:the collaborations associated with the record. Return type:list(str)

Examples

>>> record = {'collaborations': [{'value': 'CMS'}]}
>>> get_collaborations(record)
['CMS']

inspirehep.utils.record.get_inspire_categories(record)

Return all the INSPIRE categories of a record.

Parameters:record (InspireRecord) – a record. Returns:all the INSPIRE categories of the record. Return type:list(str)

Examples

>>> record = {
...     'inspire_categories': [
...         {'term': 'Experiment-HEP'},
...     ],
... }
>>> get_inspire_categories(record)
['Experiment-HEP']

inspirehep.utils.record.get_keywords(record)

Return the keywords assigned to a record.

Parameters:record (InspireRecord) – a record. Returns:the keywords assigned to the record. Return type:list(str)

Examples

>>> record = {
...     'keywords': [
...         {
...             'schema': 'INSPIRE',
...             'value': 'CKM matrix',
...         },
...     ],
... }
>>> get_keywords(record)
['CKM matrix']

inspirehep.utils.record.get_method(record)

Return the acquisition method of a record.

Parameters:record (InspireRecord) – a record. Returns:the acquisition method of the record. Return type:str

Examples

>>> record = {
...     'acquisition_source': {
...         'method': 'oai',
...         'source': 'arxiv',
...     }
... }
>>> get_method(record)
'oai'

inspirehep.utils.record.get_source(record)

Return the acquisition source of a record.

Parameters:record (InspireRecord) – a record. Returns:the acquisition source of the record. Return type:str

Examples

>>> record = {
...     'acquisition_source': {
...         'method': 'oai',
...         'source': 'arxiv',
...     }
... }
>>> get_source(record)
'arxiv'

inspirehep.utils.record.get_subtitle(record)

Return the first subtitle of a record.

Parameters:record (InspireRecord) – a record. Returns:the first subtitle of the record. Return type:str

Examples

>>> record = {
...     'titles': [
...         {
...             'subtitle': 'A mathematical exposition',
...             'title': 'The General Theory of Relativity',
...         },
...     ],
... }
>>> get_subtitle(record)
'A mathematical exposition'

inspirehep.utils.record.get_title(record)

Return the first title of a record.

Parameters:record (InspireRecord) – a record. Returns:the first title of the record. Return type:str

Examples

>>> record = {
...     'titles': [
...         {
...             'subtitle': 'A mathematical exposition',
...             'title': 'The General Theory of Relativity',
...         },
...     ],
... }
>>> get_title(record)
'The General Theory of Relativity'

inspirehep.utils.record_getter module

Resource-aware json reference loaders to be used with jsonref.

exception inspirehep.utils.record_getter.RecordGetterError(message, cause)

Bases: exceptions.Exception

inspirehep.utils.record_getter.get_db_record(*args, **kwargs)

inspirehep.utils.record_getter.get_db_records(pids)

Get an iterator on record metadata from the DB.

Parameters:pids (Iterable[Tuple[str, Union[str, int]]) – a list of (pid_type, pid_value) tuples. Yields:dict – metadata of a record found in the database.

Warning

The order in which records are returned is different from the order of the input.

inspirehep.utils.record_getter.get_es_record(*args, **kwargs)

inspirehep.utils.record_getter.get_es_record_by_uuid(*args, **kwargs)

inspirehep.utils.record_getter.get_es_records(pid_type, recids, **kwargs)

Get a list of recids from ElasticSearch.

inspirehep.utils.record_getter.raise_record_getter_error_and_log(f)

inspirehep.utils.references module

inspirehep.utils.references.get_and_format_references(record)

Format references.

Deprecated since version 2018-06-07.

inspirehep.utils.references.local_refextract_kbs_path(*args, **kwds)

Get the path to the temporary refextract kbs from the application config.

inspirehep.utils.references.map_refextract_to_schema(extracted_references, source=None)

Convert refextract output to the schema using the builder.

inspirehep.utils.robotupload module

Utils for sending robotuploads to other Invenio instances.

inspirehep.utils.robotupload.make_robotupload_marcxml(url, marcxml, mode, **kwargs)

Make a robotupload request.

inspirehep.utils.schema module

inspirehep.utils.schema.ensure_valid_schema(record)

Make sure the $schema key of the record is valid.

This is done by setting the correct url to the schema, in case it only contains the schema filename.

inspirehep.utils.stats module

inspirehep.utils.stats.calculate_h_index(citations)

Calculate the h-index of a citation dictionary.

An author has h-index X if she has X papers with at least X citations each. See: https://en.wikipedia.org/wiki/H-index.

Parameters:citations – a dictionary in the format {recid: citation_count} Returns:h-index of the dictionary of citations.

inspirehep.utils.stats.calculate_i10_index(citations)

Calculate the i10-index of a citation dictionary.

An author has i10-index X if she has X papers with at least 10 citations each. See: https://en.wikipedia.org/wiki/H-index#i10-index

Parameters:citations – a dictionary in the format {recid: citation_count} Returns:i10-index of the dictionary of citations.

inspirehep.utils.template module

Utils related to Jinja templates.

inspirehep.utils.template.render_macro_from_template(name, template, app=None, ctx=None)

Render macro with the given context.

Parameters:

• name (string.) – macro name.
• template (string.) – template name.
• app (object.) – Flask app.
• ctx (dict.) – parameters of the macro.

Returns:

unicode string with rendered macro.

inspirehep.utils.tickets module

Functions related to the main INSPIRE-HEP ticketing system.

exception inspirehep.utils.tickets.EditTicketException

Bases: exceptions.Exception

class inspirehep.utils.tickets.InspireRt(url, default_login=None, default_password=None, proxy=None, default_queue='General', basic_auth=None, digest_auth=None, skip_login=False, verify_cert=True)

Bases: rt.Rt

get_attachments(ticket_id)

Get attachment list for a given ticket.

Copy-pased from rt library, only change is starting form 3rd line of response for attachments to look for attachments.

Parameters:ticket_id – ID of ticket Returns:List of tuples for attachments belonging to given ticket. Tuple format: (id, name, content_type, size) Returns None if ticket does not exist.

inspirehep.utils.tickets.create_ticket(*args, **kwargs)

Creates new RT ticket and returns new ticket id.

Parameters:

• queue (string) – where the ticket will be created
• requestors (string) – username to set to requestors field of the ticket
• body (string) – message body of the ticket
• subject (string) – subject of the ticket
• recid (integer) – record id to be set custom RecordID field
• kwargs –

  Other arguments possible to set:

  Cc, AdminCc, Owner, Status,Priority, InitialPriority, FinalPriority, TimeEstimated, Starts, Due, ... (according to RT fields)

  Custom fields CF.{<CustomFieldName>} could be set with keywords CF_CustomFieldName.

Returns:

ID of the new ticket or -1, if it fails

Return type:

integer

inspirehep.utils.tickets.create_ticket_with_template(queue, requestors, template_path, template_context, subject, recid=None, **kwargs)

Creates new RT ticket with a body that is rendered template

Parameters:

• queue (string) – where the ticket will be created
• requestors (string) – username to set to requestors field of the ticket
• template_path (string) – path to the template for the ticket body
• template_context (dict) – context object to be used to render template
• subject (string) – subject of the ticket
• recid (integer) – record id to be set custom RecordID field
• kwargs –

  Other arguments possible to set:

  Cc, AdminCc, Owner, Status,Priority, InitialPriority, FinalPriority, TimeEstimated, Starts, Due, ... (according to RT fields)

  Custom fields CF.{<CustomFieldName>} could be set with keywords CF_CustomFieldName.

Returns:

ID of the new ticket or -1, if it fails

Return type:

integer

inspirehep.utils.tickets.get_queues()

Returns list of all queues as {id, name} dict

Return type:dict - with name (string), id (integer) properties

inspirehep.utils.tickets.get_rt_link_for_ticket(ticket_id)

Returns rt system display link to given ticket

Return type:string

inspirehep.utils.tickets.get_tickets_by_recid(*args, **kwargs)

Returns all tickets that are associated with the given recid

inspirehep.utils.tickets.get_users()

Returns list of all users as {id, name} dict

Return type:dict - with name (string), id (integer) properties

inspirehep.utils.tickets.relogin_if_needed(f)

Repeat RT call after explicit login, if needed.

In case a call to RT fails, due session expired, this decorator will explicitly call .login() on RT, in order to refresh the session, and will replay the call.

This decorator should be used to wrap any function calling into RT.

FIXME: The real solution would be to enable auth/digest authentication on RT side. Then this trick would no longer be needed, as long as the extension is properly initialized in ext.py.

inspirehep.utils.tickets.reply_ticket(*args, **kwargs)

Replies the given ticket with the message body

Parameters:

• body (string) – message body of the reply
• keep_new – flag to keep ticket Status, 'new'

inspirehep.utils.tickets.reply_ticket_with_template(ticket_id, template_path, template_context, keep_new=False)

Replies the given ticket with a body that is rendered template

Parameters:

• template_path (string) – path to the template for the ticket body
• template_context (dict) – context object to be used to render template
• keep_new – flag to keep ticket Status, 'new'

inspirehep.utils.tickets.resolve_ticket(*args, **kwargs)

Resolves the given ticket

inspirehep.utils.url module

Helpers for handling with http requests and URL handling.

inspirehep.utils.url.copy_file(src_file, dst_file, buffer_size=8192)

Dummy buffered copy between open files.

inspirehep.utils.url.get_legacy_url_for_recid(recid)

Get a URL to a record on INSPIRE.

Parameters:

• recid (Union[int, string]) – record ID
• pattern_config_var (string) – config var with the pattern

Returns:

URL

Return type:

text_type

inspirehep.utils.url.is_pdf_link(url)

Return True if url points to a PDF.

Returns True if the first non-whitespace characters of the response are %PDF.

Parameters:url (string) – a URL. Returns:whether the url points to a PDF. Return type:bool

inspirehep.utils.url.make_user_agent_string(component='')

Return a nice and uniform user-agent string to be used by INSPIRE.

inspirehep.utils.url.retrieve_uri(*args, **kwds)

Retrieves the given uri and stores it in a temporary file.

Module contents