Common Tasks¶
Caching¶
For caching we use Invenio-Cache. For example, to set a value in the cache:
>>> from invenio_cache import current_cache
>>> current_cache.set('test', [1, 2, 3], timeout=60)
And to retrieve the value from the cache:
>>> from invenio_cache import current_cache
>>> current_cache.get('test')
Profiling a Celery Task¶
To profile a Celery task we need to make sure that the task is executed by the same Python process in which we are collecting the profiling information. That is, the configuration must contain
CELERY_TASK_ALWAYS_EAGER = True
CELERY_RESULT_BACKEND = 'cache'
CELERY_CACHE_BACKEND = 'memory'
Then, in a Flask shell, we do
>>> import cProfile
>>> import pstats
>>> from path.to.our.task import task
>>> pr = cProfile.Profile()
>>> pr.runcall(task, *args, **kwargs)
where *args
and *kwargs
are the arguments and keyword arguments that
we want to pass to task
. Then
>>> ps = pstats.Stats(pr)
>>> ps.dump_stats('task.prof')
will create a binary file containing the desired profiling information. To read it we can use snakeviz, which will create a graph such as
Essentially each layer of the graph is a level of the call stack, and the size of the slice is the total time of the function call. For a complete explanation visit the documentation of snakeviz.
Profiling a Request¶
To profile a request we need to add the following variable to our configuration:
PROFILE = True
Then we need to attach the WSGI application profiler to our WSGI application.
To do this, we need to add a few lines at the bottom of inspirehep/wsgi.py
:
import os; os.mkdir('prof')
from werkzeug.contrib.profiler import ProfilerMiddleware
application = ProfilerMiddleware(application, profile_dir='prof')
Now, after we restart the application, a profile report will be created in the
prof
folder for each request that we make. These binary files can be
visualized as above with snakeviz.
Rebuild the assets (js/css bundles)¶
From the root of the code repository, you can run the helper script:
$ workon inspire
(inspire)$ ./scripts/clean_assets
This will:
- Remove all your static assets
- Gather all the npm dependencies and write them in the file package.json in the instance static folder
- Execute npm install
- Execute inspirehep collect and inspirehep assets build
You should then find all your updated assets in the static folder of your inspire installation, if you are using virtualenv:
cdvirtualenv var/inspirehep-instance/static/
Rebuild the database, the elasticsearch indexes, and reupload the demo records¶
Same as the assets, from the root of the code repository, run the script:
$ workon inspire
(inspire)$ ./scripts/recreate_records