WP4: Infrastructure
Space shortcuts
Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

This short tutorial shows how to store data into the property service with python and requests. 

Routes

First of all we have to know which routes we need to use. For this it is recommended to take a look at the query builder or swagger ui (http://localhost:8002/ui/). Under the Edit tab there are all routes which can be used to insert or update data of the property service.

For this tutorial we are using just two of them. One to add new provenances and one to add new regions to the property service.

  • /provenance
  • /region/{provenance}

Requests

These two routes are both POST routes which requires them to be called with a POST request. To create simple post requests we recommend the python package requests.

A POST request needs two parameter: One is the address itself which is given by the route and the second one is the data which will be attached to the request.

r = requests.post("http://httpbin.org/post", json = {"key":"value"})

Implementation

Prepare

First of all you have to define the data you would like to store into the property service. In our example we are going to store some machine learning results into the provenance ml-algorithms.

# define data to store
ml_provenance = 'ml-algorithms'
ml_result = {
    'time_end': '2016-01-22T17:27:59.001Z',
    'algorithm_name': 'LassoCV',
    'algorithm_parameters': {
        'store': 'what you want',
        'also here': [1, 2, 3, 4]
    }
}

Now the problem is that the ml_result is not structured in the way that the property service would understand it. So we have to restructure it to accomplish following definition:

  • Every region needs at least a time_start attribute.
  • Every specific data has to be added to the data attribute.

This would then look like this:

# define data to store
post_data = {
    'time_start': '2016-01-21T17:27:59.001Z',
    'data': ml_result
}

Ingest

Now the ingest of this post_data is very simple. We first have to add the new provenance to the property service and then add a new region.

# add provenance
print('creating provenance...')
requests.post("http://localhost:8002/provenance", json=ml_provenance)

# add region
print('storing data...')
requests.post("http://localhost:8002/region/%s" % ml_provenance, json=post_data)

The addresses of the routes are the same we looked up before.

Retrieve

Now to check if everything worked we should retrieve all the properties which are stored under this provenance. This can be done with a GET request.

# retrieving data
print('downloading all properties...')
ml_properties = requests.get("http://localhost:8002/property/%s/list" % ml_provenance).json()

print(ml_properties)

Source Code

Here you can download the full python source code.

request_ingest.py

  • No labels