This short tutorial shows how to store data into the property service with python and requests. 

For IDL you can take a similar rout by adapting these instructions to Access to REST-Services in IDL

Routes

First of all we have to know which routes we need to use. For this it is recommended to take a look at the query builder or swagger ui (http://localhost:8002/ui/). Under the Edit tab there are all routes which can be used to insert or update data of the property service.

For this tutorial we are using just two of them. One to add new datasets and one to add new regions to the property service.

• /dataset/bulk
• /region/{dataset}

Requests

These two routes are both POST routes which requires them to be called with a POST request. To create simple post requests we recommend the python package requests.

A POST request needs two parameter: One is the address itself which is given by the route and the second one is the data which will be attached to the request.

r = requests.post("http://httpbin.org/post", json = {"key":"value"})

Implementation

Prepare

First of all you have to define the data you would like to store into the property service. In our example we are going to store some machine learning results into the provenance ml-algorithms.

# define data to store
ml_datasets = [
  	{
   	 	"name": "ml-algorithms",
    	"responsible": "John Doe",
    	"type": "algorithm",
    	"description": "no comment!"
  	}
]
ml_result = {
    'time_end': '2016-01-22T17:27:59.001Z',
    'lat_hg': '2016-01-22T17:27:59.001Z',
    'long_hg': '2016-01-22T17:27:59.001Z',
    'algorithm_name': 'LassoCV',
    'algorithm_parameters': {
        'store': 'what you want',
        'also here': [1, 2, 3, 4]
    }
}

Now the problem is that the ml_result is not structured in the way that the property service would understand it. So we have to restructure it to accomplish following definition:

• Every region needs at least a time_start, lat_hg and long_hg attribute.
• Every specific data has to be added to the data attribute and has to be a key-value pair.

This would then look like this:

# define data to store
post_data = {
    'time_start': '2016-01-21T17:27:59.001Z',
    'lat_hg': 12,
    'long_hg': 4.2,
    'data': {
	    "LassoCV": {
			'store': 'what you want',
	        'also here': [1, 2, 3, 4]
		}
    }
}

Ingest

Now the ingest of this post_data is very simple. We first have to add the new dataset to the property service and then add a new region.

# add provenance
print('creating provenance...')
requests.post("http://localhost:8002/dataset/bulk", json=ml_datasets)

# add region
print('storing data...')
requests.post("http://localhost:8002/region/%s" % ml_datasets[0]['name'], json=post_data)

The addresses of the routes are the same we looked up before.

Retrieve

Now to check if everything worked we should retrieve all the regions which are stored under this dataset. This can be done with a GET request.

# retrieving data
print('downloading all properties...')
ml_regions = requests.get("http://localhost:8002/region/%s/list" % ml_datasets[0]['name']).json()

print(ml_regions)

Source Code

Here you can download the full python source code.

request_ingest.py