Page History

This short tutorial shows how to request data from the property prediction service with python and requests (see also Access to REST-Services in Python).

For IDL you can take a similar approach by adapting these instructions to Access to REST-Services in IDL.

The

...

Prediction Service

The property prediction service represents a web interface which allows to request, insert and modify property prediction data from the database. Operations are performed by sending URL requests, whereas each operation is well defined as a so called route. The property prediction service comes along with a graphical user interface at http://localhost:80028004/ui/ which provides visual access to all available routes. Hereby, all routes involving the request of data are enlisted under the View section.

Image RemovedImage Added

For this tutorial we are using only one route two routes. One to verify wherever an algorithm configuration exists and one to request all available regions within predictions from our database.

/region/{dataset}/algoconfig/list
/prediction/list

Implementation

Request

In our example we run a machine learning algorithm which produces a set of flare predictions to store within our database. Hereby, the algorithm consists of a training phase and a testing or prediction phase. Within the training phase the algorithm learns and tunes its parameters which then can be stored within the database as a configuration for later use. Afterwards, within the testing phase, we use this configuration to compute flare predictions which are also stored within the database.

For the following example we assume, as the

...

two functions 'train_model' and 'test_model' are already defined within the python script. Otherwise, we refer to the

...

How-to: Ingest

...

prediction data in database (REST API).

The request of all available regions is then very simple. We firstly have to define the dataset ml_dataset which contains our data before we call the above given route.Given the two above functions we could define our algorithm's workflow as follows:

Code Block

language	py

import json
import requests

# definesetup theenvironment
dataset
ml_dataset = "ml-algorithms";

# retrieving data
print('downloading all properties...')
ml_regions = requests.get("http://localhost:8002/region/%s/list" % ml_dataset).json()

print(ml_regionsenvironment = {}
with open("params.json") as params_file:
    environment = json.loads(params_file.read())
  
# download required regions and properties
response = requests.get(
    'http://localhost:8002/region/%s/list?cadence=%s&time_start=between(%s,%s)'
    % (
        environment['algorithm']['dataset'],
        environment['algorithm']['cadence'],
        environment['algorithm']['time_start'],
        environment['algorithm']['time_end']
    )
).json()
if response['has_error'] == True or response['result-count'] == 0:
    raise ValueError('No data found!')
else:
    # FIXME: Bad Implementation
    # if we already have an algorithm configuration stored within the database,
    # we do not need to extract any training nor validation data.
    (train_data, validation_data, test_data) = create_data_partitions(response['data'])
  
# setup model
model = DemoMLAlgorithm(environment['algorithm']['params'])
  
# check wherever we have to train our algorithm or if we can download an already existing configuration
response = requests.get(
    'http://localhost:8004/algoconfig/list?algorithm_config_name=%s&algorithm_config_version=%s'
    % (environment['algorithm']['cfg_name'], 'latest')
).json()
if response['has_error'] == False and response['result-count'] > 0:
    # as we requested the latest configuration we expect only one result within 'data'
    algo_cfg = response['data'][0]['config_data']
    model.set_parameters(algo_cfg)
else:
    train_model(
        model, train_data, validation_data,
        environment['algorithm']['max_epoches'], environment['algorithm']['batch_size'],
        environment
    )

# run algorithm
prediction = test_model(model, test_data, environment)
prediction_ids = [get_fc_id(row) for row in prediction]

# check wherever the new prediction was successfully stored within the database
response = requests.get(
    'http://localhost:8004/prediction/list?prediction_fc_id=eq(%s)'
    % (','.join(prediction_ids))
).json()
if response['has_error'] == True or response['result-count'] == 0:
    raise AssertionError('No predictions found!')
else:
    for prediction in response['data']:
        print(prediction)

Source Code

Here you can download the full python source code.

request_ property prediction_data.py

For a more detailed implementation with dummy data you can download the following demo script (recommended).

params.json

request_prediction_data_demo.py

Info
This page was adopted from Request property data from database (REST API).

Space shortcuts

Page tree

Versions Compared

Old Version 1

New Version Current

Key

The

Prediction Service

Implementation

Request

Source Code