...
In our example we run a machine learning algorithm which produces a flare prediction to store within our database. Hereby, the algorithm consists of a training phase and a test or prediction phase. Within the training phase the algorithm learns and tunes its parameters which are then can be stored within the database as a configuration for later use. Afterwards, within the prediction test phase we use this configuration to compute flare predictions which are also stored within the database. The following code shows two corresponding functions.
| Code Block | ||||
|---|---|---|---|---|
| ||||
def train_model(model, train_data, validation_data, max_epoches, batch_size, environment): # train model (e.g. until max_epoches are reached or validation loss increases) ... model.train(train_data, validation_data, max_epoches, batch_size) # store model parameters within database post_data = { "algorithm_run_id": environment.r_id, "config_data": model.get_parameters(), "description": {}, } requests.post('http://localhost:8004/algoconfig/%s' % cfg_name, data=post_data) def test_model(model, test_data, environment): # test model (e.g. predict the test_data) ... # store predictions within database (time_start, position_hg, prediction_data) = model.get_prediction() # store predictions within database post_data = [ { "algorithm_config": environment['algorithm']['cfg_name'], "algorithm_run_id": renvironment['runtime']['run_id'], "lat_hg": position_hg[0], "long_hg": position_hg[1], "prediction_data": prediction_data, "source_data": [get_fc_id(row) for row in test_data], "time_start": time_start } ] requests.post('http://localhost:8004/prediction/bulk', data=post_data) |
Implementation
Prepare
...
Integration
Given the two above functions we can now define our algorithm's workflow.
| Code Block | ||||
|---|---|---|---|---|
| ||||
import json import requests # Setup environment = {} with open("params.json") as params_file: environment = json.loads(params_file.read()) algo_config_name = "my_ml_configuration" algo_config = {} response = requests.get('http://localhost:8004/algoconfig/list?algorithm_config_name=%s&algorithm_config_version=latest' % algo_config_name).json() if response['has_error'] == False and response['result-count'] > 0: # as we requested the latest configuration we expect only one result within 'data' algo_config = response['data'][0] else: # http://localhost:8004/algoconfig/list?algorithm_config_name=a #if |
...
| Info |
|---|
This page was adopted from Ingest property data in database (REST API). |
params_file