Page History

...

ID

Description

Solution

1

TypeError while calling requests.post() command, e.g.:

TypeError: request() got an unexpected keyword argument 'json'

This error occurs with older versions of requests < 2.4.2.

[A] Update requests (recommended):

  pip install requests --upgrade

[B] Manually set the header definitions while calling 'my_url' with 'my_json_data':

Code Block

language	py

response = requests.post(
    my_url, data=json.dumps(my_json_data),
    headers={"content-type": "application/json"}
).json()

2

Serialization of complex data structures:

Simple data structures (e.g. a dictionary of dictionaries or arrays) are directly serializable by the json module and do not need any conversion (see code example above). However, with more complex structures the following errors may occur:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x... in position ...: invalid start byte
Error message from the prediction service:
<class 'psycopg2.DataError'>: unsupported Unicode escape sequence
DETAIL: \u0000 cannot be converted to text.

In both cases we try to serialize characters which are not supported either by the codec (e.g. utf-8) or the database.

[A] Using the bas64 codec we can 'transform' each character into a simple ASCII representation (escaping all special characters):

Code Block

language	py

import base64
import pickle
...
p_object = pickle.dumps(my_object, 2)  # serializes my_object, using protocol "2"
d_object = base64.b64encode(p_object).decode('ascii')  # encodes serialized object
post_data = {
    "algorithm_run_id": environment['runtime']['run_id'],
    "config_data": {"my_object": d_object},
    "description": "..."
}
response = requests.post(
    'http://localhost:8004/algoconfig/%s' % environment['algorithm']['cfg_name'],
    json=post_data
).json()
  
# check wherever upload was successful (see example code above)
  
response = requests.get(
    'http://localhost:8004/algoconfig/list?algorithm_config_name=%s'
    % environment['algorithm']['cfg_name']
).json() 
d_object = response['data'][0]['config_data']['my_object']
p_object = base64.b64decode(d_object)  # decodes serialized object
my_object = pickle.loads(p_object)  # deserializes my_object

Info
In the code example above we pickle my_object using protocol 2 instead of the default protocol 0 (see line 4). Protocol 2 is hereby recommended especially for new-style classes, which are used by most modern libraries.

...

Space shortcuts

Page tree

Versions Compared

Old Version 17

New Version 18

Key