Page History

TypeError while calling requests.post() command, e.g.:

• TypeError: request() got an unexpected keyword argument 'json'

This error occurs with older versions of requests < 2.4.2.

[A] Update requests (recommended):

  pip install requests --upgrade

[B] Manually set the header definitions while calling 'my_url' with 'my_json_data':

response = requests.post(
    my_url, data=json.dumps(my_json_data),
    headers={"content-type": "application/json"}
).json()

Serialization of complex data structures:

Simple data structures (e.g. a dictionary of dictionaries or arrays) are directly serializable by the json module and do not need any conversion (see code example above). However, with more complex structures the following errors may occur:

• UnicodeDecodeError: 'utf-8' codec can't decode byte 0x... in position ...: invalid start byte
• Error message from the prediction service:
  <class 'psycopg2.DataError'>: unsupported Unicode escape sequence
  DETAIL: \u0000 cannot be converted to text.

In both cases we try to serialize characters which are not supported either by the codec (e.g. utf-8) or the database.

[A] Using the bas64 codec we can 'transform' each character into a simple ASCII representation (escaping all special characters):

import base64
import pickle
...
p_object = pickle.dumps(my_object, 0)  # serializes my_object
d_object = base64.b64encode(p_object).decode('ascii')  # encodes serialized object
post_data = {
    "algorithm_run_id": environment['runtime']['run_id'],
    "config_data": {"my_object": d_object},
    "description": "..."
}
response = requests.post(
    'http://localhost:8004/algoconfig/%s' % environment['algorithm']['cfg_name'],
    json=post_data
).json()
  
# check wherever upload was successful (see example code above)
  
response = requests.get(
    'http://localhost:8004/algoconfig/list?algorithm_config_name=%s'
    % environment['algorithm']['cfg_name']
).json() 
d_object = response['data'][0]['config_data']['my_object']
p_object = base64.b64decode(d_object)  # decodedecodes serialized object
my_object = pickle.loads(p_object)  # deserializedeserializes my_object