WP4: Infrastructure
Space shortcuts
Page tree
Skip to end of metadata
Go to start of metadata

The Development Infrastructure consists of two parts.

  • The Docker-based development Infrastructure to be installed on your computer.
  • The database containing HMI images and preprocessed data, which is hosted on the FLARECAST cluster in France.

This document explains the first part, i.e how to install the FLARECAST development infrastructure on your computer.  Follow this procedure if you intend to develop your own algorithms.

Access to the Infrastructure in France is documented in Setup access to FLARECAST cluster.

In case you run into problems or think we should improve things please send a message to marco.soldati@fhnw.ch or leave a comment below. Many thanks.

System Requirements

  • 64-bit host system (OSX or Linux, Windows is currently not tested/supported)
  • min. 3.5 GB free disk space (the more the better, depends on how much HMI data you want to download)
  • min 4.0 GB RAM (the more the better)

Recommended versions

(Tested, running on cluster)

Tool/LibraryRecommended version
Docker17.09
Docker Compose 
docker-py 
IDL8.3

Install software

Note: Windows has not been tested yet

SoftwareOS XLinux
Python

Install https://www.python.org/downloads/release/python-2711/

Should be preinstalled. Test with 

python --version

Otherwise check here.

PIP (Python installer)

PIP is shipped with Python 2.7.9+ and Python 3.4+.

pip --version

If you don't have it follow https://pip.pypa.io/en/stable/installing/

Python IDEWe suggest PyCharm, but any text editor or Spyder will work as well.
IDL

You can use your normal IDL setup to access data and prototype your algorithms.

Note: We do not support execution of IDL code on our cluster due to licensing issues. Let us know if this is a problem.

Please avoid SolarSoft/SSW-specific functionality in your algorithms. They tend to be hard to be ported to a cluster.

Git

(source code repository)

  1. Install XCODE though the App-Store

  2. open terminal

    xcode-select --install

  3. Click on "Install" and follow the wizard

sudo apt-get install git

Docker and

docker-compose

Install Docker community edition.

https://docs.docker.com/engine/installation/

  1. Start "Docker", it will appear in your toolbar at the top of the screen
  2. Open Terminal and continue

Requires Kernel 3.10+

sudo curl -L https://get.docker.com/ | sh
sudo pip install docker-compose

To run any docker or docker-compose commands without the use of sudo:

Add user to docker group
sudo usermod -aG docker $USER
# logout and login again to apply the changes
 

Setup Docker

Make sure you have decent internet connection.

On OS X

  1. Run "/Applications/Docker.app", if needed
  2. Open any terminal and continue

On Linux

Usually, the Docker daemon is started automatically.

If not, follow this guide: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-getting-started

 

Create a base directory for the FLARECAST development infrastructure (e.g. mkdir ~/flarecast/dev-infra). We assume the following commands are executed in this directory.

Inititalize docker
# download the helper script
local> curl -o infrastructure.sh https://dev.flarecast.eu/stash/projects/INFRA/repos/dev-infra/browse/infrastructure.sh?raw
local> chmod a+x infrastructure.sh

# setup infrastructure or update to most recent version
local> ./infrastructure.sh update

# run development infrastructure
local> ./infrastructure.sh run

# when you want to stop the development infrastructure, press <CTRL> + C

# cleanup everythin, this may be necessary in case of errors or old and incompatible containers
# this will also remove all the data you sent to the hmi/property services
local> ./infrastructure.sh clean

# in case you need already generated data, add a flag "keep_db"
local> ./infrastructure.sh clean keep_db

All FLARECAST docker containers can be found at DockerHub

Troubleshooting:

curl: SSL certificate problem: Invalid certificate chain (OS X only)

  1. Open Safari (not Firefox, Opera, Chrome or similar)
  2. visit https://dev.flarecast.eu
  3. A certificate warning should pop up. -> Click "Show Certificate"
  4. Click on "Trust" and set to "Always trust"
  5. Retry curl

curl: SSL certificate problem: unable to get local issuer certificate

  1. Run curl with the --insecure option

docker: cannot resolve hostname

  1. Add your DNS server (xx.xx.xx.xxx) to '\etc\default\docker', e.g.
    DOCKER_OPTS="--dns xx.xx.xx.xxx --dns 8.8.8.8 --dns 8.8.4.4"
    where 8.8.8.8 and 8.8.4.4 are the Google DNS servers
  2. Restart the docker service
    sudo restart docker

Test Docker

 

On OS X

After starting infrastructure.sh switch to "Kitematic"

Select Container "devinfrastest_webapp_1"

On the right side panel click on the arrow icon next to "Web Preview"

Kitematic Console

On Linux

Open http://127.0.0.1/ in a browser
  • No labels