The Development Infrastructure consists of two parts.
- The Docker-based development Infrastructure to be installed on your computer.
- The database containing HMI images and preprocessed data, which is hosted on the FLARECAST cluster in France.
This document explains how to install the FLARECAST development infrastructure on your computer and how to configure access to the data in France.
In case you run into problems or think we should improve things please send a message to marco.soldati@fhnw.ch or leave a comment below. Many thanks.
System Requirements
- 64-bit host system (OSX or Linux, Windows is currently not tested/supported)
- min. 3.5 GB free disk space (the more the better, depends on how much HMI data you want to download)
- min 4.0 GB RAM (the more the better)
Recommended versions
(Tested, running on cluster)
| Tool/Library | Recommended version |
|---|---|
| Docker | 1.9.1 |
| Docker Compose | 1.5.1 |
| docker-py | 1.5.0 |
| IDL | 8.3 |
Install software
Note: Windows has not been tested yet
| Software | OS X | Linux | ||
|---|---|---|---|---|
| Python | Install https://www.python.org/downloads/release/python-2711/ | Should be preinstalled. Test with python --version Otherwise check here. | ||
| PIP (Python installer) | PIP is shipped with Python 2.7.9+ and Python 3.4+. pip --version If you don't have it follow https://pip.pypa.io/en/stable/installing/ | |||
| Python IDE | We suggest PyCharm, but any text editor or Spyder will work as well. | |||
| IDL | You can use your normal IDL setup to access data and prototype your algorithms. Note: We do not support execution of IDL code on our cluster due to licensing issues. Let us know if this is a problem. Please avoid SolarSoft/SSW-specific functionality in your algorithms. They tend to be hard to be ported to a cluster. | |||
Git (source code repository) |
|
| ||
Docker and docker-compose | Install https://www.docker.com/docker-toolbox
|
| ||
Setup Access to Infrastructure
This step is required to get access either to the HMI files stored in France or the properties extracted by our algorithms.
If you want to do all processing on your local machine, you can skip this step.
In order to access the FLARECAST development infrastructure in France you need a dedicated account.
To get an account send a message to Eric Buchlin <eric.buchlin@ias.u-psud.fr>.
With this account you can connect to the computing infrastructure. The following figure outlines how this connection is being established.
Within this process you will be asked twice to enter your credentials. If you want to avoid this you can generate and install an SSH certificate as follows:
# Open terminal on your machine local> ssh-keygen -t rsa -C "your-mail-address" # -C is just a comment Enter file in which to save the key (/Users/mercuron/.ssh/id_rsa): # use default Enter passphrase (empty for no passphrase): # protect your private key Enter same passphrase again: # protect your private key ... Your public key has been saved in ~/.ssh/id_rsa.pub. ... local> scp ~/.ssh/id_rsa.pub username@ias-ssh.ias.u-psud.fr:~/.ssh/authorized_keys
local> ssh username@ias-ssh.ias.u-psud.fr inf-ssh2> ssh-keygen -t rsa -C "your-mail-address" # -C is just a comment # use default settings and no password inf-ssh2> scp ~/.ssh/id_rsa.pub username@cluster-head:~/.ssh/authorized_keys inf-ssh2> ssh username@cluster-head # to test the connection cluster-head> exit inf-ssh2> exit local>
# create file .ssh/config to auto-forward yourself to cluster-head local> nano .ssh/config # content of .ssh/config Host flarecast-infra ProxyCommand ssh username@ias-ssh.ias.u-psud.fr nc cluster-head.ias.u-psud.fr 22 User username ForwardX11Trusted yes ServerAliveInterval 60 # Ctrl-x y <return> to save and close local> ssh flarecast-infra # accept fingerprint # enter FLARECAST cluster password # you can ignore message: /usr/bin/xauth: file ~/.Xauthority does not exist cluster-head> exit local>
From now on you can login to the cluster as follows
local> ssh flarecast-infra # enter FLARECAST cluster password cluster-head> exit local>
Setup Docker
Make sure you have decent internet connection.
On OS X
- Run "
/Applications/Docker/Kitematic (Beta)" - Click
"Docker CLI"on the lower right side to open a terminal and use this terminal for all Docker commands.
On Linux
Usually, the Docker daemon is started automatically.
If not, follow this guide: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-getting-started
Create a base directory for the FLARECAST development infrastructure (e.g. mkdir ~/flarecast/dev-infra). We assume the following commands are executed in this directory.
local> curl -o docker-compose.yml https://dev.flarecast.eu/stash/projects/INFRA/repos/dev-infra/browse/docker-compose.yml?raw # if you want to run smart on your computer local> docker-compose pull ... # make sure that the database is running local> docker-compose up -d db_service ... # wait a few seconds before the next step in order to let the database start # start up the docker environment local> docker-compose up
All FLARECAST docker containers can be found at DockerHub
Troubleshooting:
curl: SSL certificate problem: Invalid certificate chain (OS X only)
- Open Safari (not Firefox, Opera, Chrome or similar)
- visit https://dev.flarecast.eu
- A certificate warning should pop up. -> Click "Show Certificate"
- Click on "Trust" and set to "Always trust"
- Retry curl
Test Docker
On OS X
After starting docker-compose switch to "Kitematic"
Select Container "devinfrastest_webapp_1"
On the right side panel click on the arrow icon next to "Web Preview"
On Linux