WP4: Infrastructure
Space shortcuts
Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 18 Next »

This section explains how to setup and configure the FLARECAST development infrastructure on your personal computer.

System Requirements

  • 64-bit host system (OSX or Linux, Windows is currently not tested/supported)
  • min. 3.5 GB free disk space (the more the better, depends on how much HMI data you want to download)
  • min 4.0 GB RAM (the more the better)

Install software

Note: Windows has not been tested yet

SoftwareOS XLinux
Python

Install https://www.python.org/downloads/release/python-2711/

Should be preinstalled. Test with 

python --version

Otherwise check here.

PIP (Python installer)

PIP is shipped with Python 2.7.9+ and Python 3.4+.

pip --version

If you don't have it follow https://pip.pypa.io/en/stable/installing/

Python IDEWe suggest PyCharm, but any text editor or Spyder will work as well.
IDL

You can use your normal IDL setup to access data and prototype your algorithms.

Note: We do not support execution of IDL code on our cluster due to licensing issues. Let us know if this is a problem.

Please avoid SolarSoft/SSW-specific functionality in your algorithms. They tend to be hard to be ported to a cluster.

Git

(source code repository)

  1. Install XCODE though the App-Store

  2. open terminal

    xcode-select --install

  3. Click on "Install" and follow the wizard

sudo apt-get install git

Docker and

docker-compose

Install https://www.docker.com/docker-toolbox

  1. Start "Kitematic (Beta)" and wait until Virtual Machine is created.
  2. Stop "Kitematic (Beta)"
  3. Open "VirtualBox" and look for VM called "default"
  4. Stop VM "default" (e.g. send "ACPI Shutdown")
  5. "Settings" - "System" - "Motherboard" - Base Memory: 4GB
  6. Depending on your system you may want to increase the
    number of processors.
  7. Restart "Kitematic (Beta)"

Requires Kernel 3.10+

sudo curl -L https://get.docker.com/ | sh
sudo pip install docker-compose
 

Setup Access to Infrastructure

 

This step is required to get access either to the HMI files stored in France or the properties extracted by our algorithms.

If you want to do all processing on your local machine, you can skip this step.

In order to access the FLARECAST development infrastructure in France you need a dedicated account.

To get an account send a message to Eric Buchlin <eric.buchlin@ias.u-psud.fr>.

 

With this account you can connect to the computing infrastructure. The following figure outlines how this connection is being established.

How to access the computing infrastructure in Meudon

 

Within this process you will be asked twice to enter your credentials. If you want to avoid this you can generate and install an SSH certificate as follows:

Create RSA key
# Open terminal on your machine
local> ssh-keygen -t rsa -C "your-mail-address"                        # -C is just a comment
Enter file in which to save the key (/Users/mercuron/.ssh/id_rsa):     # use default
Enter passphrase (empty for no passphrase):                            # protect your private key
Enter same passphrase again:                                           # protect your private key
...
Your public key has been saved in ~/.ssh/id_rsa.pub.
...
local> scp ~/.ssh/id_rsa.pub username@ias-ssh.ias.u-psud.fr:~/.ssh/authorized_keys
Create RSA key from login host to cluster-head
local> ssh username@ias-ssh.ias.u-psud.fr
inf-ssh2> ssh-keygen -t rsa -C "your-mail-address"         # -C is just a comment
# use default settings and no password
inf-ssh2> scp ~/.ssh/id_rsa.pub username@cluster-head:~/.ssh/authorized_keys 
inf-ssh2> ssh username@cluster-head                        # to test the connection
cluster-head> exit
inf-ssh2> exit
local>
Configure auto forwarding
# create file .ssh/config to auto-forward yourself to cluster-head
local> nano .ssh/config
# content of .ssh/config
Host flarecast-infra
 ProxyCommand ssh username@ias-ssh.ias.u-psud.fr nc cluster-head.ias.u-psud.fr 22
 User username
 ForwardX11Trusted yes
 ServerAliveInterval 60 

# Ctrl-x y <return> to save and close 

local> ssh flarecast-infra
# accept fingerprint
# enter FLARECAST cluster password
# you can ignore message: /usr/bin/xauth:  file ~/.Xauthority does not exist
cluster-head> exit
local>

From now on you can login to the cluster as follows

Login to cluster
local> ssh flarecast-infra
# enter FLARECAST cluster password
cluster-head> exit
local>

Setup Docker

 

Make sure you have decent internet connection.

 

On OS X: Run "/Applications/Docker/Docker Quickstart Terminal"

On Linux make sure that docker daemon is running.

 

Create a base directory for the FLARECAST development infrastructure. We assume the following commands are executed in this directory.

Inititalize docker
local> curl -o docker-compose.yml https://dev.flarecast.eu/stash/projects/INFRA/repos/dev-infra/browse/docker-compose.yml?raw

# if you want to run smart on your computer
local> docker pull flarecast/smart-python
...

# if you want to run the sample artracker
local> docker pull flarecast/artracker
...

# if you want to run the sample graphic plotter
local> docker pull flarecast/regiongraph
...

# start up the docker environment
local> docker-compose up
# In case you see any errors about database connection timeouts, just press Ctrl + C and run docker-compose up again.

All FLARECAST docker containers can be found at DockerHub

Troubleshooting:

curl: SSL certificate problem: Invalid certificate chain (OS X only)

  1. Open Safari (not Firefox, Opera, Chrome or similar)
  2. visit https://dev.flarecast.eu
  3. A certificate warning should pop up. -> Click "Show Certificate"
  4. Click on "Trust" and set to "Always trust"
  5. Retry curl

 

 

  • No labels