Janus Server

Update

If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.

Edit the janus.plugin.streaming.jcfg file in the /var/snap/janus-gateway/common/etc  directory and guarantee that the admin_key is defined:

general :
{
  admin_key = "elab1";
  rtp_port_range = "6000-7000";
};

Installation

To install the Janus stream server you will need to run the following commands on your Debian 11 machine:

sudo apt install snap

sudo apt install snapd

sudo snap install janus-gateway

Configuration

The configuration of Janus is done editing a file in the /var/snap/janus-gateway/common/etc directory

Edit the janus.plugin.streaming.jcfg file

Define an administration secret by uncommenting the line 104) or the one that contains the admin_key string= and changing the string to a suitable value.

Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string rtp_port_range= and define the available ports (for example to 6000-7000).

Delete the sample pre-configure endpoints by deleting all lines from line 124.

Restart Janus by running the command:

sudo snap restart janus-gateway

Example of Janus Configuration

After the configuration of a newly installed Janus Server the janus.plugin.streaming.jcfg should only containg the following`

general :
{
  admin_key = "super_pass";
  rtp_port_range = "6000-7000";
};

HTTPS configurations

If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS. It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), it must be created by the network administrator of the domain.

This certificate should be placed in a system directory before editing the janus.transport.http.jcfg configuration file:

Disable HTTP: http = false
Enable HTTPS: https = true
define the HTTPS port: secure_port = 8088 - Set the certificate directory:
- cert_pem = "/some_system_directory/janus_public.crt"
- cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:
JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"

FREE Server

New instalation

Installation of Source Code

Create a new folder for the new version of FREE.

For example mkdir wpa_free_060

Download the Zip file containing the code

wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip

Unzip the downloaded file into the new directory.

For example unzip FREE_Web_0_6_0.zip -d wpa_free_060/

Enter the directory containing the code.

For example cd wpa_free_060/

Create a new python virtual environment, it should be called free-env:

virtualenv -p python3 free-env

Activate the virtual environment:

source free-env/bin/activate

You should now see a prefix of (free-env) in your command line.

After that, install the dependent packages:

pip install -r REQUIREMENTS.txt

Installation of New Database

Since the data model was change it is necessary to download and install a new empty database.

This database is configured with two users and the WPA pendulum apparatus types and protocols.

Download the database and put it into the project root:

wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3

This database contains a Pendulum experiment, as well as two example user accounts.

wp-admin - superuser account with access to admin interface
wp-guest - guest account

Both users have a temporary123 password.

This password should be changed through admin interface.

Configuration

The application is configured using environment variables.

You can set them using the /freeweb/.env file.

There is a .env-template file in the ~freeweb folder, that can be renamed to .env :

mv freeweb/.env-template freeweb/.env

It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.

PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header
TIME_ZONE - Time zone setting for that app in TZ database name from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.
FREE_PRODUCTION - set to on to enable production mode (disables sensitive error messages etc.)
FREE_REVERSE_PROXY - set to on to be able to run the FREE behind a reverse proxy
FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir
FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.
JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.

FREE server execution

to start the FREE server run the following command:

daphne freeweb.asgi:application

By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. You can also force binding to specific address by -b <address>.

Copy of Pendulum Information

Since a new database is now being used, it is necessary to copy the various apparatus into the new database.

If running both version at the same time, one of them, should be executed on a different port

On the administration page of the you should copy some of the information from the old database to the new.

here is the list and description of the various fields:

Apparatus Type : Pendulum
Location - description of the physical place of the pendulum
Description - specific characteristics of the pendulum
Secret - a random string specific to each pendulum
Owner - Name of the person responsible for the operation and maintenance of the pendulum
Timeout - leave 60
Configuration - copy the following json and modify accordingly the id and ports_restrict to suitable values:

{
   "id": "WP_LIS_IST", 
   "serial_port": {
      "ports_restrict": ["/dev/ttyS0"], 
      "baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", 
      "listening_timeout": "100000", "death_timeout": "10000000"
   }
}

video configuration - either copy the value from the old database or leave empty

Configuration of Video

In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the .env file:

JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.

After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.

Open the Video Config menu:

On the new page, a table will contain all the configured pendulums and the associated video configuration:

In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.

The second apparatus does not have the video stream configure. To create a video stream and update the database click Configure Video

On the new page it is necessary to click on the Assign Stream button.

If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:

Upgrade of previous instalation

YOU must do a BACKUP of your BD and current FREE version installation!

Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while .env and .sqlite files won't be touched.

A new release might contain changes to the database structure. If you are preserving your database from a previous version, run

python manage.py migrate

HTTPS configuration

IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS. this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:

Cerificates

FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly. Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the freeweb/certificates/ directory For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt

Server execution

After the creation or installation of the certificate the server should be executed as follows

daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application

If running with this configuration, users should use the following url to access FREE:

https://hostname.some_domain:8000/

External Authentication

FREE now allows the use of external services from Google and Microsoft for user authentication. To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.

GOOGLE

Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:

access the Google API Console: https://console.developers.google.com/
Create a new project
Credentials
- Create Credentials -> OAuth client ID
Configure Consent Screen
- Select External User Type
- Fill The App information screen (app name, User support email, Developer contact information)
- Select scopes (may be empty)
- Add a test user
Credentials
- Create credentials -> OAuth client ID
- Application Type - Web application
- Define name
- Add Authorized redirect URIs
  - append /complete/google-oauth2/ to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)
- Create
Download json or save the Client ID and Client Secret
Edit .env file:
- activate FREE_GOOGLE_OAUTH:
  - FREE_GOOGLE_OAUTH=true
- copy the value of Your Client ID to SOCIAL_AUTH_GOOGLE_OAUTH2_KEY
- copy Client Secret to SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET
- restart the FREE installation

Microsoft

Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:

Sign in to the Azure portal: https://portal.azure.com/
Manage Azure Active Directory - View
Select App registrations on the menu
New registration
- Define the name of teh application
- Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)
- Define the redirect URI
  - Select a platform - Web
  - to create the URI, append /complete/microsoft-graph/ to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)
- Register
- Save the Application (client) ID
  - this value should be put in the .env file
- Select Certificates and secrets on the menu
  - New client secret
  - Add
- Save the Value
  - this value should be put in the .env file The secret Value dissapears after closing this page
Edit .env file:
- activate FREE_MS_OAUTH:
  - FREE_MS_OAUTH=true
- copy the value of Application (client) ID to SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY
- copy the value of Secret ID to SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET
- restart the FREE installation

Raspberry Pi Proxy

Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.

Installation of Source Code

Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).

For example mkdir Proxy_051

Download the Zip file containing the code

wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip

Unzip the downloaded file into the new directory.

For example unzip RPi_Proxy_0_5_1.zip -d Proxy_051/

Enter the directory containing the code.

For example cd Proxy_051/

After that, install the dependent package:

For the Proxy:

sudo apt install python3

sudo apt install python3-pip

pip3 install pyserial

And for streaming the video (for gstreamer ):

sudo apt-get install gstreamer1.0-tools

sudo apt-get install gstreamer1.0-plugins-good

sudo apt-get install gstreamer1.0-plugins-bad

sudo apt-get install gstreamer1.0-plugins-ugly

sudo apt-get install gstreamer1.0-plugins-base

For ffmpeg :

sudo apt-get install ffmpeg

Configuration

Video Streaming Configuration

In order to configure the video streaming process on the Raspberry pi edit the video-stream.ini file.

The following values should be correctly set:

video_server video_port apparatus_location apparatus_name apparatus_id
- copy this information from the Video Conf administration page on the FREE server

usb_camera video_width video_height video_frame
- find the suitable values depending on the network resources and the information provided by the following commands:

v4l2-ctl --list-devices
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext

In order to debug and verify the video configuration it is possible to execute the video-stream.sh command.

Proxy Configuration

In order to configure the Proxy on Raspberry pi edit the server_info.ini file.

If the FREE server is configured to use HTTPS it needs to set the following parameter in the server_info.ini file:

HTTPS = True

Also, the following values should be correctly set:

SERVER : The IP address or domain of your FREE_Web server;
PORT : The port of the server by default is the 8000;
DEBUG: This can be change to off, after this installation is correctly done;
APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;

On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:

Proxy execution

In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.

After all the parameters are correct, both programs can be executed simultaneously using the following command:

nohup sh start-wp.sh > /dev/null 2>&1 &

This command runs until the raspberry pi is rebooted.

To make the Proxy run when the Raspberry pi reboots it is necessary to edit the /etc/rc.local and add the following lines before the exit 0 :

sleep 60
cd /INSTALLATION_FOLDER/PATH
su USER -c "sh start-wp.sh &"

Replace /INSTALLATION_FOLDER/PATH with the output of the following command:

readlink -f Proxy_051/

Replace USER the the actual user owner of the proxy.

How to add new experiments

First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.

If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones).

Apparatus Type

To create a new Apparatus Type on the database just click on the "+" on the left of the pag.

Label.

By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:

Name-> The name of the type of experiment;

Slug name-> the name of the HTML file of your experiment (explain on the section ...);

Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);

Scientific area-> what area is the experience integrated;

Lab type-> the background needed to execute the experiment (basic, intermedium, advance).

Protocol

After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.

Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols.

Label.

You will have a page like the one shown above that you will need to file:

Apparatus Type -> select your Apparatus Type;

Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;

Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema here);

Description-> a small discription can be made to give the user same information of the limitation of the protocol.

Apparatuses

After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).

Label.

You will have a similar page as shown above, so you need to fill the following information:

Apparatus Type-> the one Apparatus Type that the apparatus is connected to;

Protocol-> the protocols related to that Experiment;

Location-> location of the experiment;

Description->

Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;

Owner-> the person responsible for the maintenance of the experiment;

Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).

HTML/JS

With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:

Label.

Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration

Label.

you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.

Label.

To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment.

Label.
Label.

After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".

WPA FREE 0.6.0 Instalation

Contents

Janus Server

Update

Installation

Configuration

Example of Janus Configuration

HTTPS configurations

FREE Server

New instalation

Installation of Source Code

Installation of New Database

Configuration

FREE server execution

Copy of Pendulum Information

Configuration of Video

Upgrade of previous instalation

HTTPS configuration

Cerificates

Server execution

External Authentication

GOOGLE

Microsoft

Raspberry Pi Proxy

Installation of Source Code

Configuration

Video Streaming Configuration

Proxy Configuration

Proxy execution

How to add new experiments

Apparatus Type

Protocol

Apparatuses

HTML/JS

Navigation menu

Search