http://www.elab.tecnico.ulisboa.pt/wwwelab/wiki/api.php?action=feedcontributions&user=Ist187344&feedformat=atomwwwelab - User contributions [en]2024-03-28T20:29:58ZUser contributionsMediaWiki 1.34.2http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=Microwave_plasma_cavity&diff=4684Microwave plasma cavity2022-12-27T20:58:04Z<p>Ist187344: /* Description of the Experiment */</p>
<hr />
<div>=Description of the Experiment=<br />
[[File:MicrowaveResonantCavity_withCoils.png|229|thumb|Fig. 1 - Experimental setup with the Helmholtz coils assembled]]<br />
<br />
Plasmas are inseparable from radio-frequency studies. Effectively the most basic properties of a plasma is derived from its plasma frequency, ''ω<sub>pe</sub>''. As such, combining plasma physics studies with electromagnetic (EM) wave propagation is a challenging matter for physics with great advances achieved last century before space conquest due to the necessity of wave reflections on the ionosphere for long range communication. <br />
An electromagnetic cavity poses a good opportunity to understand the behavior of EM standing waves and how free charges in a plasma affects its resonant frequency and quality factor, interlinking the properties of matter with wave propagation.<br />
<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:380px"><br />
'''Links'''<br />
<div class="mw-collapsible-content"><br />
<br />
<!--*Video: rtsp://elabmc.ist.utl.pt/Cavidade.sdp--><br />
*Laboratory: Advanced<br />
*Control room: [https://elab.vps.tecnico.ulisboa.pt:8000/execution/create/3/3 Microwave cavity]<br />
*Level: ****<br />
<br />
</div><br />
</div><br />
<br />
==Experimental Apparatus==<br />
A microwave cavity refers to a volume enclosed by a conducting surface which can store electromagnetic (EM) energy. It can be thought of as the microwave analog of an LC circuit, albeit with multiple resonance frequencies and much larger quality factors. Three main processes govern the energy losses in a cavity: conduction losses in the cavity walls, conduction loss in the dielectric material filling the cavity, and losses through access ports or holes in the conducting surface. In the first-order approximation, these losses cause the same resonance peak broadening as the resistivity losses in an RLC circuit, causing both a resonance frequency shift and a decrease in the quality factor. <br />
<br />
Two types of resonant modes occur in electromagnetic cavities: transverse-electric modes (TE) and transverse-magnetic modes (TM). In the TE (TM) modes, the electric (magnetic) field lines are transverse to the longitudinal direction.<br />
<br />
A comprehensive derivation of all the resonant modes of a cavity is available in chapter 6 of the book by David Pozar~\cite{Pozar2011}. In a cylindrical cavity filled by a homogeneous and isotropic medium, the resonant frequencies of the TM modes are:<br />
<br />
\begin{equation}<br />
f_{nml}=\frac{c}{2\pi\sqrt{\mu_r\epsilon_r}}\sqrt{\left(\frac{p_{nm}}{a}\right)^2+\left(\frac{l\pi}{d}\right)^2},<br />
\label{eq:resonant-frequency}<br />
\end{equation}<br />
<br />
A detailed description of the cavity and all apparatus in place can be found in the paper [[:File:Resonant_cavity_final.pdf | "An accessible microwave cavity experiment for plasma density determination"]].<br />
<br />
===Discharge chamber===<br />
The present cavity is made of copper covered with a tiny surface of nickel to protect it against corrosion. A Cold Cathode Fluorescent Light (CCFL) inverter serves as a current source to generate a Penning discharge inside the cavity~\cite{Knauer1962}. The inverter converts a 12~V DC input to 1~kV/50kHz AC output which is applied to both the electrodes inside the cavity. This electrodes are basically an aluminum meshes and are electrically insulated from the cylinder's lateral surface and close the extremes of the cylinder (see figure). This configuration ensures that the discharge permeates the entire cavity uniformly. The CCFL inverter managed to sustain discharges with working gas pressures between ~10 and ~300Pa.<br />
The schematic in figure depicts the resonant cavity used in this experiment. The copper cylinder have a width of 64 mm diameter and 50 mm length. The cylinder bases consist of an aluminum mesh with a spacing inferior to one-tenth of the microwave wavelength used in the experiment. A spring bolted to the opposite side to the vacuum window holds the mesh in place, as shown in next figure. The whole cavity lies between two Helmholtz coils that generate a magnetic field of 1.4~mT per Ampere, up to the~maximum current of ~18A.<br />
<br />
Two loop antennas lie in the middle of the cylinder wall opposite to each other. The antennas have a 4~mm radius with their axis coincident with the magnetic field line of highest intensity, the TM<sub>010</sub> mode.<br />
One of the antennas injects the microwaves in the cavity, and the other antenna collects the propagated signal. A observed resonance is expected close to a frequency of 3560~MHz (see figure).<br />
<br />
===Vacuum and gas injection setup===<br />
The chamber is kept under vacuum by a rotary pump that can reach pressures of the order of 2 to 5Pa. The pump drives a common-rail primary vacuum distribution system. The rail connection is done by a solenoid valve, allowing for the isolation during the experiment. <br />
<br />
The working gas flows into the chamber through another solenoid valve equipped with a capillary injector which allows to save gas usage. It is possible to select a working pressure by adjusting the valve inflow controlling carefully the opening time window. This setup has a injection rail feed by three different working gases: helium, nitrogen and argon.<br />
<br />
===Spectrum analyzer===<br />
To generate and receive the microwaves passing through the cavity, we used the<br />
inexpensive ARNIST SSA-TG R2 spectrum analyzer. The device has a frequency<br />
range from 35 to 4500 MHz with the resolution of 200 kHz. The<br />
analyzer sweeps the frequency spectrum measuring the attenuation (in dBm) for each<br />
frequency. As the generated signal for <br />
<br />
The required number of spectra are retrieved allowing the identification and changes in the resonant frequency and Q-factor by online visualization .<br />
<br />
=Protocol=<br />
This experiment allow to determine the impact on plasma density by: (i) the influence of different atomic and molecular particles structures in the plasma generation, (ii) the effect produced by magnetic confinement and (iii) various background pressure. All this three parameters are prone to be tuned in the experiment interface.<br />
very <br />
''Note that after being not used for a long time, the firsts readings may be misleading due to wall impurities, a few experiments have to be conduced in order to validate the results.''<br />
<br />
=Data Analysis=<br />
[[File:Cavity frequency shift.png|240|thumb| Expected drift in frequency when plasma is generated inside the chamber.]]<br />
<br />
The experiment return a sequence of power spectra selected by the user, usually around the first cavity's TM mode. Due to noise and because a precise value for the peak frequency is desired, the peak central frequency should be measured taking the half-difference at -3bd. Actually, the imaginary cut at -3db intercepts the spectrum's characteristic in a steeper section of the curve, allowing for a better determination.<br />
As the plasma RF generator induces some noise due to spiky effects on the electrodes (small arching spots), eventually a low-pass data filtering shall be conducted prior to extract the peak and broadness, the later for the quality factor (Q) determination.<br />
<br />
Assuming that the resonance frequency shift ''Δf'' caused by the plasma is small with respect to the vacuum resonance frequency ''f<sub>010</sub>'', the plasma electron density with respect to ''Δf'' is given by:<br />
<br />
\begin{equation}<br />
n_e = \frac{8\pi^2m_e\epsilon_0}{e^2}f_{010}\Delta f.<br />
\label{eq:freq-shift}<br />
\end{equation}<br />
<br />
If an external magnetic field is present, this equation is only valid as long as the electric field of the mode is parallel to the external magnetic field. This approximation further assumes that the plasma does not change the shape of the mode's electric field.<br />
<br />
Nevertheless, when an external magnetic field is applied and due to confinement, it is visually clearly seen that the plasma stays aside from the walls leading to a reduced effective chamber diameter filled with plasma. Three values of magnetic field are available to study this effect (0T, 20mT, 25mT).<br />
<br />
Expected results are presented in the figure. Several types of experiments can be conducted by selecting different pressures and gases (plasma color should always be checked in the webcam to confirm the correct gas being injected).<br />
<br />
=Links=</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=Microwave_plasma_cavity&diff=4683Microwave plasma cavity2022-12-27T20:57:50Z<p>Ist187344: /* Description of the Experiment */</p>
<hr />
<div>=Description of the Experiment=<br />
[[File:MicrowaveResonantCavity_withCoils.png|229|thumb|Fig. 1 - Experimental setup with the Helmholtz coils assembled]]<br />
<br />
Plasmas are inseparable from radio-frequency studies. Effectively the most basic properties of a plasma is derived from its plasma frequency, ''ω<sub>pe</sub>''. As such, combining plasma physics studies with electromagnetic (EM) wave propagation is a challenging matter for physics with great advances achieved last century before space conquest due to the necessity of wave reflections on the ionosphere for long range communication. <br />
An electromagnetic cavity poses a good opportunity to understand the behavior of EM standing waves and how free charges in a plasma affects its resonant frequency and quality factor, interlinking the properties of matter with wave propagation.<br />
<br />
<br />
<div class="toccolours mw-collapsible mw-collapsed" style="width:380px"><br />
'''Links'''<br />
<div class="mw-collapsible-content"><br />
<br />
<!--*Video: rtsp://elabmc.ist.utl.pt/Cavidade.sdp--><br />
*Laboratory: Advanced<br />
*Control room: [http://elab.vps.tecnico.ulisboa.pt:8000/execution/create/3/3 Microwave cavity]<br />
*Level: ****<br />
<br />
</div><br />
</div><br />
<br />
==Experimental Apparatus==<br />
A microwave cavity refers to a volume enclosed by a conducting surface which can store electromagnetic (EM) energy. It can be thought of as the microwave analog of an LC circuit, albeit with multiple resonance frequencies and much larger quality factors. Three main processes govern the energy losses in a cavity: conduction losses in the cavity walls, conduction loss in the dielectric material filling the cavity, and losses through access ports or holes in the conducting surface. In the first-order approximation, these losses cause the same resonance peak broadening as the resistivity losses in an RLC circuit, causing both a resonance frequency shift and a decrease in the quality factor. <br />
<br />
Two types of resonant modes occur in electromagnetic cavities: transverse-electric modes (TE) and transverse-magnetic modes (TM). In the TE (TM) modes, the electric (magnetic) field lines are transverse to the longitudinal direction.<br />
<br />
A comprehensive derivation of all the resonant modes of a cavity is available in chapter 6 of the book by David Pozar~\cite{Pozar2011}. In a cylindrical cavity filled by a homogeneous and isotropic medium, the resonant frequencies of the TM modes are:<br />
<br />
\begin{equation}<br />
f_{nml}=\frac{c}{2\pi\sqrt{\mu_r\epsilon_r}}\sqrt{\left(\frac{p_{nm}}{a}\right)^2+\left(\frac{l\pi}{d}\right)^2},<br />
\label{eq:resonant-frequency}<br />
\end{equation}<br />
<br />
A detailed description of the cavity and all apparatus in place can be found in the paper [[:File:Resonant_cavity_final.pdf | "An accessible microwave cavity experiment for plasma density determination"]].<br />
<br />
===Discharge chamber===<br />
The present cavity is made of copper covered with a tiny surface of nickel to protect it against corrosion. A Cold Cathode Fluorescent Light (CCFL) inverter serves as a current source to generate a Penning discharge inside the cavity~\cite{Knauer1962}. The inverter converts a 12~V DC input to 1~kV/50kHz AC output which is applied to both the electrodes inside the cavity. This electrodes are basically an aluminum meshes and are electrically insulated from the cylinder's lateral surface and close the extremes of the cylinder (see figure). This configuration ensures that the discharge permeates the entire cavity uniformly. The CCFL inverter managed to sustain discharges with working gas pressures between ~10 and ~300Pa.<br />
The schematic in figure depicts the resonant cavity used in this experiment. The copper cylinder have a width of 64 mm diameter and 50 mm length. The cylinder bases consist of an aluminum mesh with a spacing inferior to one-tenth of the microwave wavelength used in the experiment. A spring bolted to the opposite side to the vacuum window holds the mesh in place, as shown in next figure. The whole cavity lies between two Helmholtz coils that generate a magnetic field of 1.4~mT per Ampere, up to the~maximum current of ~18A.<br />
<br />
Two loop antennas lie in the middle of the cylinder wall opposite to each other. The antennas have a 4~mm radius with their axis coincident with the magnetic field line of highest intensity, the TM<sub>010</sub> mode.<br />
One of the antennas injects the microwaves in the cavity, and the other antenna collects the propagated signal. A observed resonance is expected close to a frequency of 3560~MHz (see figure).<br />
<br />
===Vacuum and gas injection setup===<br />
The chamber is kept under vacuum by a rotary pump that can reach pressures of the order of 2 to 5Pa. The pump drives a common-rail primary vacuum distribution system. The rail connection is done by a solenoid valve, allowing for the isolation during the experiment. <br />
<br />
The working gas flows into the chamber through another solenoid valve equipped with a capillary injector which allows to save gas usage. It is possible to select a working pressure by adjusting the valve inflow controlling carefully the opening time window. This setup has a injection rail feed by three different working gases: helium, nitrogen and argon.<br />
<br />
===Spectrum analyzer===<br />
To generate and receive the microwaves passing through the cavity, we used the<br />
inexpensive ARNIST SSA-TG R2 spectrum analyzer. The device has a frequency<br />
range from 35 to 4500 MHz with the resolution of 200 kHz. The<br />
analyzer sweeps the frequency spectrum measuring the attenuation (in dBm) for each<br />
frequency. As the generated signal for <br />
<br />
The required number of spectra are retrieved allowing the identification and changes in the resonant frequency and Q-factor by online visualization .<br />
<br />
=Protocol=<br />
This experiment allow to determine the impact on plasma density by: (i) the influence of different atomic and molecular particles structures in the plasma generation, (ii) the effect produced by magnetic confinement and (iii) various background pressure. All this three parameters are prone to be tuned in the experiment interface.<br />
very <br />
''Note that after being not used for a long time, the firsts readings may be misleading due to wall impurities, a few experiments have to be conduced in order to validate the results.''<br />
<br />
=Data Analysis=<br />
[[File:Cavity frequency shift.png|240|thumb| Expected drift in frequency when plasma is generated inside the chamber.]]<br />
<br />
The experiment return a sequence of power spectra selected by the user, usually around the first cavity's TM mode. Due to noise and because a precise value for the peak frequency is desired, the peak central frequency should be measured taking the half-difference at -3bd. Actually, the imaginary cut at -3db intercepts the spectrum's characteristic in a steeper section of the curve, allowing for a better determination.<br />
As the plasma RF generator induces some noise due to spiky effects on the electrodes (small arching spots), eventually a low-pass data filtering shall be conducted prior to extract the peak and broadness, the later for the quality factor (Q) determination.<br />
<br />
Assuming that the resonance frequency shift ''Δf'' caused by the plasma is small with respect to the vacuum resonance frequency ''f<sub>010</sub>'', the plasma electron density with respect to ''Δf'' is given by:<br />
<br />
\begin{equation}<br />
n_e = \frac{8\pi^2m_e\epsilon_0}{e^2}f_{010}\Delta f.<br />
\label{eq:freq-shift}<br />
\end{equation}<br />
<br />
If an external magnetic field is present, this equation is only valid as long as the electric field of the mode is parallel to the external magnetic field. This approximation further assumes that the plasma does not change the shape of the mode's electric field.<br />
<br />
Nevertheless, when an external magnetic field is applied and due to confinement, it is visually clearly seen that the plasma stays aside from the walls leading to a reduced effective chamber diameter filled with plasma. Three values of magnetic field are available to study this effect (0T, 20mT, 25mT).<br />
<br />
Expected results are presented in the figure. Several types of experiments can be conducted by selecting different pressures and gases (plasma color should always be checked in the webcam to confirm the correct gas being injected).<br />
<br />
=Links=</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4662WPA FREE 0.6.0 Instalation2022-10-27T09:06:55Z<p>Ist187344: /* HTTPS configuration */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== Pre-requisites ==<br />
<br />
* Git (2.30.2+)<br />
sudo apt install git<br />
* Python (3.8.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide (as described next). These commands should be executed in the current FREE installation directory and will overwrite existing files with the exception of '''.env ''' and '''.sqlite''' files.<br />
<br />
=== Installation of Source Code ===<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the FREE directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip'''<br />
<br />
Activate the virtual environment created at the previous FREE installation:<br />
source free-env/bin/activate<br />
<br />
Install any new necessary packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
Also, make sure that the file '''.env''' contanies the following key (<span style="color:#FFFFFF; background:#F50000">if they are missing copy them from here</span>):<br />
<br />
FREE_FENIX_OAUTH=off<br />
SOCIAL_AUTH_FENIX_AUTH_KEY=<br />
SOCIAL_AUTH_FENIX_AUTH_SECRET=<br />
<br />
FREE_GOOGLE_OAUTH=off<br />
SOCIAL_AUTH_GOOGLE_OAUTH2_KEY=<br />
SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET=<br />
<br />
FREE_MS_OAUTH=off<br />
SOCIAL_AUTH_MICROSOFT_GRAPH_KEY=<br />
SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET=<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
This requires the installation of the new version of the proxy 0.6.0 on every experiment and putting the HTTPS key as True on the '''server_info.ini''', the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes in the API and the database you will need to make an upgrade to this version of the Proxy to 0.6.0. <br />
<br />
<span style="color:#FFFFFF; background:#F50000">If upgrading, remember to make a backup of the old version!</span><br />
<br />
== Pre-requisites ==<br />
<br />
Operating System:<br />
* Raspbian GNU/Linux 10 (buster)<br />
For the Proxy it self: <br />
* Python (3.7.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
pip3 install pyserial<br />
* Git (2.30.2+)<br />
sudo apt-get install git<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible with the Version of the FREE server).<br />
* For example '''mkdir Proxy_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_6_0.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_6_0.zip -d Proxy_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_060/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_060/<br />
<br />
Replace USER with the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4661WPA FREE 0.6.0 Instalation2022-10-23T23:29:36Z<p>Ist187344: /* Upgrade of previous instalation */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== Pre-requisites ==<br />
<br />
* Git (2.30.2+)<br />
sudo apt install git<br />
* Python (3.8.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide (as described next). These commands should be executed in the current FREE installation directory and will overwrite existing files with the exception of '''.env ''' and '''.sqlite''' files.<br />
<br />
=== Installation of Source Code ===<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the FREE directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip'''<br />
<br />
Activate the virtual environment created at the previous FREE installation:<br />
source free-env/bin/activate<br />
<br />
Install any new necessary packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
Also, make sure that the file '''.env''' contanies the following key (<span style="color:#FFFFFF; background:#F50000">if they are missing copy them from here</span>):<br />
<br />
FREE_FENIX_OAUTH=off<br />
SOCIAL_AUTH_FENIX_AUTH_KEY=<br />
SOCIAL_AUTH_FENIX_AUTH_SECRET=<br />
<br />
FREE_GOOGLE_OAUTH=off<br />
SOCIAL_AUTH_GOOGLE_OAUTH2_KEY=<br />
SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET=<br />
<br />
FREE_MS_OAUTH=off<br />
SOCIAL_AUTH_MICROSOFT_GRAPH_KEY=<br />
SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET=<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes in the API and the database you will need to make an upgrade to this version of the Proxy to 0.6.0. <br />
<br />
<span style="color:#FFFFFF; background:#F50000">If upgrading, remember to make a backup of the old version!</span><br />
<br />
== Pre-requisites ==<br />
<br />
Operating System:<br />
* Raspbian GNU/Linux 10 (buster)<br />
For the Proxy it self: <br />
* Python (3.7.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
pip3 install pyserial<br />
* Git (2.30.2+)<br />
sudo apt-get install git<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible with the Version of the FREE server).<br />
* For example '''mkdir Proxy_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_6_0.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_6_0.zip -d Proxy_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_060/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_060/<br />
<br />
Replace USER with the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4660WPA FREE 0.6.0 Instalation2022-10-23T23:28:43Z<p>Ist187344: /* Upgrade of previous instalation */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== Pre-requisites ==<br />
<br />
* Git (2.30.2+)<br />
sudo apt install git<br />
* Python (3.8.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide (as described next). These commands should be executed in the current FREE installation directory and will overwrite existing files with the exception of '''.env ''' and '''.sqlite''' files.<br />
<br />
=== Installation of Source Code ===<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the FREE directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip'''<br />
<br />
Activate the virtual environment created at the previous FREE installation:<br />
source free-env/bin/activate<br />
<br />
Install any new necessary packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
Also, make sure that the file '''.env''' contanies the following key (<span style="color:#FFFFFF; background:#F50000">if they are missing copy them from there</span>):<br />
<br />
FREE_FENIX_OAUTH=off<br />
SOCIAL_AUTH_FENIX_AUTH_KEY=<br />
SOCIAL_AUTH_FENIX_AUTH_SECRET=<br />
<br />
FREE_GOOGLE_OAUTH=off<br />
SOCIAL_AUTH_GOOGLE_OAUTH2_KEY=<br />
SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET=<br />
<br />
FREE_MS_OAUTH=off<br />
SOCIAL_AUTH_MICROSOFT_GRAPH_KEY=<br />
SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET=<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes in the API and the database you will need to make an upgrade to this version of the Proxy to 0.6.0. <br />
<br />
<span style="color:#FFFFFF; background:#F50000">If upgrading, remember to make a backup of the old version!</span><br />
<br />
== Pre-requisites ==<br />
<br />
Operating System:<br />
* Raspbian GNU/Linux 10 (buster)<br />
For the Proxy it self: <br />
* Python (3.7.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
pip3 install pyserial<br />
* Git (2.30.2+)<br />
sudo apt-get install git<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible with the Version of the FREE server).<br />
* For example '''mkdir Proxy_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_6_0.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_6_0.zip -d Proxy_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_060/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_060/<br />
<br />
Replace USER with the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4659WPA FREE 0.6.0 Instalation2022-10-20T21:38:01Z<p>Ist187344: /* Installation of Source Code */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== Pre-requisites ==<br />
<br />
* Git (2.30.2+)<br />
sudo apt install git<br />
* Python (3.8.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide (as described next). These commands should be executed in the current FREE instalation directory and will overwrite existing files withe the exception of '''.env ''' and '''.sqlite''' files.<br />
<br />
=== Installation of Source Code ===<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the FREE directory directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip'''<br />
<br />
Activate the virtual environment created ate the previous FREE instalation:<br />
source free-env/bin/activate<br />
<br />
Install any new necessary packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes in the API and the database you will need to make an upgrade to this version of the Proxy to 0.6.0. <br />
<br />
<span style="color:#FFFFFF; background:#F50000">If upgrading, remember to make a backup of the old version!</span><br />
<br />
== Pre-requisites ==<br />
<br />
Operating System:<br />
* Raspbian GNU/Linux 10 (buster)<br />
For the Proxy it self: <br />
* Python (3.7.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
pip3 install pyserial<br />
* Git (2.30.2+)<br />
sudo apt-get install git<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible with the Version of the FREE server).<br />
* For example '''mkdir Proxy_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_6_0.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_6_0.zip -d Proxy_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_060/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_060/<br />
<br />
Replace USER with the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4658WPA FREE 0.6.0 Instalation2022-10-20T21:35:45Z<p>Ist187344: /* Upgrade of previous instalation */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== Pre-requisites ==<br />
<br />
* Git (2.30.2+)<br />
sudo apt install git<br />
* Python (3.8.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide (as described next). These commands should be executed in the current FREE instalation directory and will overwrite existing files withe the exception of '''.env ''' and '''.sqlite''' files.<br />
<br />
=== Installation of Source Code ===<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the FREE directory directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d .'''<br />
<br />
Activate the virtual environment created ate the previous FREE instalation:<br />
source free-env/bin/activate<br />
<br />
Install any new necessary packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes in the API and the database you will need to make an upgrade to this version of the Proxy to 0.6.0. <br />
<br />
<span style="color:#FFFFFF; background:#F50000">If upgrading, remember to make a backup of the old version!</span><br />
<br />
== Pre-requisites ==<br />
<br />
Operating System:<br />
* Raspbian GNU/Linux 10 (buster)<br />
For the Proxy it self: <br />
* Python (3.7.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
pip3 install pyserial<br />
* Git (2.30.2+)<br />
sudo apt-get install git<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible with the Version of the FREE server).<br />
* For example '''mkdir Proxy_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_6_0.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_6_0.zip -d Proxy_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_060/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_060/<br />
<br />
Replace USER with the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4656WPA FREE 0.6.0 Instalation2022-10-19T20:42:39Z<p>Ist187344: /* Pre-requisites */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== Pre-requisites ==<br />
<br />
* Git (2.30.2+)<br />
sudo apt install git<br />
* Python (3.8.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes in the API and the database you will need to make an upgrade to this version of the Proxy to 0.6.0. <br />
<br />
<span style="color:#FFFFFF; background:#F50000">If upgrading, remember to make a backup of the old version!</span><br />
<br />
== Pre-requisites ==<br />
<br />
Operating System:<br />
* Raspbian GNU/Linux 10 (buster)<br />
For the Proxy it self: <br />
* Python (3.7.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
pip3 install pyserial<br />
* Git (2.30.2+)<br />
sudo apt-get install git<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible with the Version of the FREE server).<br />
* For example '''mkdir Proxy_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_6_0.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_6_0.zip -d Proxy_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_060/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_060/<br />
<br />
Replace USER with the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4655WPA FREE 0.6.0 Instalation2022-10-19T20:42:21Z<p>Ist187344: /* FREE Server */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== Pre-requisites ==<br />
<br />
For FREE_Web:<br />
* Git (2.30.2+)<br />
sudo apt install git<br />
* Python (3.8.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes in the API and the database you will need to make an upgrade to this version of the Proxy to 0.6.0. <br />
<br />
<span style="color:#FFFFFF; background:#F50000">If upgrading, remember to make a backup of the old version!</span><br />
<br />
== Pre-requisites ==<br />
<br />
Operating System:<br />
* Raspbian GNU/Linux 10 (buster)<br />
For the Proxy it self: <br />
* Python (3.7.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
pip3 install pyserial<br />
* Git (2.30.2+)<br />
sudo apt-get install git<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible with the Version of the FREE server).<br />
* For example '''mkdir Proxy_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_6_0.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_6_0.zip -d Proxy_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_060/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_060/<br />
<br />
Replace USER with the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4654WPA FREE 0.6.0 Instalation2022-10-17T17:28:10Z<p>Ist187344: /* Raspberry Pi Proxy */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes in the API and the database you will need to make an upgrade to this version of the Proxy to 0.6.0. <br />
<br />
<span style="color:#FFFFFF; background:#F50000">If upgrading, remember to make a backup of the old version!</span><br />
<br />
== Pre-requisites ==<br />
<br />
Operating System:<br />
* Raspbian GNU/Linux 10 (buster)<br />
For the Proxy it self: <br />
* Python (3.7.0+)<br />
sudo apt install python3<br />
sudo apt install python3-pip<br />
pip3 install pyserial<br />
* Git (2.30.2+)<br />
sudo apt-get install git<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible with the Version of the FREE server).<br />
* For example '''mkdir Proxy_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_6_0.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_6_0.zip -d Proxy_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_060/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_060/<br />
<br />
Replace USER with the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4653WPA FREE 0.6.0 Instalation2022-10-13T02:28:22Z<p>Ist187344: /* Proxy execution */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes in the API and the database you will need to make an upgrade to this version of the Proxy to 0.6.0. <br />
<br />
<span style="color:#FFFFFF; background:#F50000">If upgrading, remember to make a backup of the old version!</span><br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible with the Version of the FREE server).<br />
* For example '''mkdir Proxy_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_6_0.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_6_0.zip -d Proxy_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_060/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_060/<br />
<br />
Replace USER with the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4652WPA FREE 0.6.0 Instalation2022-10-13T02:27:46Z<p>Ist187344: /* Raspberry Pi Proxy */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes in the API and the database you will need to make an upgrade to this version of the Proxy to 0.6.0. <br />
<br />
<span style="color:#FFFFFF; background:#F50000">If upgrading, remember to make a backup of the old version!</span><br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible with the Version of the FREE server).<br />
* For example '''mkdir Proxy_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_6_0.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_6_0.zip -d Proxy_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_060/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4651WPA FREE 0.6.0 Instalation2022-10-13T02:26:36Z<p>Ist187344: /* Installation of Source Code */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes in the API and the database you will need to make an upgrade to this version of the Proxy to 0.6.0. <br />
<br />
If upgrading, remember to make a backup of the old version!<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible with the Version of the FREE server).<br />
* For example '''mkdir Proxy_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_6_0.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_6_0.zip -d Proxy_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_060/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4650WPA FREE 0.6.0 Instalation2022-10-13T02:25:17Z<p>Ist187344: /* Raspberry Pi Proxy */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes in the API and the database you will need to make an upgrade to this version of the Proxy to 0.6.0. <br />
<br />
If upgrading, remember to make a backup of the old version!<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_6_0.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4649WPA FREE 0.6.0 Instalation2022-10-13T02:16:40Z<p>Ist187344: /* How to add new experiments */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulum experiment you only will need to create on the database: Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small description can be made to give the user the same information about the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill in the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically, it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4648WPA FREE 0.6.0 Instalation2022-10-13T02:15:27Z<p>Ist187344: /* HTML/JS */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small discription can be made to give the user same information of the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
<!-- == HTML/JS == <br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".---></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4647WPA FREE 0.6.0 Instalation2022-10-13T02:13:39Z<p>Ist187344: /* HTML/JS */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small discription can be made to give the user same information of the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4646WPA FREE 0.6.0 Instalation2022-10-13T02:11:43Z<p>Ist187344: /* Apparatuses */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small discription can be made to give the user same information of the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
*Image-> by clicking the button '''Select file''' it is possible to put an image on the description page.<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=File:Aparatus_060_2.png&diff=4645File:Aparatus 060 2.png2022-10-13T02:08:48Z<p>Ist187344: </p>
<hr />
<div></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4644WPA FREE 0.6.0 Instalation2022-10-13T02:08:30Z<p>Ist187344: /* Apparatuses */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small discription can be made to give the user same information of the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060_2.png|center|1200px|thumb| Label. ]] </div><br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4643WPA FREE 0.6.0 Instalation2022-10-13T02:07:39Z<p>Ist187344: /* Apparatuses */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small discription can be made to give the user same information of the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> a small description of the apparatuses that the user is using;<br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connected to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4642WPA FREE 0.6.0 Instalation2022-10-13T02:06:43Z<p>Ist187344: /* Apparatuses */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small discription can be made to give the user same information of the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> <br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum), the ID to make sure of it is connect to the correct experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4641WPA FREE 0.6.0 Instalation2022-10-13T02:04:52Z<p>Ist187344: /* Apparatuses */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small discription can be made to give the user same information of the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Description-> <br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4640WPA FREE 0.6.0 Instalation2022-10-13T02:03:53Z<p>Ist187344: /* Apparatuses */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small discription can be made to give the user same information of the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer which will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as shown above, so you need to fill the following information:<br />
<br />
*Apparatus Type-> the one Apparatus Type that the apparatus is connected to;<br />
<br />
*Protocol-> the protocols related to that Experiment;<br />
<br />
*Location-> location of the experiment;<br />
<br />
*Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
*Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
*Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=File:Aparatus_060.png&diff=4639File:Aparatus 060.png2022-10-13T02:01:52Z<p>Ist187344: </p>
<hr />
<div></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4638WPA FREE 0.6.0 Instalation2022-10-13T02:00:32Z<p>Ist187344: /* Apparatuses */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small discription can be made to give the user same information of the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programming the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus_060.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4637WPA FREE 0.6.0 Instalation2022-10-13T01:59:43Z<p>Ist187344: /* Protocol */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Apparatus Type you may need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experiment can do different execution with different initial parameters, so one experiment can have multiple Protocols. <br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
*Apparatus Type -> select your Apparatus Type;<br />
<br />
*Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
*Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]);<br />
<br />
*Description-> a small discription can be made to give the user same information of the limitation of the protocol.<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=File:Protocol_page_admin.png&diff=4636File:Protocol page admin.png2022-10-13T01:55:12Z<p>Ist187344: </p>
<hr />
<div></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4635WPA FREE 0.6.0 Instalation2022-10-13T01:55:01Z<p>Ist187344: /* Protocol */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocol_page_admin.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4634WPA FREE 0.6.0 Instalation2022-10-13T01:53:57Z<p>Ist187344: /* Apparatus Type */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
*Name-> The name of the type of experiment;<br />
<br />
*Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
*Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
*Scientific area-> what area is the experience integrated;<br />
<br />
*Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4633WPA FREE 0.6.0 Instalation2022-10-13T01:53:19Z<p>Ist187344: /* Apparatus Type */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
Name-> The name of the type of experiment;<br />
<br />
Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4632WPA FREE 0.6.0 Instalation2022-10-13T01:52:55Z<p>Ist187344: /* Apparatus Type */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
Name-> The name of the type of experiment;<br />
<br />
Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
Description-> the description of the theory of the experiment, with the information that the user needs to understand it(formulas,links,...);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=File:Apparatus_Type.png&diff=4631File:Apparatus Type.png2022-10-13T01:49:29Z<p>Ist187344: </p>
<hr />
<div></div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4630WPA FREE 0.6.0 Instalation2022-10-13T01:48:55Z<p>Ist187344: /* Experiment */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Apparatus Type ==<br />
To create a new Apparatus Type on the database just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Apparatus_Type.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar page as the above, so you need to fill in the information about your experiment:<br />
<br />
Name-> The name of the type of experiment;<br />
<br />
Slug name-> the name of the HTML file of your experiment (explain on the section ...);<br />
<br />
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;<br />
<br />
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4629WPA FREE 0.6.0 Instalation2022-10-13T01:46:57Z<p>Ist187344: /* How to add new experiments */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
First, one has to have an experiment programmed in a controller of such an experiment and install the Proxy on a computer capable to communicate with the main server, the developer of the experiment needs to create the following entries on the database: Apparatus types, Apparatuses and Protocol.<br />
<br />
If your installing a Pendulo experiment you only will need to create on the database the Apparatuses and Protocol (if the protocol of the pendulum that you are installing is different from other ones). <br />
<br />
== Experiment ==<br />
To create a new Experiment on the database is just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Experiment.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:<br />
<br />
Name-> with the some of your experiment;<br />
<br />
Slug name-> the name of the html file of your experiment (explain on the section ...);<br />
<br />
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;<br />
<br />
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4628WPA FREE 0.6.0 Instalation2022-10-12T23:18:31Z<p>Ist187344: /* Configuration */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
The first one has to have an experiment programmed in a controller of such experiment, after that and after installing the Proxy on a computer capable to communicate to the main server, the developer of the experiment needs to create the following entries on the database: Protocol, Experiment and Apparatuses.<br />
<br />
== Experiment ==<br />
To create a new Experiment on the database is just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Experiment.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:<br />
<br />
Name-> with the some of your experiment;<br />
<br />
Slug name-> the name of the html file of your experiment (explain on the section ...);<br />
<br />
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;<br />
<br />
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4627WPA FREE 0.6.0 Instalation2022-10-12T23:17:38Z<p>Ist187344: /* Raspberry Pi Proxy */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
If the FREE server is configured to use HTTPS it needs to set the following parameter in the '''server_info.ini''' file:<br />
*HTTPS = True<br />
<br />
Also, the following values should be correctly set:<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done; <br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
==== ====<br />
<br />
<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
The first one has to have an experiment programmed in a controller of such experiment, after that and after installing the Proxy on a computer capable to communicate to the main server, the developer of the experiment needs to create the following entries on the database: Protocol, Experiment and Apparatuses.<br />
<br />
== Experiment ==<br />
To create a new Experiment on the database is just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Experiment.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:<br />
<br />
Name-> with the some of your experiment;<br />
<br />
Slug name-> the name of the html file of your experiment (explain on the section ...);<br />
<br />
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;<br />
<br />
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4626WPA FREE 0.6.0 Instalation2022-10-12T23:12:29Z<p>Ist187344: /* Apparatus proxy configuration */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
The first one has to have an experiment programmed in a controller of such experiment, after that and after installing the Proxy on a computer capable to communicate to the main server, the developer of the experiment needs to create the following entries on the database: Protocol, Experiment and Apparatuses.<br />
<br />
== Experiment ==<br />
To create a new Experiment on the database is just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Experiment.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:<br />
<br />
Name-> with the some of your experiment;<br />
<br />
Slug name-> the name of the html file of your experiment (explain on the section ...);<br />
<br />
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;<br />
<br />
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4625WPA FREE 0.6.0 Instalation2022-10-12T23:11:15Z<p>Ist187344: /* Upgrade of previous instalation */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">YOU must do a BACKUP of your BD and current FREE version installation!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
=== Apparatus proxy configuration ===<br />
If the FREE server is configured to use HTTPS it is necessary to also configure all apparatus proxies to use HTTPS.<br />
the version of the proxy should the the most recent change the '''server_info.ini''' file:<br />
HTTPS = True<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
The first one has to have an experiment programmed in a controller of such experiment, after that and after installing the Proxy on a computer capable to communicate to the main server, the developer of the experiment needs to create the following entries on the database: Protocol, Experiment and Apparatuses.<br />
<br />
== Experiment ==<br />
To create a new Experiment on the database is just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Experiment.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:<br />
<br />
Name-> with the some of your experiment;<br />
<br />
Slug name-> the name of the html file of your experiment (explain on the section ...);<br />
<br />
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;<br />
<br />
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4624WPA FREE 0.6.0 Instalation2022-10-12T23:08:29Z<p>Ist187344: /* Installation of New Database */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_6_0.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">FAZER backup as BD e codigo!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
=== Apparatus proxy configuration ===<br />
If the FREE server is configured to use HTTPS it is necessary to also configure all apparatus proxies to use HTTPS.<br />
the version of the proxy should the the most recent change the '''server_info.ini''' file:<br />
HTTPS = True<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
The first one has to have an experiment programmed in a controller of such experiment, after that and after installing the Proxy on a computer capable to communicate to the main server, the developer of the experiment needs to create the following entries on the database: Protocol, Experiment and Apparatuses.<br />
<br />
== Experiment ==<br />
To create a new Experiment on the database is just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Experiment.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:<br />
<br />
Name-> with the some of your experiment;<br />
<br />
Slug name-> the name of the html file of your experiment (explain on the section ...);<br />
<br />
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;<br />
<br />
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4623WPA FREE 0.6.0 Instalation2022-10-12T23:06:06Z<p>Ist187344: /* Apparatus proxy configuration = */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_5_1.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">FAZER backup as BD e codigo!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
=== Apparatus proxy configuration ===<br />
If the FREE server is configured to use HTTPS it is necessary to also configure all apparatus proxies to use HTTPS.<br />
the version of the proxy should the the most recent change the '''server_info.ini''' file:<br />
HTTPS = True<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
The first one has to have an experiment programmed in a controller of such experiment, after that and after installing the Proxy on a computer capable to communicate to the main server, the developer of the experiment needs to create the following entries on the database: Protocol, Experiment and Apparatuses.<br />
<br />
== Experiment ==<br />
To create a new Experiment on the database is just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Experiment.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:<br />
<br />
Name-> with the some of your experiment;<br />
<br />
Slug name-> the name of the html file of your experiment (explain on the section ...);<br />
<br />
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;<br />
<br />
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4622WPA FREE 0.6.0 Instalation2022-10-12T23:05:29Z<p>Ist187344: /* HTTPS cfigurations */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS configurations ==<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_5_1.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">FAZER backup as BD e codigo!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
=== Apparatus proxy configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also configure all apparatus proxies to use HTTPS.<br />
the version of the proxy should the the most recent change the '''server_info.ini''' file:<br />
HTTPS = True<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
The first one has to have an experiment programmed in a controller of such experiment, after that and after installing the Proxy on a computer capable to communicate to the main server, the developer of the experiment needs to create the following entries on the database: Protocol, Experiment and Apparatuses.<br />
<br />
== Experiment ==<br />
To create a new Experiment on the database is just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Experiment.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:<br />
<br />
Name-> with the some of your experiment;<br />
<br />
Slug name-> the name of the html file of your experiment (explain on the section ...);<br />
<br />
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;<br />
<br />
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4621WPA FREE 0.6.0 Instalation2022-10-12T23:04:50Z<p>Ist187344: /* janus configuration */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS cfigurations ==<br />
TODO!!!<br />
<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_5_1.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">FAZER backup as BD e codigo!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
=== Apparatus proxy configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also configure all apparatus proxies to use HTTPS.<br />
the version of the proxy should the the most recent change the '''server_info.ini''' file:<br />
HTTPS = True<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
The first one has to have an experiment programmed in a controller of such experiment, after that and after installing the Proxy on a computer capable to communicate to the main server, the developer of the experiment needs to create the following entries on the database: Protocol, Experiment and Apparatuses.<br />
<br />
== Experiment ==<br />
To create a new Experiment on the database is just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Experiment.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:<br />
<br />
Name-> with the some of your experiment;<br />
<br />
Slug name-> the name of the html file of your experiment (explain on the section ...);<br />
<br />
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;<br />
<br />
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4620WPA FREE 0.6.0 Instalation2022-10-12T23:04:34Z<p>Ist187344: /* Upgrade of previous instalation */</p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS cfigurations ==<br />
TODO!!!<br />
<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_5_1.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">FAZER backup as BD e codigo!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
=== Apparatus proxy configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also configure all apparatus proxies to use HTTPS.<br />
the version of the proxy should the the most recent change the '''server_info.ini''' file:<br />
HTTPS = True<br />
<br />
=== janus configuration ===<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
The first one has to have an experiment programmed in a controller of such experiment, after that and after installing the Proxy on a computer capable to communicate to the main server, the developer of the experiment needs to create the following entries on the database: Protocol, Experiment and Apparatuses.<br />
<br />
== Experiment ==<br />
To create a new Experiment on the database is just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Experiment.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:<br />
<br />
Name-> with the some of your experiment;<br />
<br />
Slug name-> the name of the html file of your experiment (explain on the section ...);<br />
<br />
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;<br />
<br />
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4619WPA FREE 0.6.0 Instalation2022-10-12T22:59:45Z<p>Ist187344: </p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS cfigurations ==<br />
TODO!!!<br />
<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_5_1.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">Background color|red|FAZER backup as BD e codigo!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
=== Apparatus proxy configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also configure all apparatus proxies to use HTTPS.<br />
the version of the proxy should the the most recent change the '''server_info.ini''' file:<br />
HTTPS = True<br />
<br />
=== janus configuration ===<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.<br />
<br />
= How to add new experiments =<br />
<br />
The first one has to have an experiment programmed in a controller of such experiment, after that and after installing the Proxy on a computer capable to communicate to the main server, the developer of the experiment needs to create the following entries on the database: Protocol, Experiment and Apparatuses.<br />
<br />
== Experiment ==<br />
To create a new Experiment on the database is just click on the "+" on the left of the pag.<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Experiment.png|center|1200px|thumb| Label. ]] </div><br />
By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:<br />
<br />
Name-> with the some of your experiment;<br />
<br />
Slug name-> the name of the html file of your experiment (explain on the section ...);<br />
<br />
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;<br />
<br />
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);<br />
<br />
Scientific area-> what area is the experience integrated;<br />
<br />
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).<br />
<br />
== Protocol ==<br />
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.<br />
<br />
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a page like the one shown above that you will need to file:<br />
<br />
Experiment-> select your exeperiment;<br />
<br />
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;<br />
<br />
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).<br />
<br />
== Apparatuses ==<br />
<br />
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div><br />
<br />
You will have a similar page as a shown above, so you need to fill the follow information:<br />
<br />
Experiment-> the one experiment that apparatus is connected to;<br />
<br />
Protocol-> the protocols related that Experiment;<br />
<br />
Location-> Location of the experiment;<br />
<br />
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;<br />
<br />
Owner-> the person responsible for the maintenance of the experiment;<br />
<br />
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).<br />
<br />
<br />
== HTML/JS ==<br />
<br />
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div><br />
<br />
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div><br />
<br />
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.<br />
<br />
<div style="display: inline; width: 520px; float: center;"><br />
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div><br />
<br />
<br />
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment. <br />
<br />
<br />
<div style="display: flex;align-items: center;justify-content: center;"><ul><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li><br />
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]] </li> <br />
</ul><br />
</div><br />
<br />
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".<br />
<br />
<!--= Video Streaming = --!><br />
<br />
Intro<br />
<br />
== Video capture and stream ==<br />
<br />
On the raspberry it is running ....<br />
<br />
== Janus stream server ==<br />
<br />
== Accessing the stream ==<br />
<br />
The experiment stream can be access by a html ....<br />
<br />
or by using VLC, allowing to grab the video</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4618WPA FREE 0.6.0 Instalation2022-10-10T21:54:37Z<p>Ist187344: </p>
<hr />
<div>= Janus Server=<br />
<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
== HTTPS cfigurations ==<br />
TODO!!!<br />
<br />
<br />
= FREE Server=<br />
<br />
== New instalation == <br />
<br />
=== Installation of Source Code ===<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
=== Installation of New Database ===<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_5_1.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
=== Configuration ===<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
<br />
=== FREE server execution ===<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
=== Copy of Pendulum Information ===<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
=== Configuration of Video ===<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
<br />
== Upgrade of previous instalation == <br />
<br />
<span style="color:#FFFFFF; background:#F50000">Background color|red|FAZER backup as BD e codigo!</span><br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
== HTTPS configuration ==<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
=== Cerificates ===<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
=== Server execution ===<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
=== Apparatus proxy configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also configure all apparatus proxies to use HTTPS.<br />
the version of the proxy should the the most recent change the '''server_info.ini''' file:<br />
HTTPS = True<br />
<br />
=== janus configuration ===<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
== External Authentication ==<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
=== GOOGLE ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
=== Microsoft ===<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4611WPA FREE 0.6.0 Instalation2022-10-09T20:04:19Z<p>Ist187344: /* Installation of Source Code */</p>
<hr />
<div>= Janus Server=<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
= FREE Server=<br />
<br />
<br />
== Update guide == <br />
<br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
=== HTTPS configuration ===<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
==== Cerificates ====<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
==== Server execution ====<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
==== Apparatus proxy configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also configure all apparatus proxies to use HTTPS.<br />
the version of the proxy should the the most recent change the '''server_info.ini''' file:<br />
HTTPS = True<br />
<br />
==== janus configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
=== External Authentication ===<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
==== GOOGLE ====<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
==== Microsoft ====<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_060 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
== Installation of New Database ==<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_5_1.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
== Configuration ==<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
== FREE server execution ==<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
== Copy of Pendulum Information ==<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
== Configuration of Video ==<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4610WPA FREE 0.6.0 Instalation2022-10-09T20:03:41Z<p>Ist187344: </p>
<hr />
<div>= Janus Server=<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
= FREE Server=<br />
<br />
<br />
== Update guide == <br />
<br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
=== HTTPS configuration ===<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
==== Cerificates ====<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
==== Server execution ====<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
==== Apparatus proxy configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also configure all apparatus proxies to use HTTPS.<br />
the version of the proxy should the the most recent change the '''server_info.ini''' file:<br />
HTTPS = True<br />
<br />
==== janus configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
=== External Authentication ===<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
==== GOOGLE ====<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
==== Microsoft ====<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file <span style="color:#FFFFFF; background:#F50000">'''The secret Value dissapears after closing this page '''</span><br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
== Installation of New Database ==<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_5_1.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
== Configuration ==<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
== FREE server execution ==<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
== Copy of Pendulum Information ==<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
== Configuration of Video ==<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4609WPA FREE 0.6.0 Instalation2022-10-09T19:56:16Z<p>Ist187344: /* Microsoft */</p>
<hr />
<div>= Janus Server=<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
= FREE Server=<br />
<br />
<br />
== Update guide == <br />
<br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
=== HTTPS configuration ===<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
==== Cerificates ====<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
==== Server execution ====<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
==== Apparatus proxy configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also configure all apparatus proxies to use HTTPS.<br />
the version of the proxy should the the most recent change the '''server_info.ini''' file:<br />
HTTPS = True<br />
<br />
==== janus configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
=== External Authentication ===<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
==== GOOGLE ====<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
==== Microsoft ====<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
***Select a platform - Web<br />
***to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file<br />
{{warning}} '''The secret Value dissapears after closing this page'''<br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
== Installation of New Database ==<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_5_1.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
== Configuration ==<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
== FREE server execution ==<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
== Copy of Pendulum Information ==<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
== Configuration of Video ==<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.</div>Ist187344http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=WPA_FREE_0.6.0_Instalation&diff=4608WPA FREE 0.6.0 Instalation2022-10-09T19:53:17Z<p>Ist187344: /* Update guide */</p>
<hr />
<div>= Janus Server=<br />
== Update ==<br />
If the Janus server is already running with the FREE 0.3.0 version it is only necessary to guarantee that the admin secret is defined.<br />
<br />
Edit the '''janus.plugin.streaming.jcfg ''' file in the '''/var/snap/janus-gateway/common/etc ''' directory and guarantee that the '''admin_key ''' is defined:<br />
<br />
<br />
general :<br />
{<br />
admin_key = "elab1";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
<br />
<br />
<br />
== Installation ==<br />
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:<br />
<br />
sudo apt install snap<br />
<br />
sudo apt install snapd<br />
<br />
sudo snap install janus-gateway<br />
<br />
===Configuration===<br />
<br />
The configuration of Janus is done editing a file in the '''/var/snap/janus-gateway/common/etc''' directory <br />
<br />
Edit the '''janus.plugin.streaming.jcfg''' file<br />
<br />
Define an administration secret by uncommenting the line 104) or the one that contains the '''admin_key''' string= and changing the string to a suitable value.<br />
<br />
Define the possible ports that Janus will use to receive the video stream from the pendulum by uncomment line 106) or the one containing the string '''rtp_port_range'''= and define the available ports (for example to 6000-7000).<br />
<br />
Delete the sample pre-configure endpoints by deleting all lines from line 124. <br />
<br />
Restart Janus by running the command:<br />
<br />
sudo snap restart janus-gateway<br />
<br />
=== Example of Janus Configuration ===<br />
After the configuration of a newly installed Janus Server the '''janus.plugin.streaming.jcfg''' should only containg the following`<br />
<br />
general :<br />
{<br />
admin_key = "super_pass";<br />
rtp_port_range = "6000-7000";<br />
};<br />
<br />
= FREE Server=<br />
<br />
<br />
== Update guide == <br />
<br />
<br />
Follow the first two steps of the installation guide. The commands already contain overwrite switches so that the old version will be overwritten, while '''.env ''' and '''.sqlite''' files won't be touched.<br />
<br />
A new release might contain changes to the database structure. If you are preserving your database from a previous version, run<br />
<br />
python manage.py migrate<br />
<br />
=== HTTPS configuration ===<br />
IN order to allow remote OAuth authentication (Google or Microsoft) it is necessary to activate HTTPS.<br />
this requires the installation of SSL certificates and the execution of daphne in a with a different configuration:<br />
<br />
==== Cerificates ====<br />
FREE should execute with certificates produced by the organization, can use self-signed cerificates temporarly.<br />
Certificates cshould be generated by the network administrator and installed with the suitable names (free_private.key and free_public.crt) in the '''freeweb/certificates/''' directory<br />
For testing purposed it is also possible to create a self signed certificate by running the following command on the FREE server:<br />
<br />
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout freeweb/certificates/free_private.key -out freeweb/certificates/free_public.crt<br />
<br />
<br />
==== Server execution ====<br />
After the creation or installation of the certificate the server should be executed as follows<br />
<br />
daphne -e ssl:8000:privateKey=freeweb/certificates/free_private.key:certKey=freeweb/certificates/free_public.crt freeweb.asgi:application<br />
<br />
If running with this configuration, users should use the following url to access FREE:<br />
<br />
https://hostname.some_domain:8000/<br />
<br />
<br />
<br />
==== Apparatus proxy configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also configure all apparatus proxies to use HTTPS.<br />
the version of the proxy should the the most recent change the '''server_info.ini''' file:<br />
HTTPS = True<br />
<br />
==== janus configuration ====<br />
If the FREE server is configured to use HTTPS it is necessary to also further configure janus to also use HTTPS.<br />
It is necessary to create the certificate similar rto the one used in daphne. Unfortunetly the certificate can not be created in the host (self-signed certificate), '''it must be created by the network administrator of the domain'''.<br />
<br />
This certificate should be placed in a system directory before editing the '''janus.transport.http.jcfg''' configuration file:<br />
*Disable HTTP: http = false<br />
*Enable HTTPS: https = true<br />
*define the HTTPS port: secure_port = 8088 - Set the certificate directory:<br />
**cert_pem = "/some_system_directory/janus_public.crt"<br />
**cert_key = "/some_system_directory/janus_private.key" In the .env file it is also necessary to update the protocolo of janu from http to https:<br />
*JANUS_SERVER_ADDRESS="https://janus-server-address:8088/janus"<br />
<br />
=== External Authentication ===<br />
FREE now allows the use of external services from Google and Microsoft for user authentication.<br />
To configure these services it is necessary to have HTTPS working and have a fixed public numeric address (or DNS name) for the FREE installation (for instance https://free.some-university.edu:8000/) and follow the next instructions.<br />
==== GOOGLE ====<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/google.html#google-oauth2 / https://developers.google.com/identity/protocols/oauth2?csw=1#Registering:<br />
<br />
*access the Google API Console: https://console.developers.google.com/<br />
*Create a new project<br />
*Credentials<br />
**Create Credentials -> OAuth client ID<br />
*Configure Consent Screen<br />
**Select External User Type<br />
**Fill The App information screen (app name, User support email, Developer contact information)<br />
**Select scopes (may be empty)<br />
**Add a test user<br />
*Credentials<br />
**Create credentials -> OAuth client ID<br />
**Application Type - Web application<br />
**Define name <br />
**Add Authorized redirect URIs<br />
***append '''/complete/google-oauth2/''' to the installation URL (example: `https://free.some-university.edu:8000/complete/google-oauth2/`)<br />
**Create<br />
*Download json or save the Client ID and Client Secret<br />
*Edit .env file:<br />
**activate '''FREE_GOOGLE_OAUTH''':<br />
***FREE_GOOGLE_OAUTH=true<br />
**copy the value of '''Your Client ID''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_KEY'''<br />
**copy '''Client Secret''' to '''SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET'''<br />
**restart the FREE installation<br />
==== Microsoft ====<br />
Follow the instructions in https://python-social-auth.readthedocs.io/en/latest/backends/microsoftgraph.html.html#google-oauth2 / https://learn.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app:<br />
<br />
*Sign in to the Azure portal: https://portal.azure.com/<br />
*Manage Azure Active Directory - View<br />
*Select App registrations on the menu<br />
*New registration<br />
**Define the name of teh application<br />
**Select Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)<br />
**Define the redirect URI<br />
- Select a platform - Web<br />
- to create the URI, append '''/complete/microsoft-graph/''' to the installation URL (exampe: `https://free.some-university.edu:8000/complete/microsoft-graph/`)<br />
**Register<br />
**Save the '''Application (client) ID'''<br />
***this value should be put in the .env file<br />
**Select Certificates and secrets on the menu<br />
***New client secret<br />
***Add<br />
**Save the '''Value'''<br />
***this value should be put in the .env file<br />
{{warning}} '''The secret Value dissapears after closing this page'''<br />
*Edit .env file:<br />
**activate '''FREE_MS_OAUTH''':<br />
***FREE_MS_OAUTH=true<br />
**copy the value of '''Application (client) ID''' to '''SSOCIAL_AUTH_MICROSOFT_GRAPH_KEY'''<br />
**copy the value of '''Secret ID''' to '''SOCIAL_AUTH_MICROSOFT_GRAPH_SECRET'''<br />
**restart the FREE installation<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of FREE.<br />
* For example '''mkdir wpa_free_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/FREE_Web_0_6_0.zip<br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip FREE_Web_0_6_0.zip -d wpa_free_060/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd wpa_free_060/ '''<br />
<br />
Create a new python virtual environment, it should be called '''free-env''':<br />
virtualenv -p python3 free-env<br />
<br />
Activate the virtual environment:<br />
source free-env/bin/activate<br />
<br />
You should now see a prefix of (free-env) in your command line. <br />
<br />
After that, install the dependent packages:<br />
pip install -r REQUIREMENTS.txt<br />
<br />
== Installation of New Database ==<br />
Since the data model was change it is necessary to download and install a new empty database.<br />
<br />
This database is configured with two users and the WPA pendulum apparatus types and protocols.<br />
<br />
Download the database and put it into the project root:<br />
<br />
<br />
wget https://github.com/e-lab-FREE/FREE_Web/releases/latest/download/db_template_0_5_1.sqlite3 -O db.sqlite3<br />
<br />
This database contains a Pendulum experiment, as well as two example user accounts.<br />
* wp-admin - superuser account with access to admin interface<br />
* wp-guest - guest account<br />
Both users have a '''temporary123 ''' password. <br />
<br />
This password should be changed through admin interface.<br />
<br />
== Configuration ==<br />
<br />
The application is configured using environment variables. <br />
<br />
You can set them using the '''/freeweb/.env ''' file. <br />
<br />
There is a '''.env-template ''' file in the '''~freeweb ''' folder, that can be renamed to '''.env ''' :<br />
mv freeweb/.env-template freeweb/.env<br />
<br />
It is necessary to define correct values for the following variables. Some of these values can be copied form the previous version .env file.<br />
* PROJECT_NAME, PROJECT_ACRONYMUM, SITE_NAME - Texts that will appear in the page header<br />
* TIME_ZONE - Time zone setting for that app in '''TZ database name''' from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List.<br />
* FREE_PRODUCTION - set to '''on ''' to enable production mode (disables sensitive error messages etc.)<br />
* FREE_REVERSE_PROXY - set to '''on ''' to be able to run the FREE behind a reverse proxy<br />
* FREE_SECRET - String used in hashing function. Set either to a random string of your choice, or generate one here: https://djecrety.ir<br />
* FREE_ALLOWED_HOSTS - comma separated list of domain names/addresses; only requests to these hosts will be processed by the application. This is necessary to prevent HTTP Host header attacks.<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
== FREE server execution ==<br />
to start the FREE server run the following command:<br />
daphne freeweb.asgi:application<br />
<br />
By default, the webserver will be available at port 8000. To change the port, pass -p <portnumber> parameter to the daphne command. <br />
You can also force binding to specific address by -b <address>.<br />
<br />
<br />
== Copy of Pendulum Information ==<br />
Since a new database is now being used, it is necessary to copy the various apparatus into the new database.<br />
<br />
If running both version at the same time, one of them, should be executed on a different port<br />
<br />
On the administration page of the you should copy some of the information from the old database to the new.<br />
<br />
here is the list and description of the various fields:<br />
* Apparatus Type : Pendulum<br />
* Location - description of the physical place of the pendulum<br />
* Description - specific characteristics of the pendulum<br />
* Secret - a random string specific to each pendulum<br />
* Owner - Name of the person responsible for the operation and maintenance of the pendulum<br />
* Timeout - leave 60 <br />
* Configuration - copy the following json and modify accordingly the '''id ''' and '''ports_restrict ''' to suitable values:<br />
{<br />
"id": "WP_LIS_IST", <br />
"serial_port": {<br />
"ports_restrict": ["/dev/ttyS0"], <br />
"baud": "115200", "numbits": "8", "stopbits": "1", "partitybits": "0", <br />
"listening_timeout": "100000", "death_timeout": "10000000"<br />
}<br />
}<br />
* video configuration - either copy the value from the old database or leave empty<br />
<br />
== Configuration of Video ==<br />
In order to have video stream from the Raspberry Pi to the server and then to the browser it is necessary to verify the following data on the '''.env ''' file:<br />
* JANUS_SERVER_ADDRESS, JANUS_STREAM_KEY - Configuration of Janus video streaming server.<br />
<br />
After this configuration is correct it is necessary to assign each pendulum one video stream so that video can go from the pendulum to the server and to the browser.<br />
<br />
Open the '''Video Config ''' menu:<br />
<br />
[[File:Video-config-menu.png]]<br />
<br />
On the new page, a table will contain all the configured pendulums and the associated video configuration:<br />
<br />
[[File:Video_Config_table.png]]<br />
<br />
In the previous example the first apparatus was automatically configure from the previous identifier copied from the old database.<br />
<br />
The second apparatus does not have the video stream configure. To create a video stream and update the database click '''Configure Video '''<br />
<br />
<br />
[[File:Video Config assign stream.png]]<br />
<br />
On the new page it is necessary to click on the '''Assign Stream ''' button.<br />
<br />
If the Janus server is well configure the new page will show the Video Stream Configuration and the data to be copied to the Raspberry Pi:<br />
<br />
[[File:Video Config RP data.png]]<br />
<br />
= Raspberry Pi Proxy =<br />
<br />
Due to changes on the API and the database you will need to make a upgrade to this version of the Proxy.<br />
<br />
== Installation of Source Code ==<br />
<br />
Create a new folder for the new version of Proxy (compatible to the Version of the FREE server).<br />
* For example '''mkdir Proxy_051 '''<br />
<br />
Download the Zip file containing the code<br />
wget https://github.com/e-lab-FREE/RPi_Proxy/releases/latest/download/RPi_Proxy_0_5_1.zip <br />
<br />
Unzip the downloaded file into the new directory.<br />
* For example '''unzip RPi_Proxy_0_5_1.zip -d Proxy_051/ '''<br />
<br />
Enter the directory containing the code.<br />
* For example '''cd Proxy_051/ '''<br />
<br />
After that, install the dependent package:<br />
<br />
For the Proxy:<br />
<br />
sudo apt install python3<br />
<br />
sudo apt install python3-pip<br />
<br />
pip3 install pyserial<br />
<br />
And for streaming the video (for '''gstreamer '''):<br />
<br />
sudo apt-get install gstreamer1.0-tools<br />
<br />
sudo apt-get install gstreamer1.0-plugins-good<br />
<br />
sudo apt-get install gstreamer1.0-plugins-bad<br />
<br />
sudo apt-get install gstreamer1.0-plugins-ugly<br />
<br />
sudo apt-get install gstreamer1.0-plugins-base<br />
<br />
<br />
For '''ffmpeg ''':<br />
sudo apt-get install ffmpeg<br />
<br />
== Configuration ==<br />
<br />
=== Video Streaming Configuration ===<br />
<br />
In order to configure the video streaming process on the Raspberry pi edit the '''video-stream.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
* video_server video_port apparatus_location apparatus_name apparatus_id<br />
** copy this information from the Video Conf administration page on the FREE server<br />
<br />
* usb_camera video_width video_height video_frame<br />
** find the suitable values depending on the network resources and the information provided by the following commands:<br />
v4l2-ctl --list-devices<br />
v4l2-ctl -d /dev/videoXXXXX --list-formats-ext<br />
<br />
In order to debug and verify the video configuration it is possible to execute the '''video-stream.sh ''' command.<br />
<br />
<br />
=== Proxy Configuration ===<br />
<br />
<br />
In order to configure the Proxy on Raspberry pi edit the '''server_info.ini ''' file.<br />
<br />
The following values should be correctly set:<br />
<br />
*SERVER : The IP address or domain of your FREE_Web server;<br />
*PORT : The port of the server by default is the 8000;<br />
*DEBUG: This can be change to off, after this installation is correctly done;<br />
*APPARATUS_ID, SECRET: This parameters can be seen on the admin of the FREE_Web server;<br />
On the apparatus configure page, you will be able to see the APPARATUS_ID on the URL of the page (indicated with the arrow) and the SECRET on the page it self:<br />
<br />
[[File:Info for Proxy.png|1220px]]<br />
<br />
== Proxy execution ==<br />
<br />
In order to debug the various video and apparatus control configuration, the two scripts (video-stream.sh and main.py) can be executed independently on the command line.<br />
<br />
After all the parameters are correct, both programs can be executed simultaneously using the following command:<br />
<br />
nohup sh start-wp.sh > /dev/null 2>&1 &<br />
<br />
This command runs until the raspberry pi is rebooted.<br />
<br />
To make the Proxy run when the Raspberry pi reboots it is necessary to edit the '''/etc/rc.local ''' and add the following lines before the '''exit 0 ''':<br />
sleep 60<br />
cd /INSTALLATION_FOLDER/PATH<br />
su USER -c "sh start-wp.sh &"<br />
<br />
<br />
Replace /INSTALLATION_FOLDER/PATH with the output of the following command:<br />
readlink -f Proxy_051/<br />
<br />
Replace USER the the actual user owner of the proxy.</div>Ist187344