Difference between revisions of "FREE - Framework for Remote Experiments in Education"

From wwwelab
Jump to navigation Jump to search
 
(73 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
 +
= Current releases =
 +
== Version  0.5.0 ==
 +
[[WPA_-_FREE_installation_instructions | Alpha release]] Major bugs corrected, adding video configuration and checking experiments status.
 +
 +
== Version  0.3.0 ==
 +
([http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=FREE_(0.3.0) Bogotá release]) Test version including video server.
  
 
= System architecture =
 
= System architecture =
  
−
In the defined architecture, users communicate with a central server, named Main_Server, through web pages.
+
In the defined architecture, users communicate with a central server, named FREE_Web, through web pages.
  
−
This Main_Server server contains the database with the results of the experiments already carried out and the experimental executions and experimental time schedules requested by users through the web interface. The Main_Server server is also responsible for mediating real-time communication between users and experiences, exchanging configuration requests, status requests and responses and generated data between users actively connected to the apparatus and apparatus.
+
This FREE_Web server contains the database with the results of the experiments already carried out and the experimental executions and experimental time schedules requested by users through the web interface. The FREE_Web server is also responsible for mediating real-time communication between users and experiences, exchanging configuration requests, status requests and responses and generated data between users actively connected to the apparatus and apparatus.
  
 
<div style="display: inline; width: 520px; float: center;">
 
<div style="display: inline; width: 520px; float: center;">
Line 11: Line 18:
  
  
−
Along with the experimental apparatus, a computer, named RPI_Server, acts as an interface between the controller of the experimental apparatus and the Main_Server server. Each RPI_Server is turned on and represents one, and only one, experimental apparatus. It is important to note that this component was named RPI_Server because it is, in principle, implemented through the use of a Raspberry Pi, a computer that has communication modules necessary to communicate natively with the microcontrollers usually used to control the devices. However, this component can be implemented in other ways and it is only strictly necessary that it communicates according to the protocol - e.g. a virtual experience with data generated through a numerical model can implement this component with a normal computer without specialized interfaces. The communication between all components is done over the internet, with the exception of the communication between the RPi_Server and the experimental apparatus, where communications are made through the protocol that the controller of the apparatus implements.
+
Along with the experimental apparatus, a computer, named RPi_Proxy, acts as an interface between the controller of the experimental apparatus and the FREE_Web server. Each RPi_Proxy is turned on and represents one, and only one, experimental apparatus. It is important to note that this component was named RPi_Proxy because it is, in principle, implemented through the use of a Raspberry Pi, a computer that has communication modules necessary to communicate natively with the microcontrollers usually used to control the devices. However, this component can be implemented in other ways and it is only strictly necessary that it communicates according to the protocol - e.g. a virtual experience with data generated through a numerical model can implement this component with a normal computer without specialized interfaces. The communication between all components is done over the internet, with the exception of the communication between the RPi_Proxy and the experimental apparatus, where communications are made through the protocol that the controller of the apparatus implements.
  
 
= Communication protocol =
 
= Communication protocol =
−
To start the connection between the RPi_Server and the Main_Server, when the RPi_Server is turned on, it tries to connect to the Main_Server server. If not successful, it should wait 10 seconds and try again until it manages to make the connection.  
+
To start the connection between the RPi_Proxy and the FREE_Web, when the RPi_Proxy is turned on, it tries to connect to the FREE_Web server. If not successful, it should wait 10 seconds and try again until it manages to make the connection.  
  
 
<div style="display: flex;align-items: center;justify-content: center;"><ul>
 
<div style="display: flex;align-items: center;justify-content: center;"><ul>
Line 22: Line 29:
 
</div>
 
</div>
  
−
When the connection is successfully established, the server RPi_Server sends to Main_Servero its ID and Secret. These two variables are then checked on the Main_Server server side as a simple way to authenticate that the connection is coming from an authorized machine. In case of successful verification, the Main_Server responds to the RPi_Server with a message encoded in JSON format that describes the experience to be served by the RPi_Server. This configuration message allows the same RPi_Server program to be adaptable to different experiences with, for example, different numbers of actuators and sensors without changes to the source code. Based on the information contained in this message, the name of the config_file, the RPi_Server looks for the experiment at the indicated address and tries to establish communication with the controller of the experimental apparatus. In order to decouple the RPi_Server from the different communication protocols that each experimental controller may implement, the RPi_Server must communicate with the devices calling a set of standard methods whose implementation must be adapted to suit the target controller.
+
When the connection is successfully established, the server RPi_Proxy sends to FREE_Webo its ID and Secret. These two variables are then checked on the FREE_Web server side as a simple way to authenticate that the connection is coming from an authorized machine. In case of successful verification, the FREE_Web responds to the RPi_Proxy with a message encoded in JSON format that describes the experience to be served by the RPi_Proxy. This configuration message allows the same RPi_Proxy program to be adaptable to different experiences with, for example, different numbers of actuators and sensors without changes to the source code. Based on the information contained in this message, the name of the config_file, the RPi_Proxy looks for the experiment at the indicated address and tries to establish communication with the controller of the experimental apparatus. In order to decouple the RPi_Proxy from the different communication protocols that each experimental controller may implement, the RPi_Proxy must communicate with the devices calling a set of standard methods whose implementation must be adapted to suit the target controller.
  
  
Line 28: Line 35:
  
  
−
After this initial configuration, the connection of the RPi_Serverao Main_Serverand the RPi_Serverto the controller of the experiment, the system is ready for the exchange of messages between the components. The messages exchanged between components were defined following an everything is a string philosophy. Thus, the messages exchanged are JSON-formatted strings in which the JSON fields encode the message to be transmitted. The type of each message is identified by the msg_id field of the message JSON, and for the message sent, the reply, if any, must have reply_id with the same value. In order to be adaptable to various types of experimental apparatus, the system does not store apparatus status information so it is possible to send any of the defined messages at any time. The correct sequencing of messages for the successful operation of the apparatus is the responsibility of other user-controlled software layers.
+
After this initial configuration, the connection of the RPi_Proxyao FREE_Weband the RPi_Proxyto the controller of the experiment, the system is ready for the exchange of messages between the components. The messages exchanged between components were defined following an everything is a string philosophy. Thus, the messages exchanged are JSON-formatted strings in which the JSON fields encode the message to be transmitted. The type of each message is identified by the msg_id field of the message JSON, and for the message sent, the reply, if any, must have reply_id with the same value. In order to be adaptable to various types of experimental apparatus, the system does not store apparatus status information so it is possible to send any of the defined messages at any time. The correct sequencing of messages for the successful operation of the apparatus is the responsibility of other user-controlled software layers.
  
 +
= Functions documentation =
  
−
== Sent messages (Main_Server → RPi_Server)  ==
+
== FREE_Web ==
  
−
=== Message 1 ===
+
== RPi_Proxy ==
  
−
Message sent to transmit the experimental apparatus configuration parameters to the RPi_Server. Parameters must be sent in the config_file field formatted as JSON. The configuration JSON must be adapted to each type of apparatus and to the source code of the interface used.
+
=== interface.py(RPi_Proxy) ===
  
−
<pre id="json">
+
= How to install =
−
{
 
−
    "msg_id": "1",
 
−
    "config_file":config_json
 
−
}
 
−
</pre>
 
  
−
In response, Main_Serverde will see one of the following messages, indicating the initialization state of the apparatus. In case of successful initialization, the response should be as following:
+
== Pre-requisites ==
  
−
<pre id="json">
+
=== Main server ===
−
{
+
For FREE_Web:
−
    "reply_id": "1",
+
* Git (2.30.2+)
−
    "status": "Experiment initialized OK"
+
sudo apt install git
−
}
+
* Python (3.7.0+)
−
</pre>
+
sudo apt install python3
 +
sudo apt install python3-pip
  
−
In case of error, the response is:  
+
There are need more installation but on the [https://github.com/e-lab-FREE/FREE_Web repository], you will have a "REQUIREMENTS.txt" that you will run to install them.
 +
  
−
<pre id="json">
+
For the video server:
−
{
+
* Janus
−
    "reply_id": "1",
+
sudo apt install snap​
−
    "status": "Experiment initialized NOT OK",
+
sudo apt install snapd​
−
    "error": error_description
+
sudo snap install janus-gateway​
−
}
 
−
</pre>
 
  
−
In this case, the error field can be used to describe in detail the error that occurred on startup, e.g. the experience on the configured serial port could not be found.
+
For tools to test:
 +
* Flask
 +
pip3 install flask​
 +
pip3 install flask_cors​
  
−
=== Message 2 ===
+
=== Rasberry Pi ===
 +
Operating System:
 +
* Raspbian GNU/Linux 10 (buster)
 +
For the Proxy it self:
 +
* Python (3.7.0+)
 +
sudo apt install python3
 +
sudo apt install python3-pip
 +
sudo apt install python3-venv​
 +
pip3 install pyserial
 +
* Git (2.30.2+)
 +
sudo  apt-get install git
  
−
Message sent to transmit the configuration parameters of an experimental run defined by a user. The parameters of the experimental run must be sent in the config_params field encoded in JSON format. The JSON encoding the configuration parameters must be adapted to each type of apparatus and to the interface source code used.  
+
For the video stream:
 +
sudo apt-get install gstreamer1.0-tools
 +
sudo apt-get install gstreamer1.0-plugins-good
 +
sudo apt-get install gstreamer1.0-plugins-bad
 +
sudo apt-get install gstreamer1.0-plugins-ugly
 +
sudo apt-get install gstreamer1.0-plugins-base
  
−
<pre id="json">
+
== Main Server (FREE_Web) ==
−
{
+
This software is the dispatcher of the information. By allowing the communications from the frontend and each RPi_Proxy responsible for an experience. To install this you need to install at least Python 3.7.
−
    "msg_id": "2",
 
−
    "config_params": run_config
 
−
}
 
−
</pre>
 
  
−
In response, Main_Server should receive one of the following messages, indicating the configuration and execution status of the apparatus. In case of successful configuration and execution, the response should be formatted as follows:
+
Then go to the project on github https://github.com/e-lab-FREE/FREE_Web and follow the steps of installation that we present there.
  
−
<pre id="json">
+
After the installation you will have the version of FREE 0.3.0 with a database and HTML page configured for a pendulum experiment as follow:
−
{
 
−
    "reply_id": "2",
 
−
    "status": "Experiment Running"
 
−
}
 
−
</pre>
 
  
−
In case of error, the response should be as follows:
 
  
−
<pre id="json">
+
<div style="display: flex;align-items: center;justify-content: center;"><ul>
−
{
+
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_Protocols.png|center|625px|thumb| Label ]] </li>
−
    "reply_id": "2",
+
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_Pendulum.png|center|600px|thumb| Label ]] </li>
−
    "status": status_description,
+
</ul>
−
    "error": error_description
+
</div>
−
}
 
−
</pre>
 
  
−
The status and error fields should be used to report what state the experimental apparatus was in after the error and specify in detail which error was detected, respectively.
+
After confirming that this initial installation of FREE is run as expected (you obtained a similar page as shown above on your webpage), you can install a video stream serve if your experiment has a video camera connected to the computer, this computer needs to be able to connect to the main server via the internet (VPN, ...).
  
−
=== Message 3 ===
 
  
−
Message sent to stop the ongoing experimental run. The behavior when sent to an already stopped experimental apparatus is not specified.
+
=== Janus stream server ===
  
−
<pre id="json">
+
To install the Janus stream server you will need to run the following commands on your Debian 11 machine:
−
{
 
−
    "reply_id": "3"
 
−
}
 
−
</pre>
 
  
−
In response, Main_Server should receive one of the following messages, indicating the state of the apparatus. In case of successful stop, the response should be formatted as follows:
+
sudo apt install snap​
  
−
<pre id="json">
+
sudo apt install snapd
−
{
 
−
    "reply_id": "3",
 
−
    "status": "Experiment Stopped"
 
−
}
 
−
</pre>
 
  
−
In case of error, the response should be as follows:
+
sudo snap install janus-gateway​
  
−
<pre id="json">
+
After the installation of the Janus you will need to go to the folder of the configuration of Janus by:
−
{
 
−
    "reply_id": "3",
 
−
    "status": "Experiment didn't stop",
 
−
    "error": error_description
 
−
}
 
−
</pre>
 
  
−
The error field should be used to specify in detail what error was detected.
+
cd /var/snap/janus-gateway/common/etc​
  
−
=== Message 4 ===
+
and make delete every file that is not in this list: "janus.jcfg​", "janus.plugin.streaming.jcfg", "janus.transport.http.jcfg​" and "janus.transport.websockets.jcfg​".
  
−
Message sent to command the apparatus reset.  
+
Then edit the "janus.plugin.streaming.jcfg"​ and define an administration secret​ by uncommenting line 104. After that define the possible ports that Janus will use to ​receive the video stream from the source by uncomment line 106​and put "rtp_port_ranges" for example to 6000-7000​.
  
−
<pre id="json">
+
After this, you can delete the sample endpoints by deleting all lines from line 124. And your Janus installation will be completed after restarting the Janus process by running the command:
−
{
+
−
    "reply_id": "4"
+
sudo snap restart janus-gateway​
−
}
 
−
</pre>
 
  
−
In response, Main_Server should receive one of the following messages, indicating whether it was possible to successfully reset the apparatus. In case of successful reset, the response should be formatted as follows:  
+
Now to add new video stream ports before you will need to go back to your home folder and clone the repository:
  
−
<pre id="json">
+
git clone https://github.com/e-lab-FREE/Video_Tester.git
−
{
 
−
    "reply_id": "4",
 
−
    "status": "Experiment Reseted"
 
−
}
 
−
</pre>
 
  
−
In case of error, the response should be as follows:
+
inside of this repository, you will find two python files for now we will focus on the "janus_tool.py", run it by typing the command:
  
−
<pre id="json">
+
python3 janus_tool.py
−
{
 
−
    "reply_id": "4",
 
−
    "status": "Experiment didn't Reset",
 
−
    "error": error_description}
 
−
</pre>
 
  
−
The error field should be used to specify in detail what error was detected.
+
you will get the following interface on your command line:
  
−
=== Message 5 ===
+
<div style="display: inline; width: 520px; float: center;">
 +
[[File:Janus_tools_interface.png|center|820px|thumb| Label. ]] </div>
 +
 
 +
By pressing "N", you can give a name to your stream and a description (this area just to help you distinguish the different streams).
 +
 
 +
 
 +
<div style="display: inline; width: 520px; float: center;">
 +
[[File:Janus_tools_id_port.png|center|820px|thumb| Label. ]] </div>
  
−
Message sent to request the status of the apparatus.  
+
After filling the info you will get a port and an ID as shown in the image above. This will be important when you will setup your RPi_Proxy.
  
−
<pre id="json">
+
== Raspberry PI (RPI_Proxy) ==
−
{
 
−
    "reply_id": "5"
 
−
}
 
−
</pre>
 
  
−
In response, Main_Server should receive one of the following messages, indicating whether it was possible to successfully determine the state of the apparatus. In case of successful status determination, the response should be formatted as follows:  
+
First checkout the project from github (https://github.com/e-lab-FREE/RPi_Proxy), and you can get the code using:
  
−
<pre id="json">
+
     $ git clone https://github.com/e-lab-FREE/RPi_Proxy
−
{
 
−
     "reply_id": "5",
 
−
    "status": exp_status
 
−
}
 
−
</pre>
 
  
−
In case of error, the response should be as follows:  
+
After following the installation guide on the github, you will get a version of the RPi_Proxy that is able to communicate with a pic experiment and it is configured in particular to communicate with a pic that performed a Pendulum experiment, for changing the experiment the only thing you need is to change the "experiment_details.py" on the pic_intreface folder. We have some of these files for different experiments such as the Langmuir on the following repository: [https://github.com/e-lab-FREE/pic_interface pic_interface].
  
−
<pre id="json">
+
It's possible to connect experiments that are not controlled with a pic, for example, we have on e-lab an experiment controlled by a Rasberry Pi. To do that it was needed to change the code on the "interface.py" but the name of the function need to be maintained. The code for this RPi_Proxy is presented on the following github: [https://github.com/e-lab-FREE/RPi_Proxy/tree/Cavity Cavity].
−
{
 
−
    "reply_id": "5",
 
−
    "status": "Couldn't get status",
 
−
    "error": error_description
 
−
}
 
−
</pre>
 
  
−
The error field should be used to specify in detail what error was detected.
+
=== Video capture and stream ===
  
−
== Sent messages (RPi_Server → Main_Server)  ==
+
This information is also on the github so if you follow all the github README.md of the repository "RPi_Proxy" you can sick this section.
  
−
=== Message 6 ===
+
So after getting the Repository of the Proxy from [https://github.com/e-lab-FREE/RPi_Proxy/tree/Dev_Pendulum here] you will get a file called "start-video.sh" with the following structure:
  
−
Message sent to request registration of RPi_Server in Exp_Server. The id_RP and segredo fields should be used to encode the id and secret associated with the RPi_Server requesting registration.  
+
#!/bin/bash
 +
video_server= 0.0.0.0
 +
video_port= 6000
 +
usb_camera=/dev/video0
 +
video_width=1280
 +
video_height=720
 +
video_frame=30/1
 +
 +
killall gst-launch-1.0
 +
gst-launch-1.0 v4l2src device=$usb_camera ! video/x-raw,width=$video_width,height=$video_height,framerate=$video_frame ! clockoverlay time-format="%x - %X" ! videoconvert ! omxh264enc ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=$video_server port=$video_port async=false
  
−
<pre id="json">
+
so where you will need to put the port that you configured on the "janus_tool.py" for your experiment. And the video_server with the IP of your video server.
−
{
+
One more thing needs to be checked on the "start-video.sh", the USB port where ur camera is connected, the resolution of it, and the frame rate.
−
    "msg_id":"6",
 
−
    "id_RP":id,
 
−
    "segredo":secret
 
−
}
 
−
</pre>
 
  
−
In case of successful registration, the answer should be as follows:  
+
To check your resolution and USB connection these commands may help:
  
−
<pre id="json">
+
v4l2-ctl --list-devices
−
{
 
−
    "reply_id":"6",
 
−
    "status":"Connected"
 
−
}
 
−
</pre>
 
  
−
In case of error, the answer should be as follows:
+
v4l2-ctl -d /dev/$your_dive$ --list-formats-ext
  
−
<pre id="json">
+
After this you can run the following command to start the stream:
−
{
 
−
    "reply_id":"6",
 
−
    "status":"Connection failed",
 
−
    "error":error_description
 
−
}
 
−
</pre>
 
  
−
The field error should be used to specify in detail what error was detected, e.g. the id sent does not exist in the database of authorized experiments.  
+
sh start-video.sh
  
−
=== Message 7 ===
+
==== Accessing the stream ====
  
−
Message sent to transmit the complete results of the requested trial run. Camporesults should be used to encode, in JSON format, the complete experimental results. The JSON format depends on the type of data generated by the apparatus.  
+
After putting the start-video.sh process on you can go back to your video server and run the "janus_view_streams.py" this tool will help you to check if your webcam is streaming to the server. The "janus_view_streams.py" is a simple flask server that will run on your ip on the port 5001, if you don't want to run this tester on this port in particular you will need to edit the last line of the "janus_view_streams.py":
 +
app.run(debug=True, host= '0.0.0.0', port= 5001)
 +
and change there the port.
  
−
<pre id="json">
+
After running the command: python3 janus_view_streams.py you will get the following page:
−
{
 
−
    "msg_id":"7",
 
−
    "results":run_results
 
−
}
 
−
</pre>
 
  
−
=== Message 8 ===
+
<div style="display: inline; width: 520px; float: center;">
 +
[[File:Janus_view_page_1.png|center|1200px|thumb| Label. ]] </div>
  
−
Message sent to signal an error in an experimental run that has already started. The status and error fields should be used to encode the state of the apparatus after the error and which error was detected, respectively.
+
And if you click on the second option you should be able to see your stream like this:
  
−
<pre id="json">
+
<div style="display: inline; width: 520px; float: center;">
−
{
+
[[File:Janus_view_page_2.png|center|900px|thumb| Label. ]] </div>
−
    "msg_id":"8",
 
−
    "status":status,
 
−
    "error":error_id
 
−
}
 
−
</pre>
 
  
−
=== Message 9 ===
+
= How to add new experiments =
  
−
Message sent to transmit, asynchronously, the state of the apparatus to the Exp_Server. The fields timestamp, status and current_config should be used to encode the time when the message was generated, the current state of the apparatus and the configuration of the apparatus, respectively.  
+
The first one has to have an experiment programmed in a controller of such experiment, after that and after installing the Proxy on a computer capable to communicate to the main server, the developer of the experiment needs to create the following entries on the database: Protocol, Experiment and Apparatuses.
  
−
<pre id="json">
+
== Experiment ==
−
{
+
To create a new Experiment on the database is just click on the "+" on the left of the pag.
−
    "msg_id":"9",
+
<div style="display: inline; width: 520px; float: center;">
−
    "timestamp":timestamp,
+
[[File:Experiment.png|center|1200px|thumb| Label. ]] </div>
−
    "status":status,
+
By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:
−
    "current_config":current_config
 
−
}
 
−
</pre>
 
  
−
=== Message 10 ===
+
Name-> with the some of your experiment;
  
−
Message sent to asynchronously transmit an experimental point which cannot be efficiently encoded as text, e.g. an image, and which for that reason is stored in a location accessible by the Exp_Server in binary format. The fields timestamp and id_dados_bin must be used to encode the time at which the message was generated and the address where the client will be able to obtain the data sent by the RPi_Server, respectively.  
+
Slug name-> the name of the html file of your experiment (explain on the section ...);
  
−
<pre id="json">
+
Description-> the particular html with the description of the experiment, with the information that the user need to understand it;
−
{
 
−
    "msg_id":"10",
 
−
    "timestamp":timestamp,
 
−
    "bin_data_address":data_address
 
−
}
 
−
</pre>
 
  
−
=== Message 11 ===
+
Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);
  
−
Message sent to asynchronously transmit an experimental point encoded as text. The timestamp and date fields should be used to encode the time the message was generated and the experimental data to be transmitted, formatted in JSON, respectively. The formatting of experimental data depends on the type of data generated by the apparatus and should be appropriate to it.
+
Scientific area-> what area is the experience integrated;
  
−
<pre id="json">
+
Lab type-> the background needed to execute the experiment (basic, intermedium, advance).
−
{
 
−
    "msg_id":"11",
 
−
    "timestamp":timestamp,
 
−
    "data":data_point
 
−
}
 
−
</pre>
 
  
−
= How to install =
+
== Protocol ==
 +
After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.
 +
 
 +
Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.
 +
 
 +
<div style="display: inline; width: 520px; float: center;">
 +
[[File:Protocols.png|center|1200px|thumb| Label. ]] </div>
 +
 
 +
You will have a page like the one shown above that you will need to file:
 +
 
 +
Experiment-> select your exeperiment;
 +
 
 +
Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;
 +
 
 +
Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema [https://bjdash.github.io/JSON-Schema-Builder/ here]).
 +
 
 +
== Apparatuses ==
 +
 
 +
After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).
 +
 
 +
<div style="display: inline; width: 520px; float: center;">
 +
[[File:Aparatus.png|center|1200px|thumb| Label. ]] </div>
 +
 
 +
You will have a similar page as a shown above, so you need to fill the follow information:
  
−
== Pre-requisites ==
+
Experiment-> the one experiment that apparatus is connected to;
  
−
=== Build environment ===
+
Protocol-> the protocols related that Experiment;
  
−
* Git (2.30.2+)
+
Location-> Location of the experiment;
−
* Python (3.7.0+)
 
  
−
== Raspberry PI Server (RPI_Server) ==
+
Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;
−
Setup python3 environment variable pointing to run the version installed (in this exemple is for python3.7) by:
 
  
−
    $ sudo nano ~/.bashrc
+
Owner-> the person responsible for the maintenance  of the experiment;
  
−
edit it adding to the end:
+
Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).
−
    $ alias python3="python3.7"
 
  
−
First checkout the project from github:
 
  
−
    $ git clone https://github.com/e-lab-tecnico-ulisboa-pt/RPi_Server-PG
+
== HTML/JS ==
  
−
then edit the "main.py" file and change the following global variables:
+
With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:
  
−
    SERVER = "main_server_IP"
+
<div style="display: inline; width: 520px; float: center;">
−
    MY_IP = "RPi_IP"
+
[[File:Elab_page_1.png|center|820px|thumb| Label. ]] </div>
−
    SEGREDO = "password"
 
−
    PORT = Port_to_comunication
 
−
    BINARY_DATA_PORT = Port_to_transfer_Binary_data
 
  
−
After this changes the RPi_Server is ready to run by typing:
+
Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration
  
−
    $ pyhton3 main.py
+
<div style="display: inline; width: 520px; float: center;">
 +
[[File:Elab_page_2.png|center|820px|thumb| Label. ]] </div>
  
−
This will try to connect to the IP defined as SERVER every 10 sec.
+
you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.
  
−
== Main Server (Main_Server) ==
+
<div style="display: inline; width: 520px; float: center;">
−
This software is the dispatcher of the information. By allowing the communications form the frontend and each RPi_Server responsable for a experience. To install this you need to install at least Python 3.7 and the lib Flask (https://pypi.org/project/Flask/).
+
[[File:Elab_page_3.png|center|820px|thumb| Label. ]] </div>
  
−
Then go to the project from github:
 
  
−
    $ git clone https://github.com/e-lab-tecnico-ulisboa-pt/Main_Server-RexC
+
To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment.  
  
−
Edit the "main.py" file and change the following global variables:
 
  
−
    SERVER = "main_server_IP"
+
<div style="display: flex;align-items: center;justify-content: center;"><ul>
−
    MY_IP = "RPi_IP"
+
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_templates.png|center|820px|thumb| Label. ]] </li>
−
    SEGREDO = "password"
+
<li style="display: inline-block; horizontal-align: center; "> [[File:FREE_static.png|center|253px|thumb| Label. ]]  </li>
−
    PORT = Port_to_comunication
+
</ul>
−
    BINARY_DATA_PORT = Port_to_transfer_Binary_data
+
</div>
  
−
After that you need to change or edit this last one variable "segredos",
+
After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS  file that you need to display the HTML you will need to add them to the respective folders on the folder  "/free/static/free".
  
−
[[File:ADD_the_new_experiment.jpg]]
+
<!--= Video Streaming = --!>
  
−
the struct of it's the following:
+
Intro
  
−
<pre>
+
== Video capture and stream ==
−
{
 
−
    "RPi_IP" :
 
−
    {
 
−
        "segredo": "password",
 
−
        "nome": "Name_of_the_exp"
 
−
    },
 
−
    ...
 
−
}
 
−
</pre>
 
  
−
=== Config File (JSON) ===
+
On the raspberry it is running ....
  
−
In the project of the Main_Server there is a folder called Configs, in there are all the configuration files of the experience linked to the Main_Server. To add a new experiment you just have to copy one of the existing files and change its name to the experiment's and change the parameters in the config file:
+
== Janus stream server ==
  
−
[[File:Config_JSON.jpg|850 px]]
+
== Accessing the stream ==
  
−
After this changes done you can run the Main_Server, typing the cmd:
+
The experiment stream can be access by a html ....
  
−
    $ start.bat
+
or by using VLC, allowing to grab the video

Latest revision as of 09:35, 1 June 2022

Current releases

Version 0.5.0

Alpha release Major bugs corrected, adding video configuration and checking experiments status.

Version 0.3.0

(Bogotá release) Test version including video server.

System architecture

In the defined architecture, users communicate with a central server, named FREE_Web, through web pages.

This FREE_Web server contains the database with the results of the experiments already carried out and the experimental executions and experimental time schedules requested by users through the web interface. The FREE_Web server is also responsible for mediating real-time communication between users and experiences, exchanging configuration requests, status requests and responses and generated data between users actively connected to the apparatus and apparatus.

Architecture of the new system. It is possible to observe the different layers of software that separate the users, on the left side of the image, from the experimental apparatus, on the right side of the image.


Along with the experimental apparatus, a computer, named RPi_Proxy, acts as an interface between the controller of the experimental apparatus and the FREE_Web server. Each RPi_Proxy is turned on and represents one, and only one, experimental apparatus. It is important to note that this component was named RPi_Proxy because it is, in principle, implemented through the use of a Raspberry Pi, a computer that has communication modules necessary to communicate natively with the microcontrollers usually used to control the devices. However, this component can be implemented in other ways and it is only strictly necessary that it communicates according to the protocol - e.g. a virtual experience with data generated through a numerical model can implement this component with a normal computer without specialized interfaces. The communication between all components is done over the internet, with the exception of the communication between the RPi_Proxy and the experimental apparatus, where communications are made through the protocol that the controller of the apparatus implements.

Communication protocol

To start the connection between the RPi_Proxy and the FREE_Web, when the RPi_Proxy is turned on, it tries to connect to the FREE_Web server. If not successful, it should wait 10 seconds and try again until it manages to make the connection.

  • Label
  • Label

When the connection is successfully established, the server RPi_Proxy sends to FREE_Webo its ID and Secret. These two variables are then checked on the FREE_Web server side as a simple way to authenticate that the connection is coming from an authorized machine. In case of successful verification, the FREE_Web responds to the RPi_Proxy with a message encoded in JSON format that describes the experience to be served by the RPi_Proxy. This configuration message allows the same RPi_Proxy program to be adaptable to different experiences with, for example, different numbers of actuators and sensors without changes to the source code. Based on the information contained in this message, the name of the config_file, the RPi_Proxy looks for the experiment at the indicated address and tries to establish communication with the controller of the experimental apparatus. In order to decouple the RPi_Proxy from the different communication protocols that each experimental controller may implement, the RPi_Proxy must communicate with the devices calling a set of standard methods whose implementation must be adapted to suit the target controller.


(TODO)


After this initial configuration, the connection of the RPi_Proxyao FREE_Weband the RPi_Proxyto the controller of the experiment, the system is ready for the exchange of messages between the components. The messages exchanged between components were defined following an everything is a string philosophy. Thus, the messages exchanged are JSON-formatted strings in which the JSON fields encode the message to be transmitted. The type of each message is identified by the msg_id field of the message JSON, and for the message sent, the reply, if any, must have reply_id with the same value. In order to be adaptable to various types of experimental apparatus, the system does not store apparatus status information so it is possible to send any of the defined messages at any time. The correct sequencing of messages for the successful operation of the apparatus is the responsibility of other user-controlled software layers.

Functions documentation

FREE_Web

RPi_Proxy

interface.py(RPi_Proxy)

How to install

Pre-requisites

Main server

For FREE_Web:

  • Git (2.30.2+)
sudo apt install git
  • Python (3.7.0+)
sudo apt install python3
sudo apt install python3-pip

There are need more installation but on the repository, you will have a "REQUIREMENTS.txt" that you will run to install them.


For the video server:

  • Janus
sudo apt install snap​
sudo apt install snapd​
sudo snap install janus-gateway​

For tools to test:

  • Flask
pip3 install flask​
pip3 install flask_cors​

Rasberry Pi

Operating System:

  • Raspbian GNU/Linux 10 (buster)

For the Proxy it self:

  • Python (3.7.0+)
sudo apt install python3
sudo apt install python3-pip
sudo apt install python3-venv​
pip3 install pyserial
  • Git (2.30.2+)
sudo  apt-get install git

For the video stream: 

sudo apt-get install gstreamer1.0-tools
sudo apt-get install gstreamer1.0-plugins-good
sudo apt-get install gstreamer1.0-plugins-bad
sudo apt-get install gstreamer1.0-plugins-ugly
sudo apt-get install gstreamer1.0-plugins-base

Main Server (FREE_Web)

This software is the dispatcher of the information. By allowing the communications from the frontend and each RPi_Proxy responsible for an experience. To install this you need to install at least Python 3.7.

Then go to the project on github https://github.com/e-lab-FREE/FREE_Web and follow the steps of installation that we present there.

After the installation you will have the version of FREE 0.3.0 with a database and HTML page configured for a pendulum experiment as follow:


  • Label
  • Label

After confirming that this initial installation of FREE is run as expected (you obtained a similar page as shown above on your webpage), you can install a video stream serve if your experiment has a video camera connected to the computer, this computer needs to be able to connect to the main server via the internet (VPN, ...).


Janus stream server

To install the Janus stream server you will need to run the following commands on your Debian 11 machine: 

sudo apt install snap​

sudo apt install snapd

sudo snap install janus-gateway​

After the installation of the Janus you will need to go to the folder of the configuration of Janus by: 

cd /var/snap/janus-gateway/common/etc​

and make delete every file that is not in this list: "janus.jcfg​", "janus.plugin.streaming.jcfg", "janus.transport.http.jcfg​" and "janus.transport.websockets.jcfg​".

Then edit the "janus.plugin.streaming.jcfg"​ and define an administration secret​ by uncommenting line 104. After that define the possible ports that Janus will use to ​receive the video stream from the source by uncomment line 106​and put "rtp_port_ranges" for example to 6000-7000​.

After this, you can delete the sample endpoints by deleting all lines from line 124. And your Janus installation will be completed after restarting the Janus process by running the command: 

sudo snap restart janus-gateway​

Now to add new video stream ports before you will need to go back to your home folder and clone the repository: 

git clone https://github.com/e-lab-FREE/Video_Tester.git

inside of this repository, you will find two python files for now we will focus on the "janus_tool.py", run it by typing the command: 

python3 janus_tool.py

you will get the following interface on your command line:

Label.

By pressing "N", you can give a name to your stream and a description (this area just to help you distinguish the different streams).


Label.

After filling the info you will get a port and an ID as shown in the image above. This will be important when you will setup your RPi_Proxy.

Raspberry PI (RPI_Proxy)

First checkout the project from github (https://github.com/e-lab-FREE/RPi_Proxy), and you can get the code using: 

   $ git clone https://github.com/e-lab-FREE/RPi_Proxy

After following the installation guide on the github, you will get a version of the RPi_Proxy that is able to communicate with a pic experiment and it is configured in particular to communicate with a pic that performed a Pendulum experiment, for changing the experiment the only thing you need is to change the "experiment_details.py" on the pic_intreface folder. We have some of these files for different experiments such as the Langmuir on the following repository: pic_interface.

It's possible to connect experiments that are not controlled with a pic, for example, we have on e-lab an experiment controlled by a Rasberry Pi. To do that it was needed to change the code on the "interface.py" but the name of the function need to be maintained. The code for this RPi_Proxy is presented on the following github: Cavity.

Video capture and stream

This information is also on the github so if you follow all the github README.md of the repository "RPi_Proxy" you can sick this section.

So after getting the Repository of the Proxy from here you will get a file called "start-video.sh" with the following structure: 

#!/bin/bash
video_server= 0.0.0.0
video_port= 6000
usb_camera=/dev/video0
video_width=1280
video_height=720
video_frame=30/1

killall gst-launch-1.0
gst-launch-1.0 v4l2src device=$usb_camera ! video/x-raw,width=$video_width,height=$video_height,framerate=$video_frame ! clockoverlay time-format="%x - %X" ! videoconvert ! omxh264enc ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=$video_server port=$video_port async=false

so where you will need to put the port that you configured on the "janus_tool.py" for your experiment. And the video_server with the IP of your video server. One more thing needs to be checked on the "start-video.sh", the USB port where ur camera is connected, the resolution of it, and the frame rate.

To check your resolution and USB connection these commands may help: 

v4l2-ctl --list-devices

v4l2-ctl -d /dev/$your_dive$ --list-formats-ext

After this you can run the following command to start the stream: 

sh start-video.sh

Accessing the stream

After putting the start-video.sh process on you can go back to your video server and run the "janus_view_streams.py" this tool will help you to check if your webcam is streaming to the server. The "janus_view_streams.py" is a simple flask server that will run on your ip on the port 5001, if you don't want to run this tester on this port in particular you will need to edit the last line of the "janus_view_streams.py": 

app.run(debug=True, host= '0.0.0.0', port= 5001)

and change there the port.

After running the command: python3 janus_view_streams.py you will get the following page:

Label.

And if you click on the second option you should be able to see your stream like this:

Label.

How to add new experiments

The first one has to have an experiment programmed in a controller of such experiment, after that and after installing the Proxy on a computer capable to communicate to the main server, the developer of the experiment needs to create the following entries on the database: Protocol, Experiment and Apparatuses.

Experiment

To create a new Experiment on the database is just click on the "+" on the left of the pag.

Label.

By doing that you we get to a similar pag as the above, so you need to fill the information about your experiment:

Name-> with the some of your experiment;

Slug name-> the name of the html file of your experiment (explain on the section ...);

Description-> the particular html with the description of the experiment, with the information that the user need to understand it;

Configuration-> is a JSON with basic information that the Proxy needs to know about the experiment such as the serial port that is connected (for example in the case of the experiment Pendulum);

Scientific area-> what area is the experience integrated;

Lab type-> the background needed to execute the experiment (basic, intermedium, advance).

Protocol

After finishing your Experiment you need to create a new Protocol to do that again click on the "+" next to the Protocols tab on the left.

Every Experiment should have at least one Protocol. A Protocol is a way to describe the parameters that a user can select and modify to run an experimental execution. One physical setup of an experimental can do different execution with different initial parameters so that is why one experiment can have multiple Protocols.

Label.

You will have a page like the one shown above that you will need to file:

Experiment-> select your exeperiment;

Name-> with the name of the protocol, this name should be descriptive of the experimental setup that the controller will use. That you can translate;

Configuration-> this is a JSON schema that will test the inputs that the user will give, and check that their configuration are within the range of the experiment ( You can do your JSON Schema here).

Apparatuses

After programing the Protocols one last thing needs to be done on the database. You need to provide to the database the information of the Apparatus (the computer witch will connect to the FREE_Web).

Label.

You will have a similar page as a shown above, so you need to fill the follow information:

Experiment-> the one experiment that apparatus is connected to;

Protocol-> the protocols related that Experiment;

Location-> Location of the experiment;

Secret-> the password that you need to configure and the "server_info.ini" on the RPi_Proxy;

Owner-> the person responsible for the maintenance of the experiment;

Video Configuration -> the ID that the Port of the stream of the video is connected to the server of Janus (explained in the section ...).


HTML/JS

With the FREE installed you will have the following page to run an experiment Pendulum, basically it starts with a discretion that is on the database:

Label.

Then you can go to the Configuration tab and set up the initial condition of the experiment, and after Save and Submit the configuration

Label.

you will be sent to the tab Execution where you will be able to see the video stream and the plot of the data sent by the experiment.

Label.


To finalize the process of creating a new experiment and adding it to the FREE you will need to do an HTML and JS similar to the one that we provide for the Pendulum experiment. Basically, you will need to add as much inputs on the Configuration tab, and change the JS that is plotting, and put the data point on a table so it displays the data points of your experiment.


  • Label.
  • Label.

After you make sure that everything is working as intended you need to add the HTML fill on the "/free/templates/free/experiments" folder and make sure that the name of the file is the same as the Slug that you put on the database ($Slug_name$.html) and the JS, image and CSS file that you need to display the HTML you will need to add them to the respective folders on the folder "/free/static/free".


Retrieved from "http://www.elab.tecnico.ulisboa.pt/wiki/index.php?title=FREE_-_Framework_for_Remote_Experiments_in_Education&oldid=4556"