Webserver Development

[TerryJC]

On the log in issue if it's not too much extra work or a security compromise I'd prefer the status page unprotected and with the controls behind a log in.
That would allow the WMT volunteers to monitor but not change settings.

Penri,

Thanks for your response. I have actually got a working version of what you've asked for because I assumed that this would be the preferred solution.

I'm making the assumption that the username and password would be the same as for the Pi, since that is what we would be using to log in via SSH. If a different set of credentials are needed, then that can be done too.

[Penri]

Excellent, thank you.

[TerryJC]

Hamish,

Can you spare a bit of time to look at my implementation of the code in my Flask App to fetch the System Status from the Music Player program? I pushed a version of the code a few days ago, so you can look at it on GitLab if you need to. I raised a query on this on the Raspberry Pi Forums (see https://www.raspberrypi.org/forums/view ... 2&t=303229), but unfortunately, some of the answers have confused me more than they help.

To summarise the current situation, if I leave out my additional functions then the pages render satisfactorily, but they do not do anything As soon as I add the sockets code and my fetch_system_status() function, the pages no longer render.

I suspect that the structure of my App is wrong because I think that maybe the program can't find its way to the default Route after the Status List variable has been populated; one of the people on the RPi Forums has suggested a suitable structure. Unfortunately, it is totally different to what I've seen elsewhere (it doesn't even have any Routes), so I'm somewhat lost.

The same guy has suggested that the sockets code may be blocking the App. His reasoning for this is that when I put the Flask Development Server into Debug Mode, I get this:

Code: Select all

pi@minster-music:~/html $ export FLASK_DEBUG=1
pi@minster-music:~/html $ flask run             
 * Serving Flask app "statusmonitor.py" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
 * Restarting with stat
About to run Startup
Minster Webserver Software Version0.2.1 (26/01/2021)
System startup sequence initiated.
System startup sequence initiated.
Waiting 20 seconds for the Music Player to start
 * Debugger is active!
 * Debugger PIN: 172-504-382
About to run Startup
Minster Webserver Software Version0.2.1 (26/01/2021)
System startup sequence initiated.
System startup sequence initiated.
Waiting 20 seconds for the Music Player to start
Sockets._create_and_connect(): (Music Pi Client Socket): Error connecting:

Traceback (most recent call last):
  File "/home/pi/html/Tools/sockettools.py", line 432, in _create_and_connect
    self._create_socket()
  File "/home/pi/html/Tools/sockettools.py", line 550, in _create_socket
    self.server_socket.bind(('', self.port_number))
OSError: [Errno 98] Address already in use

Sockets._create_and_connect(): (Music Pi Client Socket): Unknown error, possibly address already in use?
Sockets._create_and_connect(): (Music Pi Client Socket): Retrying in 10 seconds...
Connection Timed Out (Music Pi Client Socket): [Errno 98] Address already in use. Retrying in 10 seconds...
Connected to peer (Music Pi Client Socket).
Sockets._create_and_connect(): (Music Pi Client Socket): Error connecting:

Traceback (most recent call last):
  File "/home/pi/html/Tools/sockettools.py", line 432, in _create_and_connect
    self._create_socket()
  File "/home/pi/html/Tools/sockettools.py", line 550, in _create_socket
    self.server_socket.bind(('', self.port_number))
OSError: [Errno 98] Address already in use

which repeats ad infinitum.

Can you comment on that aspect of the problem?

[TerryJC]

Hamish,

Please ignore my blatherings above. I've just realised that I had forgotten to do something rather fundamental and so I wasn't looking at what I thought I was when I pointed my browser at the Pi.

So the good news is that I have made a major step forward and am now able to retrieve the System Status and display it on the Web Page. I have a fair bit of work to get all the control functions working, but I can now see a definite way forward.

I also need to resolve the permissions question that I raised on the LUG List the other day. I now know what needs to happen, but not quite how to do it yet.

[TerryJC]

Sitrep2:

OK. I now have the Webserver running on the Flask Development Server and the functionality seems OK to me.

Penri,

If you have time over the next day or so, could you check it out? When you want to look at it just let me know and I'll ensure that my Router is forwarding data from the Webserver on the Pi to the Internet. You'll need my IP Address and you'll also need the Pi's password so you can get into the Control Page.

The Control Page is a bit slow at the moment. For various reasons I had to put a number of delays into the system to ensure that the Webserver had retrieved the latest status information and once we're happy about the layout and content, I'll look into removing some of those delays.

[Penri]

Hello Terry

This afternoon or tomorrow afternoon work for me.


Penri

[Penri]

Reviewed status and control webpages for the Minster system this afternoon, herewith my report:

Looks nice and everything works as advertised, although I couldn’t hear the music or bells!

BTW have tried it on my Mac, iPhone and iPad, all worked the same, both status and control.

Typeface small on the iPhone but text very clear and readable with my reading glasses on.

Good spacing to buttons so little chance of hitting the wrong one.

One of the frustrations of the initial installation was knowing if the bells / quarterjack would do their stuff at the appointed time, especially having primed guests’ expectation, now knowing system time is going to be invaluable.

Very nice, good job.

[TerryJC]

Penri,

Thanks. I'm removing the Port Forward on my Router now that you finished testing.

Someone had a go at hacking the system earlier today as evidenced by the Webserver logs!

[TerryJC]

Having spent a while on other things, including completing the hardware build, I'm now trying to deploy the system as a whole.

Webserver wise, I seem to have resolved the permissions problem that I had earlier, but I cannot actually make the Webserver work in production mode. When I asked Penri to look at the running system a few weeks ago, I was using the Flask Development Server and everything worked as required. At that time, I had all of the code owned by the Pi User and since the Flask Development Server was being launched by the Pi user, everything was fine.

The problem I had before and may still have is that nginx is deployed with the uwsgi option and this runs as the www-data user. This necessitated me making the /html directory and all its contents owned by the www-data user and in the www-data group. I also had to make the /home/pi/logs/ directory writable by others to ensure that the log files could be created.

However, it still doesn't work. I have done some diagnostics and posted the output below:

Code: Select all

pi@minster-music:~/logs $ sudo systemctl status uwsgi.service
● uwsgi.service - uWSGI Service
   Loaded: loaded (/etc/systemd/system/uwsgi.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2021-03-16 13:02:47 GMT; 1min 7s ago
 Main PID: 518 (uwsgi)
    Tasks: 4 (limit: 2062)
   CGroup: /system.slice/uwsgi.service
           ├─518 /usr/local/bin/uwsgi --ini /etc/uwsgi.ini
           └─630 /usr/local/bin/uwsgi --ini /etc/uwsgi.ini

Mar 16 13:02:50 minster-music uwsgi[518]: Minster Webserver Software Version0.2.1 (26/01/2021)
Mar 16 13:02:50 minster-music uwsgi[518]: System startup sequence initiated.
Mar 16 13:02:50 minster-music uwsgi[518]: Waiting 20 seconds for the Music Player to start
Mar 16 13:02:56 minster-music uwsgi[518]: Connected to peer (Music Pi Client Socket).
Mar 16 13:03:10 minster-music uwsgi[518]: Music Pi System Status Request Sent
Mar 16 13:03:25 minster-music uwsgi[518]: Music Pi System Status Received
Mar 16 13:03:25 minster-music uwsgi[518]: WSGI app 0 (mountpoint='') ready in 37 seconds on interpreter 0x1d96380 pid
Mar 16 13:03:25 minster-music uwsgi[518]: *** uWSGI is running in multiple interpreter mode ***
Mar 16 13:03:25 minster-music uwsgi[518]: spawned uWSGI master process (pid: 518)
Mar 16 13:03:25 minster-music uwsgi[518]: spawned uWSGI worker 1 (pid: 630, cores: 2)
pi@minster-music:~/logs $ tail -f Minster-Music.log 
16/03/2021 13:02:56 - Tools.sockettools - INFO: Sockets._create_and_connect(): (Webserver Socket): Done!
16/03/2021 13:03:03 - Tools.sockettools - INFO: Sockets._connect_socket(): (Bells Pi Sockets Client): Done!
16/03/2021 13:03:03 - Tools.sockettools - INFO: Sockets._create_and_connect(): (Bells Pi Sockets Client): Done!
16/03/2021 13:03:10 - Tools.sockettools - INFO: Sockets._read_pending_messages(): (Webserver Socket): Received data.
16/03/2021 13:03:23 - Minster Software - INFO: Webserver Status Request Received
16/03/2021 13:03:23 - Tools.sockettools - INFO: Sockets._send_pending_messages(): (Bells Pi Sockets Client): Sending 
16/03/2021 13:03:23 - Tools.sockettools - INFO: Sockets._send_pending_messages(): (Bells Pi Sockets Client): Sending 
16/03/2021 13:03:24 - Tools.sockettools - INFO: Sockets._read_pending_messages(): (Bells Pi Sockets Client): Received
16/03/2021 13:03:24 - Minster Software - INFO: Bells Pi Status Received
16/03/2021 13:03:25 - Tools.sockettools - INFO: Sockets._send_pending_messages(): (Webserver Socket): Sending data...
pi@minster-music:~/logs $ tail -f Minster-Webserver.log 
16/03/2021 13:02:50 - Minster Webserver Software - INFO: Minster Webserver Software Version 0.2.1 (26/01/2021)
16/03/2021 13:02:50 - Minster Webserver Software - INFO: System startup sequence initiated.
16/03/2021 13:02:50 - Minster Webserver Software - INFO: Creating sockets...
16/03/2021 13:03:10 - Minster Webserver Software - INFO: Music Pi System Status Request Sent
16/03/2021 13:03:25 - Minster Webserver Software - INFO: Music Pi System Status Received

The thing is that the Flask App statusmonitor.py seems to start OK and even successfully communicates with the minster-music program, but then it seems to disappear in less than a minute. Viewing the running processes in htop shows many instances of python3, but they are all running minstermusic.py.

Any ideas how I can resolve this?

[hamishmb]

Quick thought: does the program return once it is started?

If not, systemd may just time out the start operation. "journalcyl -xe" should help with diagnosing this.

You may just need to put an "&" at the end of the command so it returns and systemd registers that the job started.