Current status of new river system software features (15 March 2022 Onwards)

[hamishmb]

This page tracks the current status of the latest gamut of river system software changes. Unfortunately there's no good order to do some of these in, so there's going to have to be some re-work along the way. I have tried to present these in a vaguely chronological order, but we will see what issues I encounter along the way.

As with the documentation, I have ambition but no set timescale in mind for these.

• Integrate and test Patrick's temporary top up logic for the Lady Hanham site.
  
  • Update the software on hanhampi and the associated gate valves (Done 15 March 2022].
  • Install the automatic software update script (Done 15 March 2022)
  • Create new, up-to-date river system VMs to aid testing (Done ~June 2022)
  • Test and fix any issues with the top up logic (Not Started)
  • Release a minor new version of the river system software once this milestone is reached (Not Started)
• Create and integrate water backup logic (for someone else to do).
  
  • Terry wrote the Sump Pi and Wendy Pi backup logic (Done ~June 2022)
  • Test and integrate Sump Pi backup logic into the system (Not Started)
  • Test and integrate Wendy Pi backup logic into the system (Not Started)
  • Determine if any backup logic needs to be created for the other sites. (Not Started)
• General clean-up and refactor.
  
  • Refactor various parts of the code to improve maintainability (In Progress 40% - 26 August 2022)
  • Improve in-software documentation for any undocumented/incorrectly documented methods and classes (In Progress 20% - 26 August 2022).
  • Improve unit and integration test coverage to increase confidence in the software (Not Started)
  • Fix some outstanding minor issues (In Progress - 26 August 2022)
• Database changes
  
  • Update the database schema on the NAS box to support manual control features and improve stability (Not Started)
  • ^ This will mean clearing the database on the NAS box and starting again.
  • Change the way the database integrity check works so it doesn't delay software startup on the NAS box (Not Started)
  • Update the NAS Box VM to reflect these changes (Not Started)
  • Upload this new database schema to this webserver to ensure easy set-up in the future (Not Started)
  • Update the database backup and restore scripts (if needed) to ensure they remain operable (Not Started)
  • Update the NAS Box setup instructions (if needed) to reflect these changes (Not Started)
  • Implement a watchdog to automatically restart the river system software in the event of a crash (Not Started)
  • Attempt to finally fix the random-crashing issue with the NAS box (Not Started)
  • After this, non-glossary design documentation can be written as there will be no more major changes.
• Improve ability to test without hardware.
  
  • A CLI will be devised to be able to set different probe levels, valve positions, motor states, and float switch values to test the system more completely using the VMs (Not Started)
• Other improvements.
  
  • Attempt to prevent control logic hangs when the database is responding slowly or is offline (Not Started)
  • Make each site store a short history of the system state to ensure functionality is maintained if the database is unavailable for a short period of time (Not Started)
  • Fix clean shutdown of the software when unhandled errors have occurred (Not Started)
  • Add a way for each software thread to report unrecoverable errors and trigger the software to log the event (including with the database if possible) and automatically restart (Not Started)
  • Ensure a basic level of software functionality in the event of the database being empty (Not Started)
  • Fix whatever issues there are with the site-wide river system software updater that prevent reliable operation (In Progress 10% - 26 August 2022)
  • If a gate valve is stuck, back off to avoid overheating the motor, and report the error (Not Started)
  • Improve response time to requests to eg open a valve by responding as soon as messages arrive (Not Started).
  • Implement a new "SYNC" message to immediately reset the reading interval to keep the systems in sync (Not Started)
  • Various other small changes and improvements (In Progress - 26 August 2022)

Nice to have:

The following features were decided to be unimportant but nice to have. I will not be implementing them, but they are noted here in case they are wanted in the future.

Code: Select all

- Make sure eg Pumps are stopped and valves are shut before the software reboots/shuts down?
	- Or could this lead to overflows or valve/pump damage if this isn't coordinated very carefully?
	- Would have to be very careful to be sure that nothing would be broken.
		- Could cause the sump to run dry or overflow if everything is left as is.
	- This is not a huge issue for reboots because the reboot is quick.
	- We will need someone on-site anyway for shutdown, so not a huge issue.

Last updated 31 August 2022.

[hamishmb]

I have updated the list to note that the up-to-date VMs were created. I have also merged all the new code into main, so that I can get started with these tasks. I will post back here when progress is made, along with the current General Software Improvements thread for more general changes.

[TerryJC]

Hamish,

There's no mention of integrating the water backup software.

[hamishmb]

Good catch, I have now added that.

[hamishmb]

I have now started refactoring and improving code quality in advance of doing other work.

[hamishmb]

I have updated the list to reflect that I'm not doing any more river system work than originally agreed - hence some of what is in this list is now someone else's responsibility.

The database changes may not be necessary with the advent of the CLI to set readings and take temporary control of the devices in the system. Thoughts? If this is no longer needed, that's a fair bit of extra work removed. My opinion is that the CLI should be enough - we shouldn't need to interact with the system at all unless we're debugging.

[TerryJC]

The database changes may not be necessary with the advent of the CLI to set readings and take temporary control of the devices in the system. Thoughts? If this is no longer needed, that's a fair bit of extra work removed. My opinion is that the CLI should be enough - we shouldn't need to interact with the system at all unless we're debugging.

I personally have no problem with doing this via a CLI and if this makes proress faster, I'm all for it.

However, there is the question of what is described as the 'web-browser-accessible Graphical User Interface (GUI) in Requirement 4.3.4. This was always intended to provide a facility for Visitors to view the system and Staff to control it. I believe that your Engineer GUI concept grew out of that. Clearly the Staff would not be able to use a CLI based tool to manually control the system.

Having said that, if the necessary functions are supported by the CLI tool, I see no reason why I shouldn't produce the web-browser-accessible GUI using Flask, as I did for the Minster.

[hamishmb]

We would need database support for the feature, realistically, for it to be usable with a visitor/staff GUI.

The problem with the engineer GUI is that it was designed from the start to be view-only, and it runs on the severely-hampered and ancient webserver on the NAS box. Coding this in a separate staff GUI would make more sense to me, for sure, and Flask seems like a good choice.

So I guess we would still need the database changes, but if everyone's happy, I can remove the engineer GUI changes from the list. I think Penri requested them only for debugging purposes, and that problem will be solved with the CLI, but we should still check with him first I think.

[hamishmb]

I have updated the list to reflect this week's progress. There are a number of things that it may be possible to be skipped following some discussion, and a number of things that need to be discussed to determine if work is needed. I will post these here next week.

[Penri]

Hello

Coming in a bit late on this but what I wanted, that became the "requirement" for the Engineering GUI, was a way of simply, ie without any S/W knowledge, be able to see what each component of the system 'thought' it was doing.
So I could see on my phone for example: what a probe was reporting as the water level, the position of a valve, state of a pump, etc. so I could compare it with what I could physically see.
It would be even more useful if I could also, via the engineering GUI, open or close valves, turn on and off a pump or solenoid, but I can see that being more complex to deal with when manual control was relinquished, probably a step far too far!