General Software Improvements (2nd October 2019 onwards)

[hamishmb]

I have successfully done a preliminary test of my virtual river system, and have managed to get all the virtual pis talking to the NAS box, and to store readings in the database.

Next I need to do the control logic functions, followed by setting the reading interval and tick at the NAS box end. Good progress nevertheless

NB: CPU and memory usage on the NAS box seem nice and low as well - 5-10% CPU load, ~25% memory usage.

[TerryJC]

Excellent news!

[PatrickW]

Today I have (finally) been testing and debugging the code I wrote for the Stage Pi control logic, using the network of virtual Pis. Seems to be going OK. Fixing the usual type and and quantity of errors that I would expect to encounter (mostly pretty dumb ones). Having seen that the design basically works, I will follow up with more thorough unit tests for the classes I wrote.

One issue I've run into is that, when main.py runs on Stage Pi (G6), it does not populate its `readings` dictionary with any readings from G4, which the control logic needs (by specification).

This is a feature of the existing code in main.py. I never previously checked whether it provided the necessary readings. As far as I can tell, it is only sites that are "HostingSockets" that are initialised with SocketsMonitors to obtain readings from other sites. In the version of the configuration that I based my git branch on, that's a club with only one member: SUMP. All the other sites only have access to their own readings.

So, my question is: Under the sockets-based design, what is the intended means by which G6 Stage Pi gets readings from G4 Wendy Butts Pi?

Is this just a simple configuration change (i.e. should G6 be configured with

Code: Select all

"HostingSockets": True

), or was it never envisaged that multiple sites would need readings from other sites, and therefore the feature requires new code or switching to the database method of distributing readings? (Was this, in fact, part of the motivation to use the database?)

This will be irrelevant when the database supplants sockets, but if it's just a simple config change I'll do that for now. If it's more complex, then I'll come up with a workaround to avoid having to have those readings available just for the time being, as I don't need it to be fully working to test and debug what I've written.

[TerryJC]

Hamish,

When i formulated the original segmentation proposal, the idea was that all devices could make themselves aware of all values if they needed them. This was intended to be fulfilled by the database,so in your example Stage Butts Pi would find out the level in the Wendy Butts and act accordingly.

At the time, I didn't think too hard about how we would do this under a pure sockets approach, because I thought that the NAS Box code and the Stage Butts code would be developed concurrently. It hasn't been so we are left with what we have.

Personally, I wouldn't want you to spend to much time on getting this to work with pure sockets code, although I appreciate that there is a bit of a chicken and egg situation here. If you think it can be made to work, then fine, but I would like to see the NAS Box on site if possible and do it that way.

[PatrickW]

OK Terry, that answers the question nicely. Thank you.
It's Patrick, by the way.

[TerryJC]

Patrick,

Sorry. I've been in deepish discussion with Hamish for a while and didn't spot that it was you posting. I thought he was being a bit busy.

[hamishmb]

I was wondering if I should reorganise the folder structures for our tests a bit:

/UnitTests contains the software unit tests, and these can be run via /unittests.py - this is fine.

/tests contains your hardware tests, as well as some software system tests - maybe I should move these into separate directories to avoid confusion?

[TerryJC]

Hamish,

I was thinking of doing that myself some time ago and decided that I wasn't sure if I could do it without breaking the structure in git.

Ideally, I think that we should have a hierarchical structure. Something like:

Testing:
____Software
______Unit Tests
______System Tests
____Hardware
______Unit Tests

I'll leave it up to you to decide the structure. Maybe something with a few less levels is all that's needed.

[TerryJC]

D**m. phbBB removes the spaces...

(I've edited the original)

[hamishmb]

I know what you meant anyway

I'll have a go now.