Implementing Stage Pi's control logic and various newbie questions

[PatrickW]

Hi all. Sorry for my slow start on the Stage Pi algorithm.

I spent the end half of last week digging into the problem.

I'm now at a point where I think I need to discuss this with others who have a good understanding of how the software has been designed.

Here are my thoughts and current understanding. I'm hoping someone can fill me in on the details I've not quite grasped and correct any misunderstandings I may have arrived at. Sorry: this is rather lengthy!


State machines

It seems to me that the control logic for each Pi, combined with the state of the devices in the system (sensors, actuators) essentially forms a finite state machine.

For example, Sump Pi has states for "sump level getting high", "sump level okay", "sump level good", "sump level getting low" and "sump level very low".

It occurred to me that this would lend itself to being implemented using the State Pattern. I think I'll do that, because it should reduce code duplication between different control logic functions.

The State Pattern would also make it easier to keep track of whether water levels are falling or rising. By having a variable holding a pointer to the current state object, you can have separate states for when you're pumping water out of the sump or flowing it in.

You could also have separate states and associated behaviours to deal with not knowing the water level after a restart.


Stage Pi states

For the initial implementation of the Stage Pi control logic, without the matrix pump and without further specification than the Control Strategy Document Iss 0.4, I could only think of two states for the Stage Pi control logic state machine: "idle" and "flow G6 into G4".

They're derived from the state of the system by the following (pseudocode) logic:

Code: Select all

if G4_full or G6_empty:
    state = idle
else:
    if G6_full:
        state = flow_G6_to_G4
    else:
        if G4_level < G4_fill_limit_from_G6:
            state = flow_G6_to_G4
        else:
            state = idle

G4_fill_limit_from_G6 is a constant that I thought might be useful because we can't yet pump from G4 to G6. If G4 is nearly full, and G6 is not full, then it seems to me that we should not top up G4 beyond a certain limit, in case Sump Pi needs to pump water from the sump into G4. However, if G6 is full then we should always top up G4 to prevent G6 overflowing, because the certainty of G6 overflowing is worse than the heightened chance of G4 overflowing.

Without G4_fill_limit_from_G6, the logic is ridiculously simple:

Code: Select all

if G4_full or G6_empty:
    state = idle
else:
    state = flow_G6_to_G4

Of course, each state encapsulates certain actions (i.e. fully open or fully close V12).


One Pi-state-oh, two Pi-state-oh, three Pi-state-oh, four, five Pi-state-oh, six Pi-state-oh, seven Pi-state-oh, more?

Seeing that I'd only come up with two states for Stage Pi led me to think "Why does Sump Pi need so many states: What have I missed?" In both cases, the state is derived from the same kind of data: the water level in two vessels.

The answer I came up with is that Sump Pi needs extra states largely because it is using them to adjust the reading_interval, which allows it to take infrequent readings when it is not in imminent danger of letting the sump overflow or fall empty, and frequent readings when it is.

But (and correct me if I've got this wrong) reading_interval is global and synchronised across all of the Pis, so that all the other algorithms must use the reading interval defined by Sump Pi.

So, if I write a stagepi_control_logic() function, then it will be slaved to the reading_interval set by sumppi_control_logic(), based on the level in the sump. This seems to diverge from the documented control strategy.

Why is that? I'm sure it must have been derived from some requirement, but the reason is not immediately obvious. Does it have to do with synchronising the readings between devices? My initial assumption before I properly read that part of the code was that reading_interval was local to a specific Pi. I notice there is also a comment "#TODO: Do we still want this?" in main.py in the vicinity of the code that handles the reading_interval, but it's not clear to me what "this" is in that context! Are there thoughts of not synchronising the reading_interval after all?

Given the complexity added by varying the reading interval (rather than screeching along at full speed the whole time), I wondered what justified it. I came up with these possibilities:

• In testing, it was found to reduce the power consumption of the Pis.
• It saves storage space for the log files.

Were either of these a justification for the design?


Variable flow rates?

Another thing that Sump Pi does with its many states is adjust how far open V4 is. I can't figure out why this should be. At first glance, it should not be necessary at all; the valve could just be fully opened whenever the sump needs topping up and fully closed whenever it doesn't. Like a thermostat calling for heat.

I can think of a couple of reasons why the flow rate might be varied:

• Reducing the flow rate brings the rate of change of the water level down to something that can be monitored more easily with a limited sample rate as the sump approaches its optimal level.
• If we don't vary the rate, then we will end up with fairly rapid on/off cycling and it's better to instead find a constant flow rate somewhere in between that causes the sump level to stabilise, rather than to adjust the valve position very frequently. (i.e. we want the valve to be open just enough to continually replace the losses from the river)

Are either these the reason why it was done?


Matrix pump

I also gave some thought to the way the matrix pump might be used by the control algorithms. I see that there is already a means of locking access to valves and pumps so that only one Pi can control them. We don't currently have an abstraction of the matrix pump as a whole, though.

Presumably we can create a Matrix Pump device class, and then whichever Pi is physically connected to the motor for the pump part of the matrix pump can also serve as the Pi which is assigned the "Matrix Pump" device. Then, that Pi can claim exclusive control of the four matrix pump valves and other Pis can send it instructions to "pump up" "pump down" and "close", rather than worrying about opening and closing the right valves themselves and having to figure out what state the previous Pi left the valves in.

Has any thought been given to what should happen if a Pi crashes while holding a lock? I would think the locks should time out unless re-claimed periodically. This is particularly important if we're going to store the locks in non-volatile storage (i.e. in a database). (I haven't looked deeply into the locks, so perhaps we already do this.)


Lady Hanham Pi control logic

Another thing I don't understand is the reasoning behind the sequential filling and draining of the Lady Hanham butts that's documented in the Control Strategy Document. There are two ways in which I fail to understand this.

Firstly, I don't see how water can be pumped between any of the Lady Hanham butts groups. Since they are connected only via pipes and valves and are at roughly the same height, their levels will just equalise. Perhaps that's the intended result, but I don't get that from the document.

(Incidentally, I think there might be a typo in the Control Strategy Document, because water is never put into G1, only taken out of it. If there is an error, then it's probably in the line "Water in G1 will be used to fill G2 by opening V1." on page 7.)

Secondly, I don't see the point in filling any non-G4 butt group to the brim when other non-G4 butt groups are much less full, since that just creates opportunity for that particular butt group to overflow when the rain comes. I don't think I've completely grasped the idea of how this is supposed to work. Surely the goal should be to keep all the butt groups at roughly the same level. Or, more properly, at the same "time until full". We could calculate a "time until full" that's comparable across butt groups from the total capacity and current water level of the butts and the area of the roofs supplying them.

(To know the actual time until full in units of time, we'd need to know the actual rainfall rate from the sky per unit roof area, or calculate it by seeing how the water level changes over time, but for comparison between butt groups we don't need to know the actual rate; we just need to assume it's the same across all compared butt groups.)




I feel like a lot of this might be better discussed in person, but I will never remember all the points if I don't write them down, and I figured this was probably the best place to write them!

[hamishmb]

State Machines

That sounds like a good idea to me.

Stage Pi States

This seems useful as this means the status fields in the database could have some meaning to the software as well as being human-readable.

Reading Interval/many pi states

The reading interval is planned to eventually be set by the NAS box. The not-very-helpful note I left there was about the sump pi eventually needing to continuously check for new reading intervals from the NAS box, as all the other pis currently do for Sump Pi.

I think space saving was the initial reason, but I'm not sure we gave it that much thought.

Variable Flow Rates

I don't remember why we did this.

Matrix Pump

I believe Terry and I talked about this, but we weren't sure how to do it or exactly what we wanted to do.

The idea with the locks was that they would be released by the NAS box after a certain amount of time had passed, if the controlling pi didn't release them. Also, if the controlling pi goes down, the NAS box should detect it and free the lock.

Lady Hanham pi control logic

I think the idea was the use G4 as a buffer of sorts, so we could use it as a go-between for transferring water between the Lady Hanham butts.

[PatrickW]

In addition to Hamish's response above, I think Terry answered my questions yesterday regarding the sequencing/ordering of butts group filling and emptying.

I believe my main misunderstanding in that regard was to think that, for example, G3 was supposed to be completely drained into G4 before G2 was ever touched, and G2 was then supposed to be completely drained into G3 before G1 was ever touched, and the reverse for filling.

Instead, it's a more abstract concept of a sequence in which the control logic can consider the butts groups, without the intention of implying that it must fixate on one butts group for an extended period.

Though, at the end of the day, the basic principle of keeping the G4 Wendy Butts filled up is the main consideration.

Thanks, I think that cleared it all up for me.

[TerryJC]

I meant to respond to some of these points after we discussed them at Tuesday's LUG Meeting, but the past day or so has been pretty hectic.

Variable Flow Rates

I don't remember why we did this.

There were two reasons for this. The first is that the design of the Gate Valves meant that we could Since the valve is opened and closed using a closed-loop servo mechanism, we can set any valve opening we want between 0 and 100% Clearly there is no benefit in tuning the flow rate that finely, so we settled on five positions; 0% (closed), 25% (low flow), 50% (half flow), 75% (fairly fast flow) and 100% (full flow).

The second reason for adopting variable flow rated is because when the pump is running (bearing in mind the size of the interconnecting pipes), it was felt that we might end up with 'hunting'. We thought that the Butts would fill so fast that by the time the software detected the 'Full' level, it would be too late to stop the flow before the source butts became empty. This scenario might never come to pass but if it did it would lead to water being pumped backwards and forwards ad infinitum.

Matrix Pump

I believe Terry and I talked about this, but we weren't sure how to do it or exactly what we wanted to do.

The idea with the locks was that they would be released by the NAS box after a certain amount of time had passed, if the controlling pi didn't release them. Also, if the controlling pi goes down, the NAS box should detect it and free the lock.

This I think will be resolved in development.

Lady Hanham pi control logic

I think the idea was the use G4 as a buffer of sorts, so we could use it as a go-between for transferring water between the Lady Hanham butts.

I think we thrashed this to death and I accept that there are lots of ways that this could be done. At the LUG Meeting Paul was arguing that the proposed system was far too complicated and all we needed was a ball valve on each Butts group to allow water to cascade down to the next group as each successive sector was filled and I argued that that would not allow us to pump water back up to the higher Groups. Paul then said that it was unlikely that we would ever want to pump water upwards and that may be true, but we have what we have and I don't think the system is necessarily a bad one and least we have the flexibility.

Anyway as the original proposer of the strategy, my contention is that the concept is reasonably logical, (once you've got your head round it) and fulfils the goals of the system:

1. Always have sufficient water available in the Sump.
2. Waste as little water as possible.