Hi all,
I was wondering if any of you have some ideas for the software design for monitoring sensor values in the database.
The simplest way to use is is just call the methods of the DatabaseConnection class from the control logic functions.
However, I could also make a DatabaseMonitor class in monitortools.py. This would be similar to SocketsMonitor, but would use the database instead and could remember the previous reading, like the SocketsMonitors (and the normal monitors for local devices) do. It would essentially provide a layer of abstraction over the top.
I'm wary of overcomplicating things, but I do wonder if the abstraction layer might be useful. Thoughts?
Software design for monitoring database readings
Software design for monitoring database readings
Last edited by hamishmb on 12/06/2020, 14:54, edited 2 times in total.
Hamish
Re: Software design for monitoring database readings
I'll leave that to Patrick and yourself to decide.
Terry
Re: Software design for monitoring database readings
Patrick, this is the thing I was talking to you about.
When we talked about it on Tuesday, I think we decided that:
When we talked about it on Tuesday, I think we decided that:
- We could have a DatabaseMonitor class, but it's not clear how well it would work with polling the database, or if it would interfere with state modelling.
- So we'll have the main loop grab the readings for us, but by calling the methods in the DatabaseConnection class directly.
- If we encountered any issues with this (such as the main loop getting delayed by the NAS box response time), we could later change the model to move the delay into a different thread.
Hamish
Re: Software design for monitoring database readings
I was just in the middle of writing a reply, but your reply (Hamish) is much better anyway!
Yes, that sounds about right.
Just to clarify: the main loop has always been the part that grabs the readings (from SocketsMonitors). I don't think this was clear at the start of this thread.
With the sockets design, a monitor is needed because the readings are pushed, not pulled. There has to be a monitor thread ready to pick them up when they arrive.
With the database design, the messages are pulled from the database, and this can be done without a monitor.
If we need a monitor to deal with slow NAS box response to database requests, then the monitor would have to poll the database in the background at regular intervals, to make sure there are up-to-date readings available instantly. To be avoided if possible, but not ruled out if necessary.
I suggested that DatabaseMonitor should implement the same interface (i.e. the same method(s)) for grabbing readings as DatabaseConnection. Then it will be a drop-in replacement.
Yes, that sounds about right.
Just to clarify: the main loop has always been the part that grabs the readings (from SocketsMonitors). I don't think this was clear at the start of this thread.
With the sockets design, a monitor is needed because the readings are pushed, not pulled. There has to be a monitor thread ready to pick them up when they arrive.
With the database design, the messages are pulled from the database, and this can be done without a monitor.
If we need a monitor to deal with slow NAS box response to database requests, then the monitor would have to poll the database in the background at regular intervals, to make sure there are up-to-date readings available instantly. To be avoided if possible, but not ruled out if necessary.
I suggested that DatabaseMonitor should implement the same interface (i.e. the same method(s)) for grabbing readings as DatabaseConnection. Then it will be a drop-in replacement.
Last edited by PatrickW on 17/06/2020, 17:26, edited 1 time in total.
Re: Software design for monitoring database readings
Sorry, I didn't realise! 
Yes, that all sounds right to me. We could make the DatabaseMonitor implement the same interface as DatabaseConnection, but that would be in breach of the existing interface in BaseMonitorClass (which just has a get_reading() method and a few others), which is simpler and all that is needed, if we stick to the concept of there being one monitor for each device/probe we need to monitor (how it works with the sockets).
Realistically we could do it either way, and there would be minimal effort to change the main loop to make it work - I don't think it's an issue. Hopefully we won't need a DatabaseMonitor class at all though.
Yes, that all sounds right to me. We could make the DatabaseMonitor implement the same interface as DatabaseConnection, but that would be in breach of the existing interface in BaseMonitorClass (which just has a get_reading() method and a few others), which is simpler and all that is needed, if we stick to the concept of there being one monitor for each device/probe we need to monitor (how it works with the sockets).
Realistically we could do it either way, and there would be minimal effort to change the main loop to make it work - I don't think it's an issue. Hopefully we won't need a DatabaseMonitor class at all though.
Hamish
Re: Software design for monitoring database readings
I just had a quick look at the use-database branch on GitLab to try to make sure I am on the same page as you. It hadn't occurred to me before that monitor objects are specific to a particular system ID and probe ID, whereas a DatabaseConnection object is not.hamishmb wrote: ↑17/06/2020, 15:35 We could make the DatabaseMonitor implement the same interface as DatabaseConnection, but that would be in breach of the existing interface in BaseMonitorClass (which just has a get_reading() method and a few others), which is simpler and all that is needed, if we stick to the concept of there being one monitor for each device/probe we need to monitor (how it works with the sockets).
I retract my previous suggestion to standardise the interface between DatabaseConnection and DatabaseMonitor. These are not classes that are meant to be used directly by the control logic. They don't need to serve the needs of the control logic by providing a standard interface for it. That's the job of main.py's run_standalone() and the helper functions it calls.
Arguably run_standalone() should not have to deal with these socket/database/monitor details either, but there's a long-standing TODO comment to break run_standalone() up into smaller functions, which could be taken as an opportunity to resolve that. Regardless, the complexity is hidden from the control logic. That provides the kind of loose coupling I was vaguely fishing for, so it's all good.
Re: Software design for monitoring database readings
Okay cool
Yes, run_standalone() definitely needs a good refactor at some point, I just haven't decided how to do it yet. I'll probably do that next, seeing as I'll need to modify it anyway. But yes either way the control logic won't be coupled to any of this - that would make it hard to write new control logic methods.
We will need a way for each site to know what devices/probes it needs to get the readings for, so it knows what data to fetch from the database, but I think I can resolve that fairly easily with another slight change to config.py.
I might need to see your control algorithms to get that bit right though, so I'll leave that for the moment.
Yes, run_standalone() definitely needs a good refactor at some point, I just haven't decided how to do it yet. I'll probably do that next, seeing as I'll need to modify it anyway. But yes either way the control logic won't be coupled to any of this - that would make it hard to write new control logic methods.
We will need a way for each site to know what devices/probes it needs to get the readings for, so it knows what data to fetch from the database, but I think I can resolve that fairly easily with another slight change to config.py.
I might need to see your control algorithms to get that bit right though, so I'll leave that for the moment.
Hamish