Running Multiple Monitor Threads for Reading Probes

[TerryJC]

This is by way of a question. I am currently building the Sensor & Control Assy (SAC) for the Lady Hanham Butts. This is different to the other SACs in that it includes three A/D Converters to allow for the monitoring of levels in each of the three Butts Groups that are located behind the Lady Hanham Building. These are connected to a single Pi, so I need to identify the best way to read all three without locking up the processor. I have attached the circuit diagram of the hardware.

If I was doing this in the 'old-fashioned way', I would simply run a loop and within this loop I would read the three A/Ds in succession, storing the results in three variables that could be passed to external functions. However, the HallEffectProbe() class in the /Tools/deviceobjects.py software currently starts a thread and then leaves it to get on with reading the levels while the rest of the program does it's own thing. Clearly, I could start three such threads, (one for each probe) and gather the data in the same manner as the current software does. However, I foresee a possible snag.

Python has a limitation called the Global Interpreter Lock, which apparently only allows one thread to execute at a time. What I'm unsure of is whether this will allow the software to sort itself out without getting itself into a twist or the opposite and cause everything to lock up.

The alternative that I can see is to read all probes in succession within the single thread. This has the benefit of not starting any new threads and is more like the 'old way', but means modifying the Class for all probes or creating a new Class just for the Lady Hanham SAC.

I'm not particularly skilled in Python or threading techniques, so could do with some help. Before I propose the code for incorporation into GitLab I'm going to need some Unit Test Code for my hardware and the current test software doesn't use threading, but with three converters, maybe it should.

Thoughts?

[hamishmb]

I would suggest using threading in the classes as it currently exists. The GIL won't cause everything to lock up (as long as there aren't any other issues - for example race conditions), it will just mean we can only fully utilise one CPU core with Python.

Of course, this is just a suggestion, and there are other solutions that will work, but they may require re-implementing the framework as it is built on the premise of concerns being separated in different classes and different threads.

[TerryJC]

Hamish,

By 'using threading in the classes as it currently exists', do you mean instantiate the class three times; once for each probe?

[hamishmb]

I do.

[TerryJC]

It's been a while since I asked this question because I've had other distractions since I asked it. I hope (finally!) to complete the build of the Lady Hanham Butts SAC this weekend and have started to build a modified set of software files to allow me to do this. As this progressed some new questions came to light:

New config.py for the Lady Hanham probes

Below is a fragment of the updated config.py that I have produced:

Code: Select all

    #Settings for the G3 site (client pi behind the Lady Hanham Building).
    "G3":
        {
            "ID": "G3",
            "Name": "Lady Hanham Butts Pi",
            "Default Interval": 15,
            "IPAddress": "192.168.0.3",
            "HostingSockets": False,
            "DBUser": "test",
            "DBPasswd": "test",

            #Local probes.
            "Probes":
                {

                    "G1:M0":
                    {
                        "Type":             "Hall Effect Probe",
                        "ID":               "G1:M0",
                        "Name":             "Lady Hanaham Butts Probe (G1)",
                        "Class":            Tools.deviceobjects.HallEffectProbe,
                        "ADCAddress":       0x48,
                        "HighLimits":       (0.07, 0.17, 0.35, 0.56, 0.73, 0.92, 1.22, 1.54, 2.1, 2.45),
                        "LowLimits":        (0.05, 0.15, 0.33, 0.53, 0.7, 0.88, 1.18, 1.5, 2, 2.4),
                        "Depths100s":       (0, 100, 200, 300, 400, 500, 600, 700, 800, 900),
                        "Depths25s":        (25, 125, 225, 325, 425, 525, 625, 725, 825, 925),
                        "Depths50s":        (50, 150, 250, 350, 450, 550, 650, 750, 850, 950),
                        "Depths75s":        (75, 175, 275, 375, 475, 575, 675, 775, 875, 975),
                    },

The entries for the G2 and G3 Probes are identical with the names changed to protect the innocent.

Things to note:

1. The Pi (lhbuttspi) is identified as G3, but it caters for probes, float switches and devices in Butts Groups 1 and 2 as well as 3. To save confusion, we could call it something else, (say G123).
2. There is a new item in the dictionary definition for the Probe; "ADCAddress". This defines the address of the associated A/D Converter on the I2C Bus. G2 has the address 0x49 and G3 has 0x4B.
3. The default address for the ADS1115 board if the ADDR pin is left unconnected is 0x48, which is why we haven't needed to pass it before.

Questions on this:

1. Does the naming convention seem OK?
2. Does the overall approach seem OK?

Implementing the Monitor Threads

Below is a fragment of the current code in devicemanagement.py:

Code: Select all

    # Create the I2C bus
    i2c = busio.I2C(board.SCL, board.SDA)

    # Create the ADC object using the I2C bus
    ads = ADS.ADS1115(i2c)

So now we have an object called ads which is subsequently used in the ManageHallEffectProbe() Class:

Code: Select all

     def __init__(self, probe):
        """The constructor, set up some basic threading stuff"""
        #Initialise the thread.
        threading.Thread.__init__(self)

        #Make the probe object available to the rest of the class.
        self.probe = probe

        #Create a lock (or mutex) for the A2D.
        self.ads_lock = threading.RLock()

        # Create four single-ended inputs on channels 0 to 3
        self.chan0 = AnalogIn(ads, ADS.P0)
        self.chan1 = AnalogIn(ads, ADS.P1)
        self.chan2 = AnalogIn(ads, ADS.P2)
        self.chan3 = AnalogIn(ads, ADS.P3)

        self.start()

My thinking is that I need to modify this Class to accept the ADCAddress definition from config.py and then pass it to the ads object so that the correct A/D is read. My problem is that I'm a bit uncertain how to do that.

Please note that this question relates to the way in which I need to get the address into the ADS object and not exactly what I have to do with it once I've got it. I did find a reference to that sometime ago by looking at the code in the Adafruit module as I recall, but I've got to dig that out again. Also, I do understand that the ADCAddress will have to be read in before it can be used and I assume that will occur in coretools.py, which I will also have to get my head round.

Any thoughts on all of this?

[TerryJC]

OK. I've found the documentation for the Adafruit Library at https://circuitpython.readthedocs.io/pr ... ml#ads1115.

This tells me that the API definition for the ADS1115 Class is:

Code: Select all

classadafruit_ads1x15.ads1115.ADS1115(i2c, gain=1, data_rate=None, mode=256, address=72)

This shows where the default address comes from, since decimal 72 is 0x48. Other sources tell me that I should create the ads object as:

Code: Select all

ads1 = ADS1x15(address=0x48)
ads2 = ADS1x15(address=0x49)   # or whatever is an appropriate address

which implies that I need to create three separate ads objects in devicemanagement.py and then pass a reference to those objects into the ManageHallEffectProbe() Class. Or is there a better way?

[PatrickW]

Rather than putting the ADC address in the config file, I think I would give each ADC a zero-indexed index number. Then, you can put the ads objects into a list:

Code: Select all

ads = [
    ADS1x15(address=0x48),
    ADS1x15(address=0x49), # or other appropriate address
    ADS1x15(address=0x50)  # ditto
]

I assume ADS1x15() is happy to instantiate an object even if there's no ADC attached with the given address. If not, then you might need some exception handling code, which may or may not complicate the creation of the list.

HallEffectProbe will need a set_adc(index) method, which sets, e.g., self.adc_index.

Then, in ManageHallEffectProbe, you should be able to have:

Code: Select all

self.chan0 = AnalogIn(ads[probe.adc_index], ADS.P0)
self.chan1 = AnalogIn(ads[probe.adc_index], ADS.P1)
self.chan2 = AnalogIn(ads[probe.adc_index], ADS.P2)
self.chan3 = AnalogIn(ads[probe.adc_index], ADS.P3)

coretools.setup_devices() will need

Code: Select all

device.set_adc(device_settings["ADCIndex"])

in the part where it sets up hall effect probes.

If you are set upon using the I²C address rather than an index, then you could put the ads objects into a dictionary instead of a list (and substitute "address" for "index" in the variable naming scheme):

Code: Select all

ads = {
    "0x48": ADS1x15(address=0x48),
    "0x49": ADS1x15(address=0x49), # or other appropriate address
    "0x50": ADS1x15(address=0x50)  # ditto
}

This seems to me like a reasonable enough way of doing it, though Hamish might have a better idea.

[TerryJC]

Rather than putting the ADC address in the config file, I think I would give each ADC a zero-indexed index number. Then, you can put the ads objects into a list:

The problem with that is that it is a complete departure from the approach that we've used for every other configuration item in the system. I'd like to talk that through with Hamish before I go too far down that path.

I assume ADS1x15() is happy to instantiate an object even if there's no ADC attached with the given address. If not, then you might need some exception handling code, which may or may not complicate the creation of the list.

No. The program will barf if one of the A/D Converters is missing, but I had already realised that. There is already an exception handler to trap this situation, because one of the failure modes is a broken A/D or faulty wiring in the I2C circuit. I was thinking along the lines of using a conditional in devicemanagement.py when the objects are created to set up either one or three ADS objects, depending on the Site ID, (eg one for Sump, Wendy, etc and three for Lady Hanham). However, see below.

Then, in ManageHallEffectProbe, you should be able to have:

Code: Select all

self.chan0 = AnalogIn(ads[probe.adc_index], ADS.P0)
self.chan1 = AnalogIn(ads[probe.adc_index], ADS.P1)
self.chan2 = AnalogIn(ads[probe.adc_index], ADS.P2)
self.chan3 = AnalogIn(ads[probe.adc_index], ADS.P3)

That's sort of what I had in mind, but with the address, not an index, being passed into ManageHallEffectProbe() each time it is instantiated. Presumably, if it is possible to pass the address into the ads object, I wouldn't need to create three of them as detailed above.

If you are set upon using the I²C address rather than an index, then you could put the ads objects into a dictionary instead of a list (and substitute "address" for "index" in the variable naming scheme):

Actually, config.py is already a dictionary. It is read into the program in corettols.py

[PatrickW]

Rather than putting the ADC address in the config file, I think I would give each ADC a zero-indexed index number. Then, you can put the ads objects into a list:

The problem with that is that it is a complete departure from the approach that we've used for every other configuration item in the system. I'd like to talk that through with Hamish before I go too far down that path.

Ahh, I think the difference between how we're thinking might be that I was thinking "we could have a number of ADCs, with addresses assigned in well-defined sequence" whereas you're probably thinking "we could have a number of ADCs, with arbitrary addresses". In my approach, the indexes are basically equivalent to the addresses, but only if the addresses are non-arbitrary.

Because the hardware is standardised, it hadn't crossed my mind that the addresses could be arbitrary.

If arbitrary addresses are needed, then I think there are two approaches:

1. Have devicemanagement.py read the config file and create the ads objects needed on that particular Pi, as you suggest. I don't like this: it creates tight coupling between different parts of the code, it will duplicate some of the existing code for reading configuration and setting up device objects, and it will put a blob of complicated logic in devicemanagement.py.
2. Have ManageHallEffectProbe instantiate the ads object itself. It can easily be told which address to use, and this approach takes advantage of the existing device setup code rather than duplicating it. This might be a good solution. Need to make sure the instantiation is thread-safe.

I think I made an unchallenged assumption that there must be a good reason why the ads object is currently created outside of ManageHallEffectProbe. That assumption may well be wrong.

Then, in ManageHallEffectProbe, you should be able to have:

Code: Select all

self.chan0 = AnalogIn(ads[probe.adc_index], ADS.P0)
self.chan1 = AnalogIn(ads[probe.adc_index], ADS.P1)
self.chan2 = AnalogIn(ads[probe.adc_index], ADS.P2)
self.chan3 = AnalogIn(ads[probe.adc_index], ADS.P3)

That's sort of what I had in mind, but with the address, not an index, being passed into ManageHallEffectProbe() each time it is instantiated.

Bearing in mind this I wrote it for non-arbitrary addresses, this code would work, unchanged, with the address instead of an index, given that a dictionary instead of a list is defined to contain the ads objects.

(I was thinking of making the index/address an attribute of HallEffectProbe instead of passing it to ManageHallEffectProbe as an argument, hence using "probe." to access it, but clearly it makes more sense to just pass it in as an argument. I think I was getting my thinking mixed up with the device class: We can't add a new argument to the constructor for HallEffectProbe, because coretools.setup_devices() requires all the device classes to have the same constructor but that's not an issue for ManageHallEffectProbe.)

Presumably, if it is possible to pass the address into the ads object, I wouldn't need to create three of them as detailed above.

That depends on where you are creating them. If you are creating them at the top of devicemanagement.py, then of course you need to create all the objects that are needed on that Pi, but if you are creating them in ManageHallEffectProbe then you only need to create one. (Because there will be three ManageHallEffectProbe objects, each with a different I²C address for the ADS.)

Actually, config.py is already a dictionary. It is read into the program in corettols.py

What I meant was that you would create three ads objects at the top of devicemanagement.py, where currently just one is created. But rather than assigning each to its own variable, or putting them into a list, you would put them into a (new) dictionary that's defined right there in devicemanagement.py, using the I²C address as the dictionary key. Then, ManageHallEffectProbe can use the dictionary key (i.e. the I²C address) to easily access the correct ADS object. Otherwise, if you defined three separate variables to contain the objects, you would have to have a conditional statement in ManageHallEffectProbe to select decide which variable to access. This may all be moot though, as mentioned.

[hamishmb]

Just making a note to say I've seen this. I'll reply properly soon (tomorrow hopefully). It'll probably be long as there's a lot to unpack here. The main thought I have right now is that as long at we try to follow the ideology of not hard-coding values and keeping ad much configuration in config.py as possible, that's good.