The software for our small deployment is complete!

[TerryJC]

OK. See you tomorrow at 2 pm.

[hamishmb]

*Pis

[TerryJC]

I worked that out.

[TerryJC]

OK.

I completed the updates as follows:

1. Finalised the wiring on the sumppi installation.
2. Installed the RASClock RTC onto sumppi and configured the software.
3. Mounted buttspi on a new baseboard and fully wired in the DC-DC Converters etc.
4. Set static IP addresses on each Pi.

I then connected the system up as we did last week and booted it up. The software seems to work, although it sometimes takes ages for the client to connect. The bad news is that the NTP Server doesn't seem to be working. I posted a question on the Dorset LUG List, but I've only had one reply so far, which didn't help.

I'll bring everything along tomorrow morning and see if we can make it work.

[TerryJC]

All,

Here is an update on progress on the Real Time Clock and NTP daemon after our testing yesterday morning:

RTC:
Yesterday afternoon I ran a few tests using hwclock and confirmed that the RASClock RTC didn't seem to be keeping time very well. However, hwclock has a system for drift compensation and I ran hwclock -c (which compares the difference between the RTC and the system clock) and watched the error reduce for around 20 minutes. I then shut down for the night and restarted a few minutes ago. The clock was dead right!

NTP:
Just before I shutdown yesterday, I ran date on buttspi and noticed that the clock was right. Subsequent to this, Ralph from the Linux User Group responded to my query with bunch of links and advice on how to debug the problem. One of these links was https://www.eecis.udel.edu/~mills/ntp/html/debug.html and if you look at the beginning of the second Section you will see:

Unless using the iburst option, the client normally takes a few minutes to synchronize to a server.

I did try iburst, but it didn't seem to help, but this morning, after I had started the system, I monitored date on buttspi and after around 10 minutes the clock synchronised with the RTC on sumppi!

So in summary; the RTC is working, NTP is working and we just have to find a way of speeding up synchronisation. Ralph had one or two ideas about that too, which included running 'ntpd -q' as part of the startup routine, which should synchronise clocks and then exit. I suspect that what we actually need to do is prevent ntpd from starting initially and then running that after a long enough delay to ensure that sumppi's NTP server has started.

I'll report back once I've made a bit more progress this afternoon.

[hamishmb]

Hi,

That's great! So if the RTC is working, and NTP delay can be fixed, I guess we're ready to install.

I was wondering what on Earth I could have done wrong with the NTP config

Hamish

[TerryJC]

That's great! So if the RTC is working, and NTP delay can be fixed, I guess we're ready to install.

We will be fairly soon I think. I can't get to WMT first thing tomorrow, but I should be able to get in later. We need to ensure that Clive has finished the box before we get too excited

I was wondering what on Earth I could have done wrong with the NTP config

Nothing; we were just a bit impatient. If buttspi was starting into an environment where the NTP Server was already running, I'm sure it would have worked fine.

[TerryJC]

All,

I've made some progress on the NTP front. Part of the problem is that we have to operate correctly when the power to all the devices is applied simultaneously. In this situation, everything is booting at the same time and nothing can really start happening until the router and both Pis have settled down. However, even when the buttspi is started after the other two devices have finished booting, it still takes up to 5 minutes for NTP to sync the clocks.

What I did this afternoon was to suppress the startup of the NTP daemon on buttspi until after sumppi has booted and then ssh into buttspi (so it has definitely booted too) and start NTP using 'sudo ntpd -q'. What i saw was this http://hadrian-way.co.uk/Misc/Butts_Pi_Screenshot.png.

I'm not sure why the server is sending out the KoD (Kiss of Death) packet and I've asked on the LUG List to see if anyone knows. It may be that it takes that long for the server to be happy with the time that it's serving up, in which case all we can do is to delay processing any levels until we can be reasonably confident that the data is valid. I have discovered that the server is comparing timestamps from the RTC and the system files and suppressing the transmission of a valid signal until they are consistent, but why they shouldn't be I'm not sure.

I'll post again tomorrow with another Sitrep.

[TerryJC]

Bad news this morning I'm afraid. I turned on the system to confirm yesterday's results and ran 'sudo ntpd -q' immediately after buttspi had booted, exactly as I did yesterday. The daemon reported its progress, exactly as before, but it exited without syncing. I tried a few more times with the same results.

I then checked the time on sumppi and the RTC had lost nearly an hour overnight. I can't understand this; the battery seems fine, it kept good time on Tuesday night, although it kept bad time last night and on Monday night.

I'm going to have to investigate this further before I can say that I'm happy with the networking solution. I'll put the RTC onto one of my Pis this afternoon and see if it keeps good time on that.

I'll keep you posted.

[TerryJC]

Sitrep:

I have some good news and some bad news. Bad news first:

The RASClock is still running erratically. I spotted an error that I made in the configuration file and fixed it, but it still keeps very poor time. Last night it lost nearly an hour in 11 hours and this afternoon it gained a minute in an hour. I have posted a query on the ModMyPi Forums, (where I bought the RASClock from last year) and eagerly await their suggestions. One thing that might be relevant is that the Setting Up Instructions only cover Raspbian Jessie and Wheezy and I've just updated to Stretch....

The second bit of bad news is that no-one has been able to say definitely why the NTP server is throwing out KoD messages for anything from 5 to 10 minutes before a valid time signal is transmitted. We know what it says the problem is (it thinks that the timestamps on packets and files are different to the timestamp on the transmitted message, but that only tells us what, not why. It is generally quicker if the server (eg sumppi) and the WIreless AP are started a few minutes before buttspi, but that isn't how the system will work and in any case we only gain about 4-5 minutes. The stuff I've found online all says that it will take 'a few minutes for the system to sync' and they're referring to timeservers that are Stratum 5 or better and are running 24/7, so it's not instantaneous even then.

The good news is that there is a workround. Thanks to Ralph on the Dorset LUG List, I am now aware of the command ntp-wait. If this is inserted into the boot sequence it stops the machine finishing boot up until NTP has synced. We still have to wait, but our software doesn't have to rely on arbitrary delays to ensure that the network is up and the clock is synced. In effect, we end up with a 10 minute boot-time (makes me nostalgic for some of the old Windows machines ). Of course, when they are synced, both machines have the same time, even though it might not be the right time

I'll be going into the WMT tomorrow morning to meet Clive and check that the Butts Pi installation can be mounted inside the box he bought. However, I will only try to get the system integrated with the pump etc if everyone is happy that it's worth a try. In that case, I feel we might have to allow quite a bit of time to do it and we might have to do it early on Saturday or fairly late after the Visitors have left.