Software tasks to do after merging our branches (Done)

[PatrickW]

It doesn't particularly matter to me. It also depends on the order that we merge things, but I'm going to make a new forum topic for that. You and Terry and I may need to have a call to discuss that - I have a feeling it'll be difficult to have that discussion on here.

I can't imagine there is very much to discuss about it. It just comes down to the order of the merging, and any justifications that are to be made one way or the other about that. I am happy to see it done either way.

I could push my branch onto GitLab. I would like to squash my commits though, because I started out using one email address and then decided I would rather not publish that address in a git repository and changed it to a different one, so I need to make sure that some of my earlier commits are not published. I'm a little bit unsteady on my feet when it comes to using git in a non-local manner, though, since I've not had much practice. (Can you advise me?)

Ah I see, that makes sense. I haven't got much experience squashing commits together, but we'll need to do that when we merge I guess, so I'll do some research. I usually work online - I worry that if I only have a local copy I'll somehow wind up losing it Supposedly pushing to a separate GitLab branch will be as simple as:

Code: Select all

git remote add origin [email protected]:wmtprojectsteam/rivercontrolsystem.git (if you haven't already done this)
git push origin <your-branch-name>

And then the new branch will appear online.

After this, new commits can be added with a simple:

Code: Select all

git push origin <your-branch-name>

I do have backups. I don't just have one copy of my work.

I've used git with a remote before, but not extensively and never with branches and merging. So, I feel pretty confident pushing things to a remote, but I'm a bit out of my depth with things like merging and rebasing. And git seems to be a minefield of terminology that sounds like it should mean one thing but actually means something entirely different. I find that the man pages do not give me a very clear picture of how everything is supposed to work. I need more diagrams! I have been meaning for ages to look for a book on the subject that has some practical exercises to work through. (I've just added it to my shopping list, before I forget again.)

I've just re-read the man page for git merge --squash, and realised that I am probably using the wrong terminology when I said "squash", because it seems to be talking about something different than what I thought I was talking about. What I want to do is erase some of the history of my commits, so that it looks as though all the changes contained in those commits were made in one big commit. I think I might be talking about rebasing, perhaps? I don't know. I need to spend some time to work through it and figure out what I have to do to it. I will prioritise that, because it sounds like you would find it really helpful to have sight of my work at this point.

[hamishmb]

I can't imagine there is very much to discuss about it. It just comes down to the order of the merging, and any justifications that are to be made one way or the other about that. I am happy to see it done either way.

Okay, sounds fair. I'll make us a new thread for this soon and I guess we can go from there.

I do have backups. I don't just have one copy of my work.

I've used git with a remote before, but not extensively and never with branches and merging. So, I feel pretty confident pushing things to a remote, but I'm a bit out of my depth with things like merging and rebasing. And git seems to be a minefield of terminology that sounds like it should mean one thing but actually means something entirely different. I find that the man pages do not give me a very clear picture of how everything is supposed to work. I need more diagrams! I have been meaning for ages to look for a book on the subject that has some practical exercises to work through. (I've just added it to my shopping list, before I forget again.)

I've just re-read the man page for git merge --squash, and realised that I am probably using the wrong terminology when I said "squash", because it seems to be talking about something different than what I thought I was talking about. What I want to do is erase some of the history of my commits, so that it looks as though all the changes contained in those commits were made in one big commit. I think I might be talking about rebasing, perhaps? I don't know. I need to spend some time to work through it and figure out what I have to do to it. I will prioritise that, because it sounds like you would find it really helpful to have sight of my work at this point.

I wasn't trying to suggest you didn't have backups, just that I'd find a way to mess it up

I think squashing is something built upon the more low-level rebasing, but yeah I don't understand it that well either. I'm hoping GitLab can help us. If you find any good resources please let me know because I need to read up on this too . Also please bear in mind that there is no major rush, but yes it would be helpful to see some of the code.

[hamishmb]

I've done a few of these tasks now, but will wait for:

1. The NAS box to be deployed on site.
2. Patrick's code to be merged.

Before doing some of the rest of these.

[PatrickW]

I have now pushed my work to GitLab, in the branch stagepi_logic_dev.

I ended up needing to run git rebase -i before pushing to GitLab. That enabled me to fix my email error. Didn't get it quite right, because I lost my most recent changes, but I was able to re-apply them because I had them stashed. Not quite sure what happened there.

I think I should now do a merge from master to stagepi_logic_dev, to bring stagepi_logic_dev up-to-date with master, then see what needs changing to make the Stage Pi logic work with the changes that have occurred in master since I branched off from it, change my logic to use logiccoretools.py instead of sockets, and then look at writing a setup function to go with the logic function, and plumbing it in to config/main.

Then I need to catch up on those unit tests and we can look to merge stagepi_logic_dev into master.

I feel like I'm forgetting something, but that seems like enough to be getting on with.

[hamishmb]

List updated to include new things we identified during the meeting

I'll also make us a new thread for the deployment plan/order in a few minutes.

[PatrickW]

Hamish, the Git merge itself shouldn't be too painful. I merged master into stagepi-logic-dev after use-database was merged into master, and since then have been mostly working on new files (new from the perspective of master), so there shouldn't be many conflicts to resolve.

One important change I haven't committed on stagepi-logic-dev was to uncomment VALVE12 in config.py. This change has to be made in order to deploy the Stage Pi logic, but I didn't want to complicate the merge for the sake of such a trivial change, knowing that config.py has been changed quite a bit in master. Obviously the outcome of the merge has to include VALVE 12 being uncommented in config.py, but I don't care whether that occurs in master or stagepi-logic-dev.

Looking at items on the list in the first post, with respect to how they impact the merge of the branches:

• "Move some of the existing unit tests into Patrick's new modules (which ones?)": I think the unit tests to be moved would be the ones that test control logic. If the control logic is being moved, then the tests will have to be amended to point to the new logic location anyway, so I would suggest moving the tests when that amendment occurs.
• "Refactor run_standalone()": I think this was just a general item regarding the state of run_standalone. I'm not sure it necessarily blocks the merge. The Stage Pi logic runs fine without refactoring this, although it did require a minor change so that run_standalone executes a control logic set-up function if defined in config.py. It does need properly refactoring at some point though.
• "Control logic stuff": this is probably the bulk of the complication in the "merge" process. I say "merge" in quotes, because it's not really anything to do with the Git merge. It's more like a separate changeset to to bring the listed items in line with the new logic. This can be done post-merge, although it's probably something I would do in a separate branch rather than directly in master, then merge that branch back.
• "Sockets stuff": I haven't concerned myself with this. I assume you know what this is (or was) about. I don't imagine it has much bearing on the merge.
• "Improve unit test coverage" I'm not sure this impacts the merge much.
• "Rename master branch to "main" or similar (BLM)" Sounds good to me.

A lot of the hard work was done when you created logiccoretools. That was really the keystone for this merge.

There are a few things I had in mind for the Stage Pi logic, which don't come into play at this point. I'm not sure this thread is necessarily the right place to record them, but I don't want to let them slip my mind again, so I'll note them here anyway:

• There was some mention of restoring the logic's state from the database in the event that it has to be restarted. This is not implemented, and I'm not convinced it is necessary. It would need a time limit; if the stored state is only 5 minutes old, then it's probably worth restoring it, but if two days have passed then we may as well figure it out from scratch, because all the water will have moved around. Not a priority as far as I'm concerned.
• I would quite like to have the logic output a "current action" for the pi/software status update that goes into the database, since there is provision in logiccoretools to accept a "current action" field. Currently the logic only outputs pi status and software status. This is a nice-to-have item only.
• While testing, I have realised that the logic could handle sensor faults better (i.e. less obstinately, more robustly, and more openly). I do intend to work on this, and have set out a specific plan for doing so, but, as I realised in the meeting today, it shouldn't block the merge.
• As discussed in the meeting,today, we may later want to add hysteresis. This would be done by adding more states to the state model, so that there are both "rising" and "falling" states for each water level. (Might need to rethink the naming style though; some of the state names are ridiculously long already!) I think I did consider doing it this way when developing the state model, but I concluded that it wouldn't be necessary. We'll see.

[hamishmb]

Hamish, the Git merge itself shouldn't be too painful. I merged master into stagepi-logic-dev after use-database was merged into master, and since then have been mostly working on new files (new from the perspective of master), so there shouldn't be many conflicts to resolve.

One important change I haven't committed on stagepi-logic-dev was to uncomment VALVE12 in config.py. This change has to be made in order to deploy the Stage Pi logic, but I didn't want to complicate the merge for the sake of such a trivial change, knowing that config.py has been changed quite a bit in master. Obviously the outcome of the merge has to include VALVE 12 being uncommented in config.py, but I don't care whether that occurs in master or stagepi-logic-dev.

Yeah, I think that should probably be done post-merge.

Looking at items on the list in the first post, with respect to how they impact the merge of the branches:

• "Move some of the existing unit tests into Patrick's new modules (which ones?)": I think the unit tests to be moved would be the ones that test control logic. If the control logic is being moved, then the tests will have to be amended to point to the new logic location anyway, so I would suggest moving the tests when that amendment occurs.
• "Refactor run_standalone()": I think this was just a general item regarding the state of run_standalone. I'm not sure it necessarily blocks the merge. The Stage Pi logic runs fine without refactoring this, although it did require a minor change so that run_standalone executes a control logic set-up function if defined in config.py. It does need properly refactoring at some point though.
• "Control logic stuff": this is probably the bulk of the complication in the "merge" process. I say "merge" in quotes, because it's not really anything to do with the Git merge. It's more like a separate changeset to to bring the listed items in line with the new logic. This can be done post-merge, although it's probably something I would do in a separate branch rather than directly in master, then merge that branch back.
• "Sockets stuff": I haven't concerned myself with this. I assume you know what this is (or was) about. I don't imagine it has much bearing on the merge.
• "Improve unit test coverage" I'm not sure this impacts the merge much.
• "Rename master branch to "main" or similar (BLM)" Sounds good to me.

A lot of the hard work was done when you created logiccoretools. That was really the keystone for this merge.

Agreed on all of that and makes sense.

There are a few things I had in mind for the Stage Pi logic, which don't come into play at this point. I'm not sure this thread is necessarily the right place to record them, but I don't want to let them slip my mind again, so I'll note them here anyway:

• There was some mention of restoring the logic's state from the database in the event that it has to be restarted. This is not implemented, and I'm not convinced it is necessary. It would need a time limit; if the stored state is only 5 minutes old, then it's probably worth restoring it, but if two days have passed then we may as well figure it out from scratch, because all the water will have moved around. Not a priority as far as I'm concerned.
• I would quite like to have the logic output a "current action" for the pi/software status update that goes into the database, since there is provision in logiccoretools to accept a "current action" field. Currently the logic only outputs pi status and software status. This is a nice-to-have item only.
• While testing, I have realised that the logic could handle sensor faults better (i.e. less obstinately, more robustly, and more openly). I do intend to work on this, and have set out a specific plan for doing so, but, as I realised in the meeting today, it shouldn't block the merge.
• As discussed in the meeting,today, we may later want to add hysteresis. This would be done by adding more states to the state model, so that there are both "rising" and "falling" states for each water level. (Might need to rethink the naming style though; some of the state names are ridiculously long already!) I think I did consider doing it this way when developing the state model, but I concluded that it wouldn't be necessary. We'll see.

I reckon #2 is a particularly good idea. The existing control logic I wrote doesn't really use any of those records, and I think it'd be good to use them as much as possible - every little thing can be helpful for diagnosing/debugging.

[hamishmb]

Okay, this merge is done now, subject to testing.

I have removed some items from the list at the start of this thread as I don't think they were important/important enough to be worth blocking the merge over. I have added them to my private To-Do list as they concern work for me really, seeing as I wrote the code involved.

The next set of things to pay attention to are at viewtopic.php?f=43&t=272&p=4461#p4461, where I'll post a status update shortly.

[hamishmb]

Also noting that you may want to copy your above list somewhere else Patrick, if you haven't already - we (or at least I) will probably forget it exists if it's not somewhere more prominent, maybe in a topic of its own.

[PatrickW]

Re: my list, I think that's mostly a list for my own benefit, so I've just now made a copy in my personal notes and I'll add mention of its points onto my whiteboard, which I use as a kind of medium-term working memory for projects. If I copy the list into a new topic, I will simply forget that the topic exists! I don't find forum threads very memorable.

Thanks for reminding me though, because I had forgotten about it. Actually, I hadn't forgotten it, but I'd managed to convince myself that I must have imagined writing it, because I couldn't find it anywhere.