UPS and Safety Monitor

I couldn’t leave it half done and didn’t have much else on today so here it is:
Generic Safety Setup.zip (593.9 KB)

That will install a Generic Safety monitor which will monitor a file and check the first line for some text indicating the two states and use them to update the IsSafe property of the safety monitor

writing to the file can easily be done in a batch file with a line such as:
echo safe > safety.txt
or
echo unsafe > safety.txt

This writes the text to the file safety.txt, on my system that’s in the C:\Users,username>\ folder but could be elsewhere.

I can use it to change the colour of the safety light in SGP, I guess it should do what you want.

Have fun!

Hi Chris,

you’re amazing, thanks!
So, if I understand well, the Safety Monitor will only check the file by reading its content, which needs to be provided by the UPS or a script that monitors the UPS.

Currently I do so on logoff, but as soon as my batch file finishes, the computer goes down, which I can only delay by adding a long enough wait for the safety monitor to do its job. I can however only give a guestimate for the remaining battery lifespan, while the UPS has a good idea of it.

So, instead of the setting the unsafe mode I would like to monitor the UPS status as soon as it indicates that power is down, without setting the safety status to Unsafe as that will immediately cause End-of-Sequence options to be activated.

In an ideal world, the Generic Safety module would monitor the status of the UPS and only take appropriate action when battery lifespan is becoming too short. For this of course there are a few considerations:

  • a power failure should not cause immediate shutdown;
  • depending on the expected battery life the Safety Monitor should be allowed to overcome the power failure (I think my UPS has a 20 minutes lifetime, but need to test that).
  • My camera is set to warm up in 10 minutes, so the Unsafe status should be set 10 minutes (plus perhaps 2 minutes margin) before the UPS will fail (would be great if that is directly read from the camera settings in SGP).
  • In the meanwhile the sequence could continue, unless the remaining integration time of the current frame extends beyond the UPS’s remaining battery time.
  • What to do when power is restored, while End-of-Sequence options have been started?

Shutdown of the pc will be done by the UPS, depending on the UPS or Windows settings.

Now I do realise that this is a lot more than I initially asked for, but hope you want to consider it and I do appreciate your comments on my thoughts! For now I will start to use your Generic Safety Setup with this additional wait at the end of my batch file (as mentioned above) and see how that works.

Thanks a bunch!
best wishes,
Nicolàs

PS: just realised that when I use my .bat file to set the Unsafe status SGP is already killed… Will need WinNUT to do so… so more to think about…

Yes, that’s it.

Yes, in an ideal world. If the UPS were to write data to a file, the first line of which changed when the power failed then the Generic Safety Monitor (GSM) could monitor this file.

I implemented this solution because we didn’t have any information on how the UPS provides information about the power status. All I had was your information about what you were doing.

The problem I have is that I’ve strayed too close to a slippery slope, at the bottom of which is the swamp of unending support. I’m not going to risk falling down there. I’m not going to get sucked into a never ending cycle of speculating about how to detect a power failure, trying something, finding it doesn’t work and so on, indefinitely.

This leaves much of this to you. If you can detect the power failure and change a file then the GSM can pass this to SGP. That should be able to do all the close down that’s required.

Ideally SGP would signal that it had finished and that would trigger the rest of the close down process.

10 minutes warm up with a 20 minutes battery life seems a bit tight, can you reduce this? 5 minutes would be better than the power going off.

Hi Chris,

I appreciate your fear of the slippery slope, can’t blame you. :slight_smile:
I guess the 10 minute warm-up time is a default SGP setting, no idea where I otherwise got it from, will reduce it to 5 minutes. Thanks for this useful comment!

Maybe it is time to dive in the world of VB… :frowning:

cheers and thanks for your support so far,
Nicolàs

I’ve had a Good Idea, possibly verging on brilliant :slight_smile:

if you (or anyone else) are running on a laptop then the PowerStatus information available in the laptop can be used to set the safety state.

So I’ve added a checkbox to the GSM setup - “Unsafe if mains power is off”
With this checked SGP will get an unsafe notification within a few seconds of the power to the laptop failing. SGP can then can do all the shutdown while the UPS desperately provides power to everything else.

The laptop is not powered by the UPS while everything else is, maybe everything else is powered by a battery that’s got a float charger to keep it topped up. Lots of possibilities.

This is what the setup looks like, having an empty file name means that this is not checked.
image

I think this could a lot of applicability. It avoids the issue of needing to have multiple safety monitors for all the different UPSs and in many cases could avoid needing a UPS entirely.

Here’s the new installer
Generic Safety Setup.zip (594.5 KB)

Have fun.

Chris

Hi Chris,

thanks for this addition, especially for people in the field this will be quite useful. Coming weekend I will test to see if Windows-behind-a -UPS acts like a laptop-Windows, so perhaps the trigger works on my NUC as well.

Nicolàs

It looks as if a UPS should use the PowerStatus class to allow the system to manage everything so it will be well worth trying.

And of course everybody else. Once I get some feedback I’ll probably try to publish it on the ASCOM downloads site.

I tried it with SGP 2.6.1.114 (don’t have 3.x) and found that it would shut down a sequence, parking the mount, parking the dome and closing the shutter. The Camera cooler was set to warm up on sequence end but didn’t.

When the power failed I got a messaqe box giving me the option to cancel the shutdown and a 30 second countdown. I tried restoring the power, simulating a short power failure, but the shut down continued. It WBN if SGP could have seen the power restore and resumed.

This is with version 2 and I expect this has been updated for later versions.

Chris

Hi all,

I’m late to the party, but I had the same requirement for a remote site. I couldn’t find usable drivers for the ups and I hate chaining things that read and write files. So I ended up writing a safety driver that reads the Web page the ups drivers provide (in fact the config interface) which shows the status of the ups (an APC I think). It’s just a question of parsing the text.

The safety monitor also monitors other events such as loss of Internet connection, rain from a CW, wind, etc.

Could you do something like that to get the ups status?

The problem with that is that parsing the UPS web page will be UPS specific, and will change when the UPS vendor changes their web page. Fine for a single user but not for any sort of generic solution.

It would be really useful if people with a UPS could test this Generic driver to see if the UPS monitoring software uses the Windows PowerStatus object. If they do then a lot of people could benefit from this.

Hi Chris,

just tested the second version of your generic driver. Sadly enough my Windows 64bit NUC does not pass on the message that the UPS is running on batteries, not even when APC finally decides to shut down the computer, the Safety Monitor remains green. The only visible thing that changes is the PowerChute systray icon, but that too will differ between manufacturers and models.

Cedric’s suggestion is nice, but will only work for UPS devices that have a web-interface. My APC only has a USB connection and therefore no web-interface. I agree with Chris that reading the web-interface is not a generic solution. I have written several applications in the past that read web-sites, reading and analysing their contents for various reasons, and it is indeed a pain in the butt to keep those parsers compatible with the content provided.

Still I think that Chris’ second version is a very useful addition for those that image using a laptop.

Nicolàs

Hi Chris,

I think I found a good way to read the UPS status in a generic way. In the Windows EvenViewer the UPS logs its events:

  • When power fails and UPS switches to battery, the UPS sets event-id 174 (Battery backup transferred to battery power due to a blackout)
  • When power is restored and UPS is happy again, it sets event-id 61455 (Battery backup transferred to AC utility power.)

The only part being not generic are these two event-ids, but those can be read in the EvenViewer (pull the net-plug and insert again after a few seconds, run eventvwr from the taskbar, in the Event Viewer go to Windows Logbooks -> Application and look for the events left by the UPS.

So if you can read those events and could make two fields in the Generic Driver where the SGP user can enter these event-ids for power loss (I call it “almost-unsafe”, see below) and safe?
It would then be nice to have a third field with a configurable delay in minutes after which SGP should run End-Of-Sequence (EOS) options. On power loss the Safety Monitor should remain safe, but internally turn to almost-unsafe. If event-id 61455 is logged before the end of the delay, no action should be taken (Safety Monitor remains safe throughout the power loss and reverts from the internal almost-unsafe mode to safe). If the delay time expires without power being restored Safety Monitor can move from almost-unsafe to unsafe to allow EOS options.

If this is possible the following set-up should perform a proper shutdown:

  • Set-up the UPS to stay alive as long as possible;
  • Check how long the UPS will support the observatory at the current power consumption (e.g. 25 minutes) and at what time before batteries are drained the UPS will perform shutdown (e.g. 7 minutes);
  • Suppose we need 5 minutes to run EOS options, we set-up Safety Monitor to set the unsafe-trigger after 25-7-5 = 13 minutes of power loss (or a little less to stay on the safe side).

In this way EOS options are run 15 minutes after power loss, unless power is restored before this time. Once the EOS options run there is no point in reverting to safe-mode when power is restored, as SGP has already stopped the sequence and (partially) disconnected equipment.

Question of course is: is it possible to implement the above (so to monitor the eventlog)?

Nicolàs

Thanks for the event log info.

I’ve had a look at the event log on my laptop and can see events when I turn the power off and on. I get two System events each time:

  • Event 105, Kernel-Power - Power source change
  • Event 12, UserModePowerService - this is specific to my hardware.

The obvious things are essentially identical, the same events for power on and power off. Event 105 has EventData that reports if the AcOnLine is true or false. I think the SystemInformation property, which I am using, is already monitoring this event and I don’t see any value in duplicating Microsoft’s work.

Your events seem to be specific to your UPS, and it’s pretty obvious that different power management systems could do this in different ways.

Your request for a lot of additional monitoring and decisions is something I am going to leave to the application. It is in a much better position to know what it needs and provide UI to handle it. The driver will report the instantaneous power state.

I certainly wouldn’t gamble on the power lasting for any longer than neccessary, the 30 seconds or so that the current message box remains active seems about right to me. This should be easy to implement in SGP, if the power is restored then programmatically press the cancel button. Continuing with the sequence until the message box finishes may be worth while.

Hi Chris,

Event 105 is triggered on my laptop as well, number 12 is indeed specific for your laptop.

You wrote:

Your events seem to be specific to your UPS, and it’s pretty obvious that different power management systems could do this in different ways.

That was why I thought having them as input fields in the driver window. Obviously that would also require a pull-down to choose in which logbook they should be detected, while the implementation differs from monitoring event-id 105.

Anyway, I will be patient, hoping that one day this functionality will be an integral part of SGP. Until that time I will have to do with a hard shutdown of my cam in case of a power failure.

Cheers,
Nicolàs

The PowerStatus class has other properties relating to battery status and maybe they are being updated. Here is another version that logs the full PowerStatus properties and uses more of them to set IsSafe to false:

// make a series of decisions depending on the Power Status
if (ps.PowerLineStatus == PowerLineStatus.Offline)
    return false;       // not running on AC power
// these ignore the charging status, when charging they will return false until there is sufficient charge.
 if (ps.BatteryChargeStatus.HasFlag(BatteryChargeStatus.Critical) || ps.BatteryChargeStatus.HasFlag(BatteryChargeStatus.Low))
    return false;       // low or critical bsttery status
if (ps.BatteryLifePercent < 0.8f)
    return false;       // battery life percent lett than 80%
if (ps.BatteryLifeRemaining < 600)
    return false;       // less than 10 minutes

What this means is that if any of these things happen IsSafe will return false:

  • The mains power is off
  • The battery charge state is low or critical
  • The battery life percent is less than 80%
  • The battery life remaning is less thn 10 minutes.

The logging has been updated to record all of these.
What I’m hoping is that a UPS will record it’s battery state and that can be used to signal a shutdown.

Here is the new version
Generic Safety Setup.zip (594.8 KB)

Please try this and report what happens. It is important that you post logs. To do so check the Trace on checkbox. The logs are in Documents\ASCOM\Logs\ASCOM.GenericSafety.txt.

Chris

Hi Chris,

thanks for the update. I tested it and think the PowerStatus class is not set, all lines in the log read:

08:02:11.315 Connected Get True
08:02:11.315 IsSafe PowerLineStatus Online, batteryChargeStatus NoSystemBattery, BatteryFullLifetime -1, BatteryLifePercent 1, BatteryLifeRemaining -1

Straight from the beginning the SafetyMonitor icon in SGP turns red.
Nicolàs

The reason It’s always unsafe is the -1 for batteryLifeRemaining, this means unknown. I have fixed this but it doesn’t solve the problem that the UPS software doesn’t use PowerStatus. I guess it uses ACPI but I don’t see that and it isn’t practical for me to try to do anything blind. It needs a developer with a UPS.

It looks as if something using the PowerStatus, with setup so the unsafe conditions can be set will be useful so I’ll do that and leave it at that.

Hi Chris,

Once that is done, would you be willing to share the source code with us?

Kind regards,
Nicolàs

Hi Chris,

as your generic solution resulted in a driver that did not solve my problem, I decided to install VB yesterday. Now that was far from easy, which surprised me, but today I managed to build a Generic UPS SafetyMonitor following most of the specs I provided a few days ago.

afbeelding

The driver works with the events in the Windows Application Events Logbook (can be accessed via eventvwr).

There are three fields to be filled (the default values are those for an APC Back-UPS in combination with PowerChute):

  • the event-id that specifies when the UPS is on mains (to be determined for your UPS through the eventvwr)
  • the event-id that specifies when the UPS is on batteries (to be determined for your UPS through the eventvwr)
  • the amount of time to wait (in the hope mains will return) until we have to stop the sequence in order to be able to shutdown properly.

If mains returns with that specified time, the driver keeps the safe mode to safe. Only after the time has expired it will turn unsafe and will not return to safe, even if mains returns before the UPS shuts the computer.

The only thing not covered is the complete shutdown of the mount, but that is done using my MountCMD tool on computer shutdown.

The driver can be found on my server: http://www.dehilster.info/docs/GenericUPS-Setup.zip
The MountCMD tool is there too: http://www.dehilster.info/docs/MountCMD.zip

I have tested this driver with SGP doing its stuff with a ZWO ASI1600MM Cool, a 10Micron GM3000 mount and the slaved LesveDome, which all parked and warmed up properly. Hopefully other people on this list will test my Generic UPS Safety Monitor driver and give some feedback on how their UPS’s work. Please note that the driver will not work until the first image has been taken (that is a documented SGP feature). If you wish to postpone imaging, it is best to have two targets in the sequence: one with a single exposure directly taken, the next target being timed.

Nicolàs

Excellent! I’m glad you found a way to do this. It’s far easier if you have the hardware to hand.

You might consider letting the ASCOM people know, they can provide a link to your installer on their downloads site.

Like the seahorse logo.

As for the driver I wrote, I rewrote it to use the Windows SystemInformation.PowerStatus class and use a combination of the LinePower, BatteryStatus, Percent battery remaining and battery time remaining parameters. That will be useful for people using a laptop where the same events are generated for power on and off.

Hi Chris,

I have extended the driver to be able to monitor both the Application and System logbooks. In addition the messages left in there can be tested for a string, just like you had done.

So far it has been tested with two UPS: one from APC and one from CyberPower.

For more info and download of the most recent version, see my website.

Nicolàs

www.mainsequencesoftware.com