Double end of sequence being fired on unsafe condition met


#1

Hello all! I have encountered an issue in SGP 2.5.1.17 where if the safety driver triggers an unsafe condition two threads (the safety thread and sequence thread) try to end the sequence and park the scope. The safety thread executes first and then 4-5 seconds later the sequence thread tries to do the same thing. I also get two end of sequence dialogs that pop up, one right after the other (if I do a dismiss). The double park results in the mount getting confused and it stops and never does fully park as it should at the home position. The mount is an MI-250 with Gemini-2 controller using Gemini.NET of course.

The odd thing is that this use to work in older versions. Before I would only get one end of sequence dialog and one park command issued. It seems the threads are not communicating to each other as they should and they are both trying to end the sequence instead of one or the other.

Log file is here: https://www.dropbox.com/s/xjoimpihv9217gt/sg_logfile_20160922222508.txt?dl=0

Thank you Ken and Jared!

Best Always,

Daniel


#2

@stellarskies

Thanks for the report… this issue has been fixed in the next beta (2.5.2.2).

Not as odd as it may seem. Sometimes other fixes in other areas have unintended side effects (like this one). Also, in a multi-threaded application, bugs due to timing issues are not that uncommon… sequence works one night and not the next, just because of timing issues microseconds apart.


#3

Thanks Ken, I stated that wrong, I meant to say it is odd because I thought it was working in 2.5.1.7 and older versions. :slight_smile:

Thanks as always and I hope you are having a good weekend!

Best Always,

Daniel


#4

@Ken, our friend is back… Tonight I hit some clouds which triggered an unsafe condition and I got two end of sequence executions back to back. It was in the middle of autofocusing and it stopped it as it should have but the dual end of sequence events caused havoc on the mount as it parks in a position not anywhere close to its home position. It gets confused by the second park and stops right where it is at that moment in time.

Here is the section of the log that shows the double fire:

[03/13/18 00:28:15.731][DEBUG] [Safety Thread] Adding sequence level notification: Shutting down due to Unsafe Conditions.
[03/13/18 00:28:15.731][DEBUG] [Safety Thread] Sending Notification: Error - Shutting down due to Unsafe Conditions.
[03/13/18 00:28:15.731][DEBUG] [Safety Thread] Shutting down due to Unsafe Conditions…
[03/13/18 00:28:15.731][DEBUG] [Safety Thread] Clearing timed monitoring events…
[03/13/18 00:28:15.731][DEBUG] [Safety Thread] Checking RunEndOfSequenceEquipmentOptions, force = True
[03/13/18 00:28:15.731][DEBUG] [Safety Thread] In RunEndOfSequenceEquipmentOptions
[03/13/18 00:28:15.739][DEBUG] [Focuser Move Thread] Focuser move call complete
[03/13/18 00:28:15.794][DEBUG] [Sequence Thread] EventMarker 4 - t:Messier96WideField (0); e:0; f:8; o:* aborted *
[03/13/18 00:28:15.794][DEBUG] [Sequence Thread] EventMarker 10 - t:Messier96WideField (0); e:0; f:8; o:* aborted *
[03/13/18 00:28:15.795][DEBUG] [Sequence Thread] Run event requested sequence abort…
[03/13/18 00:28:15.795][DEBUG] [Sequence Thread] ********* Run post sequence *********
[03/13/18 00:28:15.795][DEBUG] [Sequence Thread] SGPro capture cal frame mode is OFF…
[03/13/18 00:28:15.795][DEBUG] [Sequence Thread] Clearing timed monitoring events…
[03/13/18 00:28:15.795][DEBUG] [Sequence Thread] Checking RunEndOfSequenceEquipmentOptions, force = True
[03/13/18 00:28:15.795][DEBUG] [Sequence Thread] In RunEndOfSequenceEquipmentOptions

Here is the full log file for the night:

https://cp.sync.com/dl/fdd1619b0#k7gcpydk-4yiu5599-dj2twuiy-a9yu8mrc

Best Always,

Daniel


#5

Bumping this as with the lastest release this is still happening. Tonight clouds set an unsafe condition and two end of sequence events occurred back to back. I would not be so persistent on this if it wasn’t for the Gemini control system going bonkers getting two park commands back to back as detailed in the log above.

This causes all sorts of odd problems with the Gemini I and Gemini II systems. Tonight after the second park command the mount just kept moving at center speed to the other side of the pier, past the park position. To stop this behavior I trick it by issuing a GoTo via the Gemini.NET driver first. The GoTo often moves the mount to a part of the sky the target is not in at all so I then stop the slew, again via the Gemini.NET driver. Finally, I issue a park at home command from the Gemini.NET driver.

This situation is recoverable but if I am not watching closely something bad could happen. The main thread or sequence thread needs to yield to the safety thread and it is not doing that currently.


www.mainsequencesoftware.com