The last two nights (09/14/2016-09/15/2016 and 09/15/2016-09/16/2016) my sequences have failed. The failure seems to be due to autofocus and a stuck pixel that produces a hot column from 3 quaters the way up to the bottom of the frame and AF dark subtraction not really helping (of which I’ll make another post for). The failures happened at around 4am each night.
The problem I am outlining here is that once focus is gone, and phd can’t reacquire the guide star or clouds or whatever makes the Sequence enter recovery mode, is that after recovery mode is finished and cannot recover, the sequence option to “Run non-light subs even if sequence fails” (aka “Capture calibration frames even if the sequence fails to complete” i n the global settings) does not occur.
IE:
Target 1 (lights) failed
Target 2 (daks) did not run
This was not an issue for me in 2.5.1.X
Cheers,
Aaron
SGP log from the night of 09/14/2016-09/15/2016:
SGP log from the night of 09/15/2016-09/16/2016:
PHD guide logs (did not include the huge phd debug logs):
Neither of your sequences failed due to auto focus.
The sequence in the first log failed 2 times (and recovered successfully 2 times) due to PHD2 star lost messages. The sequence in the second log failed 1 time (and did not recover successfully (also due to a PHD2 star lost message). Both sequences show that to be the only reason for sequence failure.
As to why the dark frames did not run on sequence failure:
The first sequence: There was no cause to… the sequence did not fail and dark frames started normally.
The second sequence: I am not sure. Either:
Capture calibration frames on sequence failure was accidentally turned off
The dark events were de-activated
There is a bug that caused them not to run (because of the very specific way your sequence failed). I cannot see anything obvious, so I have added a bit more trace code to further narrow down what happens when this option is selected.
I’ll keep an eye on the settings, but am pretty sure it was enabled and the darks where checked with the pause symbol waiting for their turn. For both my memory is foggy as I find these things out first thing out of bed. The first sequence, I cannot recall clear enough if iniated the darks after waking up or if the recovery dialog was still open and waiting. The second sequence, though, was in its finished state. And I remember specifically looking at the setting in both the sequence window and global prefs, and it was checked. Same with darks.
I’ll keep a detailed log of what happens after a session if this happens again and will report back here.
Here is what happened this morning. Clouds came in around 2am. I manually aborted and reset sequence (preserved progress).
Confirmed the run cal frames setting is checked in both the sequence settings and SGP options.
I set my first incomplete target to Start capture at 3:41 am and end capture at 6:15 am
Second incomplete target set to end capture at 6:15 am.
Dark targets are checked (no start or end capture time defined).
PHD was open and not looping. I cannot remember if I had the gear connected or disconnected with 100% certainty, but I think it may have been disconnected because I may have dismissed a end sequence notification without clearing the run end of sequence options check box.
Sidenote: The gear connected state of PHD has not been a problem for the most part, but I’ve found recently that depending on if I aborted during the initial begin sequence routines (guesswork by the status displayed: PHD calibration, resume autoguider, etc.) that when resuming, SGP doesn’t seem to know that PHD was aborted from SGP and sends a command to dither expecting the autoguider to be running. Which creates an error message.
I click run sequence and I see the waiting for capture time to start status. I go to bed.
This morning I woke up and SGP was displaying a modal message box saying: “Sequence was aborted because the auto guider is required, but not running.”
I clicked ok. End Sequence notification message popped, sequence ended, dark frames did not start.
So, I’m not sure where the instruction to do the calibration frames is skipped, but it could be something to do with PHD communication, or SGP not understanding that the sequence has failed.
I’ve uploaded the nights user.config (I stripped the top 9 lines, thought there may be license info there), sgp logs, sg_ui_config, the .sgf file, and both the phd guide logs and debug logs here:
As always thanks for your hardwork and the awesome program that SGP is!
There are a couple things going on here. We try to perform most of the “fatal” sequence checks as soon as the button is clicked (even if there is a delay). Some of them, though, are deferred until the sequence actually begins (like this one). So… I need to think about how to handle failure here better. That is really the main issue. Dark frames were not captured because the sequence did not technically start.
Happened again last night (9/25/2016-9/26/2016). I went to bed at 12:00, at which point SGP was finishing a target’s HA event and moving to an SII event.
From what I can gather this morning:
-After finishing the SII event, the OIII even started.
-The AF routine settled on a value that was out of focus (hot column troubles).
-The event still started as it did not have a reason to fail yet.
-Guiding looked ok, and things proceed as normal, but the first 3 frames were out of focus (small donuts).
-Then something happened, I think clouds rolled in.
-I think recovery mode kicked in. At some point it succeeded in recovering the sequence.
-The next frames were focused better, but looks like it was shooting through hi clouds.
-At some point I think it failed to guide and went into recovery mode again.
Finally this morning I awoke and found that the SGP status was Run or Sequence complete, but the target for darks had never been initiated. I unchecked manually started the darks, but by this time, it was too bright out so the light leaks in my camera were wrecking my darks.
This looks like a bug… everything on the log indicates that 5 darks should have been taken after sequence failure (at 06:06:28), but no attempt to toggle the “cal frame mode” ever occurred. I have tried to recreate the conditions that led to this, but have not yet been able to do so successfully. So… as a temporary compromise, I have added more trace logging to the code that is supposed to toggle this mode. If you can get it to happen with the next beta (2.5.2.2), it should make location of the issue MUCH easier.
Working with 2.6.0.2, cal frames did not run after the clouds rolled in and the sequence recovery failed. Wondering if it’s my settings or a bug still?
In you logs I cannot find a sequence abort event that was not initiated by user button clicking. Every abort event seems to have this preceding it:
[12/29/2016 03:55:14] [DEBUG] [Main Thread] User requested sequence abort…
[12/29/2016 03:55:14] [DEBUG] [Main Thread] User elected NOT to run end of sequence options!
Were you actually up and playing around with this stuff at 3:55? I do not know how we can generate that particular log message without a button click.
Happy new year. Correct, I was awake at that time. The event I am refferring too is at around 07:00. Cal frames did not run after that, and I can not figure out why.
[12/29/2016 07:01:22] [DEBUG] [Sequence Thread] Adding sequence level notification: Something has gone wrong and the auto guider lost the guide star
[12/29/2016 07:02:23] [DEBUG] [Recovery Sequence Thread] Something bad has happened… attempting to recover the sequence (attempt 1)…
[12/29/2016 07:02:23] [DEBUG] [Recovery Sequence Thread] Recovery method using auto center has started…
