Abort Sequence Failed to Abort!

Sequence Generator
1

Hello all,

Im assuming this will be classified as a general bug as its not specific to 2.5.0.0 / 2.5.0.1

Having a play in the house today using ‘Ascom simulator’ as the mount, PHD2 2.6.0 was using the ‘Simulator’ as a camera and ‘on-Camera’ as the mount. I ran a test sequence with M33 as the target born from the mosaic & framing wizard to slew only without any rotations etc.

On starting the sequence, SGP got as far as firing up PHD2 , connecting its equipment and began calibrating, SGP had ‘Resuming the auto guider (forcing re-calibration)’ on the screen. Then, I realised I forgot something so I aborted the sequence.

What happened then was that SGP changes the button on the sequence module to Aborting and greyed it out. PHD2 continued with the calibration and then began the simulated guiding and SGP’s PHD graph module continued to add the RA & Dec lines produced in PHD as it should under normal circumstances. SGP then stayed in this state forever and didn’t actually abort the sequence.

SGP is still sitting attempting to abort some time later so I manually stopped PHD from guiding and manually disconnected the equipment and closed it. Usually if I have ever stopped PHD in it’s tracks while a sequence is running the sequence recovery box fires up and attempts to recover the sequence (which also has an option to Abort) but this didn’t come up either, so here we are stuck in suspended animation so a forced stop was required as I was unable to disconnect my equipment In SGP as it was all greyed out.

:cry:

From what I see, SGP on abort, should have stopped the PHD calibration, told PHD to disconnect it’s equipment and then bring up the usual end of sequence box with the usual countdown to run the end of sequence actions etc.

Although it wont be complete, I am attaching the log for this one providing it saved ok. Im sure I reset the sequence and ran it again in this log so the second run at the end is where I attempted to abort just after running it.

sg_logfile_20160111171835.txt (422.1 KB)

Cheers all
Paul

2

Paul,

This is the same issue that a number of us have seen and reported in another thread. Ken had requested for a way to reproduce the problem using simulators and it sounds like you have done it!

Andy

3

That’s a good find - Perhaps we can get to the bottom of this issue now - Well done Paul!

4

I really do hope it’s indicative of the same issue. @pscammp would you mind posting the sgf file that produced the issue?

@swag72 unfortunately your log is the one that looks different from the rest. I’m not sure if a fix for this will help you, but we’ll see. Your log shows the mount never returning control to sgpro, while the others show a mysterious freeze not associated with anything in particular.

5

That’s just typical eh Ken? Oh well, I’ll keep my fingers crossed.

6

@ Andy -

Sorry, didn’t notice the other thread, should have looked harder, as a beta tester for many years with other software, and I have had this happen to me a few times, I will make it my own personal project to lock down the cause of this one should the current supplied files be no good.

@ Ken -

I’m attaching a file called M33.sgf, this is pretty much what I was running at the time of the attempted abort. I had modified the file’s single event for exposure duration and image quantity when I re-ran the sequence, just after hitting go I remembered I hadn’t Saved the modified sequence. This caused me to abort with the intention of saving and re-running the sequence again but the obvious happened. Couldn’t load it up in the forum so I’ve Drop boxed it.

Public Link:

https://dl.dropboxusercontent.com/u/4938813/M33.sgf

Let me know if this is any good to help find the cause, if it’s not I’m going to setup a new directory called ‘Lets squash this once and for all’ and dedicate it to locking down the cause of this freeze up. Cant really contribute much to SGP’s community but if I can help find the cause of this I’ll be a happy camper for sure.

:smiley: :smiley: :smiley:

STOP PRESS

I can confirm that the above M33 file reproduces the Abort freeze as I’ve just ran it again just now. A couple more things have also come to light while doing so though so listen up, follow the directions below to see if you can reproduce also:

  1. Set up a temp Profile in PHD2 to use the Simulator as the camera and ‘On-Camera’ as mount (don’t use an aux mount or anything for AO), use 5.2 for Pixel size and 400mm for Focal length and 0.5 for guide rate - All PHD2 setting are defalt except for these just in case this makes a difference

  2. Might need a temp SGP profile to load up the above PHD2 profile to connect to, also, in case the M33 file doesn’t automatically do that, check control panel first before running the sequence to ensure PHD2 is loading the temp equipment profile you just made

  3. In SGP, connect the camera - ‘Simulator’ (use 3040 x 2016 as resolution and 7.8 as pixel size and leave show stars unchecked) then, connect Mount - EQMOD ASCOM Simulator, leave everything else unconnected

  4. The M33.sgf file when loaded and connecting to the EQMOD mount sim may not be PARKED - Click park the mount in EQASCOM at the bottom if not before continuing.

  5. In control panel - Auto Guide - Ensure the ‘Re-Calibrate auto guider when target changes’ checkbox IS CHECKED just in case this is important

  6. Hit ‘Run Sequence’ - Note PHD2 fires up and connects to the temp profile and the mount sim slews to wherever M33 is

  7. Wait for SGP to write up the following message in the status bar at the bottom: Resuming the auto guider (forcing re-calibration) - watch PHD2 and ensure it has picked a guide star and has begun calibrating before the next step - DONT WAIT until SGP is guiding or dithering for the Abort, once PHD2 begins guiding and dithering starts, SGP will abort successfully

  8. Hit ‘Pause Sequence’ - in the box that comes up choose ‘Abort the current sequence’ and tick the ‘Run end of sequence options’ then hit OK before guiding starts

  9. Note that SGP’s status bar statement changes to 'Starting and calibrating the auto guider, Please wait… Also note that PHD2 is still continuing with the calibration and will begin to guide once complete. (looks to me like SGP is on a tangent to a separate area of code here)

  10. SGP then puts up an Error box with the following message: The auto guider could not be auto started, please start the guider manually and try again (Untrue - We are guiding now !)

  11. Clicking OK on the Error box then changes the status bar message to: Sequence aborted (auto guider is not running) - (looks like SGP branched to a different area of code for the last two status bar messages and as a result has lost scope as to the current ‘We are successfully guiding’ situation - Of course I could be talking complete crap)

And there we go - Sequence module still says ‘Aborting’ and is greyed out, it wont let me disconnect gear as they are all greyed out in the sequencer too, PHD2 is still happily guiding & updating the PHD graph in SGP - Incidentally, if you choose ‘Exit’ from the File menu SGP will tell you there is still gear connected and will close but SGP never fully recovers from the Aborted state.

Here are the logs for the simulated session:

SGP:
sg_logfile_20160112172736.txt (71.9 KB)

PHD2:
PHD2_GuideLog_2016-01-12_172802.txt (4.6 KB)

Regards
Paul

7

Paul @pscammp

Thank you very much for this diligent write up of the issue. While I believe this is a real issue, I am not sure if it descriptive of the locks ups @Andy (mid sequence) and @swag72 (mount problems) are seeing. The most interesting hint toward a solution to the issue @Andy is seeing is that an apparently frozen sequence finally continued on its way when it lost connection with PHD2. @swag72 sees a problem when end of sequence code parks the mount and the ASCOM code never returns control back to SGPro. I think I know how to fix that issue, but the fix is pretty invasive and changes holistically how SGPro interacts with all of ASCOM.

8

OK, well here’s good new bad news:

  • Good: This was a bug, but only for 2.5 code. It was a bug with brand new code and I’m almost certain it has been fixed.
  • Bad: I don’t think it will make one bit of difference in the other issues mentioned above (especially since at least one of them was reported against the 2.4 code base)
9

Ken, Thanks for tracking it down!
Andy

10

Thanks to Ken and Paul for spending time on trying to get to the bottom of this… Lets see how it goes.

11

Shame !

Never mind, so that’s one bug gone even though it was a new one, that’s always good right ?

what threads describe Andy and Swag’s issues then, I like a good challenge as the clouds are in

Regards
paul

12

It is good and we thank you for taking the time to reproduce and detail it.

You had posted in that thread a couple days ago, but it’s here:

13

Ow Damnation Ken, how embarrassing LOLOL

I’ll have a detailed read a play & post in there if I come up with anything interesting.

Regards
Paul

14

Nothing embarrassing about a developer finding and fixing bugs. It’s not finding and not fixing that’s embarrassing.

Chris

15

Had another occurence of this on the 14th: clouds drifted over and rain was forecast so I hit Pause/Abort Quit Immediately and had End of Sequence Options ticked.

Roof closed, mount parked, however SGP showed Abort all the time and interface became unresponsive, ccd did not warm up. I had to use Task Manager to stop SGP.

I thought I’d post the log but I’m not sure if it will reveal anything new for you. I’m using a QSI ccd BTW, not sure if that is relevant. Ver 25, PHD2dev09.

https://dl.dropboxusercontent.com/u/32050952/sg_logfile_20160114200305.txt

Thanks,

Barry

16

Hello all,
This Abort failure issue is sure a pain in the butt, the thing that is noticeable is that it seems to happen in different situations, under differing circumstances but pretty much ends with a similar result for all who experience it. This will always make it difficult to find the cause. I have intermediate programing experience with Visual Basic so I know myself that when trawling through 4500 lines of code for a bug is not easy. I have no doubt that SGP is rather larger though so I don’t envy the Guys. LOLOL

So im trying to look at this now as a single cause within SGP’s code as opposed to the understandable notion that there may be multiple bugs, each of which is solely responsible for each of our own unique described circumstances.

On the grounds of my uneducated assumption I have created a SGP file which consistently reproduces one instance of said ‘Abort Failure’ every time a sequence of events is performed, my hope is that the ‘Main Men’ can use the following attachments to identify why the abort failure is happening and ultimately solve it. Further, if the cause of this particular ‘Abort Failure’ can be squashed then my hope is that it will also solve it automatically in other circumstances, nameable the situations where Swag, Andy and Barry have experienced theirs.

Here’s to the possibility of saying goodbye to this terrible disease !

So Here we go for Bug Squash - Try 1

One thing which Ken has previously stated is that using 100% ASCOM Simulators rules out any complications of differences in peoples kit, it means that all of us can load up the sims and be all in the same boat. Operating systems are not so easy to reproduce as some are on Win 7, some on Win 10 some on MAC’s etc. So here is my Basic setup which was used to run this SGP file for reference:

Op System: Windows 7 Home Premium 64bit
ASCOM Version: ASCOM 6.2 freshly installed
SGP Version: 2.5.0.1
Camera Sim: ASCOM V2 Simulator
Mount: EQMOD ASCOM Simulator

I have no doubt that all of us have our own unique Profiles set up in SGP so here is a possible stumbling block to attempting to reproduce issues so I have set up a test Profile with absolute minimum kit which you can paste into your relevant folder and apply to the sequence, this profile uses only the V2 camera sim and the EQMOD ASCOM Simulator for the mount, ‘Park on sequence complete’ and ‘Stop tracking on sequence complete’ are the only options enabled here as when ‘I’ experience the abort failure it tends not to park the mount. So download the following file from my dropbox:

minimum.sgp
https://dl.dropboxusercontent.com/u/4938813/Minimal.sgp

Place this file in the following location:

C:\Users\ [pscamm] \AppData\Local\SequenceGenerator\Sequence Generator Pro

(where I have [pscamm] this will be a unique name on your computer)

Also download the following file so you can load up the test sequence I used:

https://dl.dropboxusercontent.com/u/4938813/Abort_Failure.sgf

** Steps To Reproduce **

We are going to do this in 3 stages so here’s stage 1

  1. Fire up SGP 2.5.0.1

  2. File->Open Sequence and load up the ‘Abort_Failure’ file - You should have one target M33 - I’d be curious if anyone experiences an ‘Unhandled Exception’ when loading this file ???

  3. Apply the ‘Minimum’ profile to the sequence

  4. Open up the V2 Camera’s settings and change the ‘Type’ to RGGB and it’s resolution to 1600W 1200H

  5. Connect the Camera and the Mount Sim

  6. Run the sequence all the way to the end, watch the Mount sim to confirm it slews to M33 and back to Park at the end of the sequence. Its only a 1min 40sec sequence

Nice an simple, your mount sim should have parked at the end of the sequence

Stage 2

Here we are going to Abort the sequence at some point

  1. Close the image in the view and reset the sequence using Sequence-> Reset Sequence Progress

  2. Run the sequence again and wait until SGP is taking the exposures

  3. Once SGP begins to take ‘ANY’ exposure click ‘Pause Sequence’ and select ‘Abort’ & check ‘Run end of sequence actions’ & click OK before the exposure completes

Sequence should have stopped pretty much immediately and the mount should have parked successfully ending with ‘Sequence Aborted’ in the status bar

Stage 3 - The Big One

  1. Close the image in the view and reset the sequence using Sequence-> Reset Sequence Progress

  2. Run Sequence & wait until the mount begins Slewing to M33

  3. WHILE MOUNT IS SLEWING - Hit ‘Pause Sequence’, select Abort & check ‘Run end of sequence actions’ - OK

If you get what I get…

‘Run Sequence’ button greys out and has ‘Aborting’ written on it. The Freeze is upon us. SGP never becomes fully functional after this point. Mount continues to slew to it’s target (which is understandable I suppose) but the ‘Parking Mount’ never comes, the mount never moves to the home position and there’s no way to reset the sequence.

Conclusion of TRY 1

Left over night imaging or generally unattended in this situation will possibly see your scope hitting your mount due to the fact that the RA is still tracking in spite of the deliberate abort. Exactly WHEN we aborted this sequence is very significant as it proves that if an ABORT occurs while your camera is exposing then all will be fine, ABORT it while the scope is moving and SGP will always end in this Frozen situation. You will also experience the same Abort Failure if you attempt to abort while PHD2 is calibrating or dithering, again.

  1. SGP will successfully Abort while it is exposing

  2. SGP will fail to Abort successfully if…The mount is slewing…PHD2 is calibrating…Dithering is in progress.

Lets look for a minuet at what happens when you are guiding and a cloud moves into the frame and breaks your imaging run, Recovery will probably kick in and will repeatedly try to restart guiding by picking a star & recalibrating to resume guiding, this is exactly one of the states when an abort will cause SGP not to be able to complete it’s abort procedure, thus you will see this frozen result which you cant recover from.

Here is the abort when initiated mid exposure:

[16/01/2016 16:53:55] [DEBUG] [Main Thread] User requested sequence abort…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] ASCOM Camera: Attempting to abort exposure…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] ASCOM Camera: Exposure aborted…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] ASCOM Camera: Blocking until image is aborted…
[16/01/2016 16:53:55] [DEBUG] [Camera Thread] ASCOM Camera: abort message received…
[16/01/2016 16:53:55] [DEBUG] [Camera Thread] ASCOM Camera: Attempting to abort exposure…
[16/01/2016 16:53:55] [DEBUG] [Camera Thread] ASCOM Camera: Exposure aborted…
[16/01/2016 16:53:55] [DEBUG] [Camera Thread] ASCOM Camera: Blocking until image is aborted…
[16/01/2016 16:53:55] [DEBUG] [Camera Thread] ASCOM Camera: Image is aborted…
[16/01/2016 16:53:55] [DEBUG] [Camera Thread] SGM_CAMERA_CAPTURE complete…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] ASCOM Camera: Image is aborted…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] Run event requested sequence abort…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] Clearing timed monitoring events…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] Checking RunEndOfSequenceEquipmentOptions, force = True
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] In RunEndOfSequenceEquipmentOptions
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] Stop telescope tracking…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] ASCOM Telescope: Setting tracking state to False
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] Parking telescope…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] ASCOM Telescope: Park message received.

And here it is when initiated while the mount was slewing to M33:

[16/01/2016 16:58:42] [DEBUG] [Main Thread] User requested sequence abort…
[16/01/2016 16:58:44] [DEBUG] [Main Thread] PopulateDataModel: Transferring view to the data model…
[16/01/2016 16:58:44] [DEBUG] [MF Update Thread] Performing serialize…
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] Scope reports it is done with synchronous slew, verifying…
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] Scope has completed slewing
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] DoEventGroupChange: Slew complete
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] DoEventGroupChange: Complete
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] Attempting to find next event…
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] Current event[0] frame count: 0/10…
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] Looking at event[0]…
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] Found event event[0] with remaining frames.
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] Event[0] frame count: 0/10…
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] Getting first event (0)…
[16/01/2016 16:59:07] [DEBUG] [Main Thread] PopulateDataModel: Transferring view to the data model…
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] Running capture event…
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] ------------- Starting capture frame for event[0] -------------
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] Sending commands…
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] SetFlatBox: Frame Type is Light
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] TEMP - Current Event2: 0
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] TEMP - Current Event10: 0
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] ASCOM Camera: Attempting to abort exposure…
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] ASCOM Camera: Exposure aborted…
[16/01/2016 16:59:07] [DEBUG] [Sequence Thread] ASCOM Camera: Blocking until image is aborted…

as you’ll see in the above, there is no image in progress to abort as we were mid slew, once the slew completed SGP should really have then run the following:

16/01/2016 16:53:55] [DEBUG] [Sequence Thread] Run event requested sequence abort…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] Clearing timed monitoring events…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] Checking RunEndOfSequenceEquipmentOptions, force = True
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] In RunEndOfSequenceEquipmentOptions
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] Stop telescope tracking…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] ASCOM Telescope: Setting tracking state to False
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] Parking telescope…
[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] ASCOM Telescope: Park message received.

But it didn’t get that far

Here is the LOG associated with this three stage test:

sg_logfile_20160116163527.txt (186.4 KB)

Hopefully this makes some sense and is helpful in resolving this issue

Regards
Paul

17

@pscammp

Thank you so much for taking a deep dive here. That said… this is killing me. Following your instructions to the letter, clicking abort during slew showed no issues at all. I am using EQMOD sim 1.28m.

No exception, loaded fine.

I got the continuation of the slew that started, then status bar updates to say parking mount, then updates to say “Sequence Aborted”.

Can’t reproduce the slewing issue, will work on the PHD2 calibrating / settling / dithering tests.

Unfortunately, this is the same bug that I already fixed in reply 8 of this thread.

18

Executing a sequence abort during PHD2 calibration, returns an untrue message that says PHD2 could not be calibrated, but the sequence did actually abort…

19

Ken,
Thanks for looking at this…AGAIN, I know this is killing you but it’s also killing me and annoying some others, the common denominator has to be here somewhere, I wont stop until we find it and we will you know, in the end it WILL present itself. Just ran this file again and the same result I always get was the result, same on my Laptop too which is even stranger. It has to be either ASCOM related or further down under the hood.

A few things may I ask ?

  1. Just curious, What operating system were you using when you did the test, mine was Win 7 64 as stated previously ?

  2. I know you usually spend all your time asking this, but, any chance you could load up your Log when you attempted to Abort while the slew was in progress, im curious as to the difference in my log to your log, if you don’t mind ?

  3. Is there a way to return ASCOM to absolute default setting so it’s as if its never been install on my system before, I want to rule out any corruption in my ASCOM system or my settings within it as it stands ?

  4. Poking at the possibility of either something missing or corrupt on my system under the hood, apart from the basic operating system, what other software/resources are required on my system to ensure SGP has everything it needs to function correctly, things like .NET Frameworks for example etc. I know that when installing ASCOM on Win 7 you have to manually enable .NET Frameworks 3.5.1 for it to function correctly ? Is there any other notable things which need to be present which MAY not be automatically present when installing Win 7 or which may not be enabled by default like the ASCOM thing ?

  5. My EQMOD is also 1.28m

  6. What exactly is the following lines doing, my logs always freeze at the upper one and never get to the second one
    when my abort fails under said circumstances:

[16/01/2016 16:53:55] [DEBUG] [Sequence Thread] ASCOM Camera: Blocking until image is aborted…

[16/01/2016 16:53:55] [DEBUG] [Camera Thread] ASCOM Camera: abort message received…

Many Thanks Ken

Keep the faith !

Regards
Paul

20

I’ve tried your sequence and it seems OK. Abort while the mount is slewing to the target seems fine. It takes a while because the mount has to complete the slew, then complete the park.

Does EQMOD have log files that would show what’s happening from it’s perspective. The Camera V2 simulator does of course.

We know there is something non standard about your setup because of the problems you had with the old camera simulator. Maybe there is something that affects this.

You may get more help with your ASCOM installation on the ASCOM-talk group.