Meridian Flip failure: SGP with APCC Pro (Mach1GTO)

lucam · April 21, 2020, 5:59pm

Description

After solving the Meridian Flip issues last month everything has been going well. Last night, I suffered a meridian flip failure again. The setup was the same as two nights prior when the same system executed a meridian flip on the same target flawlessly. I am a bit puzzled by what happened and would be grateful if @Ken and @rgralak could take a look at the logs.

Incidentally, a second system running exactly the same software combination (and versions) imaged without any issues for the whole night on the same target (different focal length, both pointed at M106) and performed a meridian flip successfully.

I launched a sequence this morning before disconnecting the equipment and again it flipped without a problem. It may just have been a gremlin but I would appreciate a second look.

Best,

Luca

Link to Logs

APCC log link: Dropbox - File Deleted

SGP log link: Dropbox - sg_logfile_20200420202115.log - Simplify your life

Useful Info

SGP 3.1.0.457
APCC Pro 8.1.0.9

Mach1 GTO CP4
Windows 10

Gringo · April 22, 2020, 8:55am

Greetings,

not much of a help, but I seem to have the same problem since a couple of days. The flips are failing from one night to the next on the same object. From the driver logs I see that the AP ASCOM driver reports “slew complete” after about one second when in fact it´s not. I wonder if something (Windows update) messed up the driver settings/configs.

Kind regards,
André

Posted already in another thread on the AP-GTO forums but got no response…

FWIW, here is a link to my logs:

https://www.dropbox.com/sh/2oooztnhimaxyan/AADL8fDKvP1gSMVzX_GPDfR0a?dl=0

Useful Info

SGP 3.1.0.457
APCC Standard v1.8.0.8

1100 GTO CP3
Windows 10 Pro

rgralak · April 22, 2020, 12:38pm

Your APCC/ASCOM logs indicate there were timeouts in communicating with APCC and the mount. It could be your system is overburdened or slowed down via a remote desktop application. Try increasing the timeout values in APCC and driver settings.

-Ray

lucam · April 22, 2020, 1:57pm

Thank you, @rgralak . I’ll give that a shot. A timeout issue would explain the intermittent nature of the problem. I agree with Andre’ that the slew complete message is way too soon and indicative on a communication issue. However, when I look at the logs I don’t see any timeout messages at the time of the missed flip (maybe i don’t know how to interpret the log file correctly).

To make sure we are looking at the same section of logs, here is the SGP relevant log section:

[04/21/20 00:03:24.132][DEBUG][Sequence Thread][SQ;] Checking if Meridian Flip is needed
[04/21/20 00:03:24.135][DEBUG][Sequence Thread][SQ;] Telescope is on the West side of the mount
[04/21/20 00:03:24.136][DEBUG][Sequence Thread][SQ;] Meridian Flip needed, Hour Angle >= Degrees Past To Flip: 12.3973166666666 >= 11.75
[04/21/20 00:03:24.136][DEBUG][Sequence Thread][SQ;] Running blocking meridian flip…
[04/21/20 00:03:24.136][DEBUG][Main Thread][SQ;] Adding sequence level notification: Running automatic pier flip…
[04/21/20 00:03:24.142][DEBUG][Sequence Thread][SQ;] Sending Notification: Status - Running automatic pier flip…
[04/21/20 00:03:24.298][DEBUG][Pier Flip Thread][SQ;MF;] Meridian Flip: Starting Meridian Flip Procedure
[04/21/20 00:03:24.298][DEBUG][Pier Flip Thread][SQ;MF;] Meridian flip: Skipped initial plate solve, use target as reference…
[04/21/20 00:03:24.301][DEBUG][Pier Flip Thread][SQ;MF;] Meridian Flip: Stopping the Auto Guider
[04/21/20 00:03:25.606][DEBUG][Pier Flip Thread][SQ;MF;] Meridian Flip: Sending Telescope command to execute meridian flip
[04/21/20 00:03:25.617][DEBUG][Telescope Thread][SQ;MF;] ASCOM Telescope: Pier side is West
[04/21/20 00:03:25.617][DEBUG][Telescope Thread][SQ;MF;] ASCOM Telescope: attempting pier flip using slew
[04/21/20 00:03:25.652][DEBUG][Telescope Thread][SQ;MF;] Telescope: Slewing to J2000 RA: 12.2673078928437 (12h16m02.31s) Dec: 47.329056422518 (47°19’44.60")
[04/21/20 00:03:25.652][DEBUG][Telescope Thread][SQ;MF;] Telescope: Slew received J2000 coordinates, mount requires JNOW, converting…
[04/21/20 00:03:25.653][DEBUG][Telescope Thread][SQ;MF;] Telescope: Slewing to JNOW RA: 12.2842222223901 Dec: 47.2188888888958
[04/21/20 00:03:26.829][DEBUG][Telescope Thread][SQ;MF;] Scope reports it is done with synchronous slew, verifying…
[04/21/20 00:03:26.956][DEBUG][Telescope Thread][SQ;MF;] Telescope: Slewing has completed
[04/21/20 00:03:27.458][DEBUG][Telescope Thread][SQ;MF;] ASCOM Telescope: Failed to flip because starting pier side and ending pier side are the same!
[04/21/20 00:03:27.460][DEBUG][Telescope Thread][SQ;MF;] Telescope thread is IDLE…
[04/21/20 00:03:27.493][DEBUG][Pier Flip Thread][SQ;MF;] Meridian Flip: Telescope command to meridian flip has completed
[04/21/20 00:03:27.493][DEBUG][Pier Flip Thread][SQ;MF;] Meridian Flip: Telescope failed to perform meridian flip

And the AP ASCOM log:

167478 2020-04-21 00:03:25.619: 167479 2020-04-21 00:03:25.642: 167480 2020-04-21 00:03:25.642: 167481 2020-04-21 00:03:25.643: 167482 2020-04-21 00:03:25.645: 167483 2020-04-21 00:03:25.647: 167484 2020-04-21 00:03:25.651: 167485 2020-04-21 00:03:25.652: 167486 2020-04-21 00:03:25.655: 167487 2020-04-21 00:03:25.655: 167488 2020-04-21 00:03:25.692: 167489 2020-04-21 00:03:25.692: 167490 2020-04-21 00:03:25.749: 167491 2020-04-21 00:03:25.767: 167492 2020-04-21 00:03:25.768: 167493 2020-04-21 00:03:25.799: 167494 2020-04-21 00:03:25.799: 167495 2020-04-21 00:03:25.828: 167496 2020-04-21 00:03:25.828: 167497 2020-04-21 00:03:25.860: 167498 2020-04-21 00:03:25.860: 167499 2020-04-21 00:03:25.860: 167500 2020-04-21 00:03:25.906: 167501 2020-04-21 00:03:25.907: 167502 2020-04-21 00:03:25.922: 167503 2020-04-21 00:03:25.922: 167504 2020-04-21 00:03:25.922: 167505 2020-04-21 00:03:25.953: 167506 2020-04-21 00:03:25.953: 167507 2020-04-21 00:03:26.123: 167508 2020-04-21 00:03:26.123: 167509 2020-04-21 00:03:26.123: 167510 2020-04-21 00:03:26.123: 167511 2020-04-21 00:03:26.124: 167512 2020-04-21 00:03:26.124: 167513 2020-04-21 00:03:26.124: 167514 2020-04-21 00:03:26.146: 167515 2020-04-21 00:03:26.313: 167516 2020-04-21 00:03:26.386: 167517 2020-04-21 00:03:26.686: 167518 2020-04-21 00:03:26.703: 167519 2020-04-21 00:03:26.704: 167520 2020-04-21 00:03:26.735: 167521 2020-04-21 00:03:26.735: 167522 2020-04-21 00:03:26.766: 167523 2020-04-21 00:03:26.767: 167524 2020-04-21 00:03:26.797: 167525 2020-04-21 00:03:26.797: 167526 2020-04-21 00:03:26.797: 167527 2020-04-21 00:03:26.829: 167528 2020-04-21 00:03:26.829: 167529 2020-04-21 00:03:26.859: 167530 2020-04-21 00:03:26.859: 167531 2020-04-21 00:03:26.859: 167532 2020-04-21 00:03:26.892: 167533 2020-04-21 00:03:26.892: 167534 2020-04-21 00:03:26.892: 167535 2020-04-21 00:03:26.892: Driver: Info : CommandString TX=’:GA#’
Driver: Info : CommandString: APCC response: ‘:APCC,64287,GA#’, Response=’+8011:33#’
ASCOM: Info : GET Altitude = 80.1958333333333
ASCOM: Info : GET Azimuth = 301.040277777778
ASCOM: Info : GET SiderealTime = 13.1114391666667
ASCOM: Info : GET SideOfPier = West
ASCOM: Info : GET Connected = True
ASCOM: Info : GET CanSlew = True
ASCOM: Info : SlewToCoordinates() RA=12.2842222223901, Dec=47.2188888888958
Telescope: Info : CommandString TX=’:SLEW,12.2842222223901,47.2188888888958#’
Telescope: Info : CommandString: APCC response: ‘:APCC,64288,SLEW,12.2842222223901,47.2188888888958#’, Response=‘1#’
SlewToCoordinates: Info : Result from APCC SLEW=1
Driver: Info : CommandString TX=’:GOS#’
Driver: Info : CommandString: APCC response: ‘:APCC,64289,GOS#’, Response=‘129000202P000#’
Driver: Info : CommandString TX=’:GR#’
Driver: Info : CommandString: APCC response: ‘:APCC,64290,GR#’, Response=‘12:17:03.2#’
Driver: Info : CommandString TX=’:GD#’
Driver: Info : CommandString: APCC response: ‘:APCC,64291,GD#’, Response=’+4713:08#’
Driver: Info : CommandString TX=’:pS#’
Driver: Info : CommandString: APCC response: ‘:APCC,64292,pS#’, Response=‘West#’
Counterweight: Info : Counterweight UP
Driver: Info : CommandString TX=’:GS#’
Driver: Info : CommandString: APCC response: ‘:APCC,64293,GS#’, Response=‘13:06:40.9#’
Driver: Info : CommandString TX=’:pS#’
Driver: Info : CommandString: APCC response: ‘:APCC,64294,pS#’, Response=‘West#’
Counterweight: Info : Counterweight UP
Driver: Info : CommandString TX=’:GA#’
Driver: Info : CommandString: APCC response: ‘:APCC,64295,GA#’, Response=’+8011:33#’
Driver: Info : CommandString TX=’:pS#’
Driver: Info : CommandString: APCC response: ‘:APCC,64296,pS#’, Response=‘West#’
Counterweight: Info : Counterweight UP
SLEW COMPLETE: Info : Slew complete: Setting tracking rates
CheckSlewComplete: Info : Restoring tracking state = 1
User: Info : Start tracking pressed
ASCOM: Info : SET TrackingRate = 0
Driver: Info : CommandBlind TX=’:Q#’
Driver: Info : CommandBlind TX=’:RT2#’
Driver: Info : CommandString TX=’:GZ#’
Driver: Info : CommandString: APCC response: ‘:APCC,64297,GZ#’, Response=’+30103:55#’
Driver: Info : CommandString TX=’:GOS#’
Driver: Info : CommandString: APCC response: ‘:APCC,64298,GOS#’, Response=‘129000202P000#’
Driver: Info : CommandString TX=’:GR#’
Driver: Info : CommandString: APCC response: ‘:APCC,64299,GR#’, Response=‘12:17:08.1#’
Driver: Info : CommandString TX=’:GD#’
Driver: Info : CommandString: APCC response: ‘:APCC,64300,GD#’, Response=’+4713:08#’
Driver: Info : CommandString TX=’:pS#’
Driver: Info : CommandString: APCC response: ‘:APCC,64301,pS#’, Response=‘West#’
Counterweight: Info : Counterweight UP
Driver: Info : CommandString TX=’:GS#’
Driver: Info : CommandString: APCC response: ‘:APCC,64302,GS#’, Response=‘13:06:42.0#’
Driver: Info : CommandString TX=’:pS#’
Driver: Info : CommandString: APCC response: ‘:APCC,64303,pS#’, Response=‘West#’
Counterweight: Info : Counterweight UP
Driver: Info : CommandString TX=’:pS#’
Driver: Info : CommandString: APCC response: ‘:APCC,64304,pS#’, Response=‘West#’
Counterweight: Info : Counterweight UP
SLEW COMPLETE: Info : Slew complete: Setting tracking rates
CheckSlewComplete: Info : Restoring tracking state = 1

One minute and a half later the mount is still counterweights up pointing at the same position. The flip wasn’t delayed due to time-outs, it just didn’t happen for some reason:

168601 2020-04-21 00:04:58.887: ASCOM: Info : GET CanSlewAsync = True
168602 2020-04-21 00:04:58.887: ASCOM: Info : GET RightAscension = 12.2936388888889
168603 2020-04-21 00:04:58.888: ASCOM: Info : GET Declination = 47.2188888888889
168604 2020-04-21 00:04:58.888: Driver: Info : CommandString TX=’:GA#’
168605 2020-04-21 00:04:58.893: Driver: Info : CommandString: APCC response: ‘:APCC,64738,GA#’, Response=’+80*02:11#’
168606 2020-04-21 00:04:58.893: ASCOM: Info : GET Altitude = 80.0366666666667
168607 2020-04-21 00:04:58.893: ASCOM: Info : GET Azimuth = 300.671388888889
168608 2020-04-21 00:04:58.894: ASCOM: Info : GET SiderealTime = 13.1373161111111
168609 2020-04-21 00:04:58.895: ASCOM: Info : GET SideOfPier = West

the computer is dedicated to controlling the mount and imaging. It is connected directly to the mount via a USB hub and short cables. At the time of the attempted flip, there was no remote desktop connection running (I was asleep) or I would have caught it.

Thanks again,

Luca

Gringo · April 22, 2020, 1:59pm

Thank you Ray, will try that. Actually I saw all these comms errors in the logs but they have always been there even when the flips were successful…well maybe not that many

Are there any downsides of increasing the timeouts too much?

Thanks, André

rgralak · April 22, 2020, 2:25pm

Luca/Andre, the intermittent nature can be caused by commands seemingly not completed because they have timed out, yet APCC will actually process them. This can cause the ASCOM driver to believe the mount is in a different state then it actually is (e.g. slewing or not),

You may also want to try a new versions of the driver and APCC Standard and Pro. The new APCC versions fix a potential high cpu utilization issue when using Horizon limits.

https://www.gralak.com/apdriver/AstroPhysics_V2_Setup_5.30.10.exe
http://www.apastrosoftware.com/apcc_download/APCC_Standard_Setup_1.8.1.0.exe
http://www.apastrosoftware.com/apcc_download/APCC_Pro_Setup_1.8.1.1.exe

-Ray

lucam · April 23, 2020, 1:16pm

Update: I made a few changes at the same time, but the result was positive. I updated APCC Pro to 1.8.1.1 and the AP ASCOM driver to version 5.30.10, I made the timeouts longer (200msec in APCC and 2 sec in AP ASCOM). Last night the meridian flip completed successfully on the same target for which the flip failed the previous night. Fingers crossed that this takes care of it.

@rgralak, is there a downside to longer timeouts? Is it just the overall responsiveness of the system that is affected or can processes overlap if timeouts are too long?

Gringo · April 23, 2020, 4:15pm

Ok, I have also updated to the new APCC and AP ASCOM driver versions and have gradually increased the timeouts until the error messages disappeared…now 2000ms ín the driver and 1000ms in APCC.

My imaging session last night was flawless, though a meridian flip was not needed with the current target. Hopefully I can try again tonight and report back.

Thanks, André

Gringo · April 24, 2020, 8:06am

Unfortunately the meridian flip is still failing for me, even with the longer timeout settings. Same problem, driver reports it´s done slewing. It seems that the errors during initialization of the mount actually got worse…more retransmits?

What puzzles me is that it has worked perfectly for almost one year with the same settings and now its failing consistently…and only` the meridian flip, everything else is perfect. By the way, it works when I push the pier flip button in APCC…

I also doubt that is a hardware or cable issue, checked everything. (and Luca is getting exactly the same errors)

@lucam , do you still get errors during mount initialization, how is your mount connected to your pc and do you have „ASCOM Serial Object“ checked in the driver? Or maybe it´s the COM port settings in the PC that are suddenly acting up, mine are still set to the defaults…

@rgralak, should I increase the timeouts even more or any other ideas what could be tried?

Thank You, André

Link to the logs:

https://www.dropbox.com/sh/2oooztnhimaxyan/AADL8fDKvP1gSMVzX_GPDfR0a?dl=0

rgralak · April 24, 2020, 1:54pm

Andre,

You should not set the timeout too high as that will cause lagged responses if APCC cannot talk to the mount. It will not affect APCC during normal operations.

That said, there were no errors in your logs from unexpected timeouts this time. However, it looks like the flip offset in APCC was not set appropriately so the mount did not flip (purposely).

Can you post a screen shot of the Meridian Limits tab in APCC?

-Ray

lucam · April 24, 2020, 2:22pm

Andre,

I am not sure in which file you are finding errors during mount initialization. I don’t think I have seen any errors even on the night when the meridian flip was missed. If you give me some more detail, I am happy to check and be on the lookout on the next clear night.

Luca

Gringo · April 24, 2020, 3:32pm

Ray,

just for the test yesterday I set up APCC to not forward the limits to SGP and let SGP handle the flip. Usually I have a flip offset of 15 minutes and the limits sent to SGP and then SGP correctly displays the negative values until the flip…but it doesn´t work either way the symptoms are the same…driver thinks the slew is complete.

Do you think I misconfigured something, maybe in SGP…but then again SGP initiates the flip exactly at the expected time.

If nothing helps I might set up everything from scratch and see how it goes…

APCCscreenshot

Luca,

the errors are in the AP ASCOM logs only after startup (for about 1 minute) but as Ray suggests these are maybe harmless. I saw a couple in the logs you posted initially and am curious if you still have them with your new settings…I´m glad it is fixed for you, so there is hope

Thank you both,

André

rgralak · April 24, 2020, 4:20pm

Andre, maybe you unchecked the option in APCC, but you must enable the checkbox “Send Limit with offset to SGPro”. It is not enabled in your screen shot.

Also,make sure the flip offset value is large enough. If your exposures are 5 minutes for example, then add another 5 minutes to that to be safe (so set flip offset to 10 minutes).

-Ray

Gringo · April 25, 2020, 8:23am

I am happy to report that the meridian flip last night was successful. I’ve changed the timeouts to 2000/500 (Driver/APCC) set flip offset in APCC to 15m and configured the driver in APCC (Now button) finally I reset the sequence in SGP…et voilà.

In the end I am still unsure why all this happened and which of my actions eventually solved it but will keep a close eye on it…obviously.

Thank You Ray and Luca,

André