Dome momentarily loses sync with J Now mount after plate solve

I have been having an on-going problem with SGP control of Shelyak Ascom Dome control – but only after Plate Solve. The dome is slaved to the scope in SGP (using the Shelyak Ascom driver for the dome). In general use, the dome will follow the scope – and I don’t have any problems. Lately though, rather than working on one target, I have tried two or three in the night. Then the plate solve routine to place the scope on the new target causes me problems. I tried once or twice to view the dome tracker log – but it is huge. There was no point in trying to send something in from a nights observing.
Tonight I did a small test. I initially synced the scope by aligning on a star (Sirius) and then syncing via The Sky X (SAE). With the scope synced, I slaved the dome – it adjusted correctly – so all well.
I then did a plate solve (PS2) and I guess the scope and dome were exactly on position – I heard no movement from the dome. On the downloaded plate solve frame, I picked a random spot and called for ‘Center Here’. SGP went through the motions and had adjusted the mount at least twice to get to the center here position. During this process the dome moved off a couple of degrees and back again – at least twice. The mount movement would have been negligible and this time the dome movement was small and did not interfere with the SGP getting other frames for plate solving. (In truth the dome should not have moved at all - the shutter is wide enough). I have seen the dome take off more than those couple of degrees at the juncture – and getting a frame for plate solving was then impossible. That did not happen during the test tonight – BUT at the final plate solve and scope sync – the dome did take off (about 70° or 80°)…and then came back to the correct position.
At this stage, I have no idea if the problem arises with SGP or the Shelyak Ascom driver. One thing that comes to mind (since the dome follows the mount without problem – when no sync is involved) is that SGP initially puts out the scope sync position in J2000 before sending JNOW positioning to the mount (Takahashi EM400). Does SGP actually SEND that sync info and then send updated (J2000) info? Is there two syncs coming in, one after the other to the mount? I know that we are not talking about 70 or 80 degrees difference between the sync positions – but is there any possibility that the problem lies here somewhere? As it is, I cannot leave the scope unattended for change of target or indeed a meridian flip for fear of a big excursion by the dome, the camera not having a clear shot at second or third frame for solving and then SGP going into recovery mode……and then another possible similar cycle at the next try.

Two logs on DropBox from tonight’s test – your thoughts would be appreciated when you get a chance.

Times – roughly:
212133 First solve & sync with PS2 (Mount already synced manually)
212822 Center here initiated
212946 Final solve & sync after which the dome went ‘walkabout’
213150 Dome back to correct position – stop the test.

DropBox Link

Looking at the dome log it’s been reporting an azimuth of 170 but then at 21:29:46.051 there is what looks like a dome set position command with a position of 280.47 deg.

There are several more set position commands for the same position and the dome moves there, then at 21:30:29.119 there’s a set position command for a position of 170.01 deg.

The dome moves back to this position. This looks as if the dome is doing what it’s asked to do.

There’s no record of what commands SGP thinks it is sending.

The sgp log shows an error at about this time:
[18/01/2016 21:30:30] [DEBUG] [Dome Thread] Dome: Caught exception in MoveAsync - DomeTracker Error : Dome shall not be rotating to perform this operation ![18/01/2016 21:31:40] [DEBUG] [Telescope Thread] Attempting to stop PHD2 guiding…

This error seems to be happening quite a lot, not sure if it’s relevant but it looks as if the Shelyak driver doesn’t like to be given a move command when it’s moving.

BTW log files are very repetitive so compress really well. the 49 Mb dome log file compresses to 3.6 Mb.

Chris

Thanks Chris for looking into this. We will wait to see if Ken/Jared see anything on their side.

Today, I wondered if there were any Ascom Temma logs for the mount. I never did have ‘log’ ticked, so what is there is absolutely of no value - less than one page written since early last year. I can do another test hopefully with something more out of the mount log - now that I have ‘log’ ticked…if anyone thinks this may be of help…and if the clouds clear enough.

Another point on my mind is the fact that SGP calls the EM400 a JNow mount. I never did understand or question this - as it always worked. But the Temma 2 (Takahashi mounts) really has no pointing model as far as I am aware. If it resets it goes back to zero hour angle and zero declination and moves X and Y from there. If I sync it with The Sky X (SAE) or even to the Zenith (having put in Lat & Long), as far as I know it just moves X & Y from where it was told it was when synced. It does not have any model as far as I am aware (but in truth, I don’t really know). So will it not do similar from getting a plate solve sync…without having to be given an epoch. I have never seen it anywhere that Temma 2 is JNow or anything else.
Perhaps someone with similar control on their EM 200 or EM 400 can shed some light on this.

Brendan.

P.S. Noted re compressing the logs.

I’ll take a look this evening. Haven’t looked at your logs yet but were you anywhere near the pole, RA 0 or the meridian? Those 3 places are generally the most likely to cause issues…

The mount tells us the Epoch, so that would mean that the Temma is reporting it talks JNow. Also @Ken uses an EM400 so it’s likely been tested against it.

Anything that is text will almost always compress to around 10:1 or a little better.

Thanks,
Jared

Thanks Jared for taking the interest. Re the 3 ‘hotspots’…no I was
looking to the SE somewhere when I did the test (RA: 7.03387686652948 Dec:
-26.1825029733717)…and in any case, this is an ongoing problem …not
just where I was pointing for the test.

Brendan.

Slán,
Brendan.

Last night I was doing some more imaging on NGC 2359. As expected I had
problems with the dome taking off again when the mount was synced just
before meridian flip.

When the mount was synced before the flip, the dome started to rotate to the
west basically from 202° to 280°. It stayed at 280° while the mount
fliped and as the mount finished its movement the dome started to return
to the correct position. However it took too long to come back - thus
the verifying plare solve did not work - it was trying to solve a
capture of the inside of the dome. SGP went into Recovery. During the
recovery it managed a first platesolve image download but
the dome again took off (when the sync was issued)- this time in the opposite direction: from 159°
to 079°. This time. it returned to correct position in time to get
enough of the capture to verify final position of the mount and the sequence
continued.

I had the Temma Ascom log enabled and although, there is very little
logged, you will note that there were two very incorrect syncs received
at 23:21:25 and 23:28:33. These are, I presume, what are causing the
dome to slew off one way or the other. (I could not copy the actual
Temma log file - I have included (in dropbox) a screen capture though.
Also included - the SGP log and the dome tracker log covering the times
of the problem).

I hope the logs sheds some further light on what is happening - basically
every time a solve and sync happens.

Dropbox Link

Times: 23 21 26.04 Dome takes off 78° clockwise and 23 28 33.313 Dome takes off 80° anticlockwise.

Brendan

Interesting find. I’m not sure what or why it’s syncing to those coordinates. It does not appear that SGP is sending those coordinates:

  • SGP: [21/01/2016 23:28:32] [DEBUG] [Telescope Thread] Telescope: Syncing to JNOW RA: 7.32099864005298 Dec: -13.2477646627205
  • Temma Log: 23:28:33 - Sync Ra 07h26m52s Dec 37d34m12s

That could certainly make the Dome not know where it’s going as it now thinks the scope is pointed someplace totally different. When the dome sync timer runs out it will slew to where the scope says it’s pointed at. The coordinates that we’re reporting in the log are exactly what we send to be synced.

Logger.debugln( "Telescope: Syncing to JNOW RA: " + ra + " Dec: " + dec );
tel.SyncToCoordinates(ra, dec);

So I’m not entirely sure where this is getting lost in translation between SGP and the Temma driver. Is anything else connected to the Temma driver besides SGP (and what appears to be PHD2)?

Thanks,
Jared

Hi Jared, yes, I suppose it was an interesting find. At least it confirms (with numbers) what Chris has said… that the dome is doing what it is asked to do. Now, we just need to be 100% sure what is doing the asking.

There is one possible other connection to to the Temma driver and I really can’t be sure if it was connected last night or not…that is The SKY X (SAE). At times I do connect it to view what is available or check the altitude of a particular target at this time or that time (and sometimes position the the scope with it). I know that I did not position the scope with it last night…but was it connected???
I am 100% sure though - that as soon as the plate solve frame was ‘solved’ - that is the exact moment the dome moved off to the wrong location.

I guess now I will have to do another test to rule that software in or out.

There are breaks in the clouds tonight - I’ll try get another test done and report back.

Brendan.

Well - I thought with your help there Jared I had found the problem. But afraid not. The SKY X (SAE) was not running. I slaved the dome to the scope, did plate solve & sync, did recenter here, chose another target and had the scope center on it, did another center here on that last plate solve image…all was going well. The dome was going where it should.
Then…a last target to check…image from the computer solved OK and when told to center on the target…the scope went one way and the dome went the other. By the time it came back the plate solve image was of the inside of the dome …same as last night.

I won’t attach all the logs - the ones from last night tell the full story…but here is the capture of the Temma log…

I wanted to go from Dec -13 to Dec +8…where Dec +37 came from in the sync, I have no idea…EXCEPT, just as I am writing this it hits me…that is my Latitude. Does the RA make a connection with my Longitude (1° 11.1’W) at that time…(Times are GMT)?

Is there something here to delve into?

Hope you guys can help get to the bottom of this…

Are your temma drivers and ascom all up to date? May be worth trying that if not.

I don’t see any place in sgp where were sending a Dec of 37. Your logs indicate that sgp sent the correct values as well.

Jared

Hi Jared,

Yes, Ascom and the Shelyak &Temma drivers are the latest available.

Seeing the erroneous sync to my latitude makes me ask…where that
information may be coming from? Both the Shelyak & Temma drivers
have those numbers but if SGP is passing the sync (from PS2 in this
case) how would the latitude information leak into the sync received and
recorded by the Temma log? This is beyond me I am afraid.

There are three things I want to try to nail this. One, I’ll do myself at
first opportunity…I’ll play with the update frequency in the slave
settings. That won’t actually ‘fix’ the source problem - but it may help
me out. (I may also try plate solving with Elbrus or Astrometry.net
(local) - but I doubt that will make any difference).

The other two things require some info. Can you or anyone else running a
Tak Temma 2 mount, check if there are similar entries in your own Temma
log (showing your particular latitude after a solve & sync). This
may be there but not affecting your particular set-up…after all I
have no problem with the mount doing what it is supposed to do. Secondly
I’d like to test an older SGP - before the JNow - J2000 info exchange
came in. If you know offhand which version that appeared in, I’d
appreciate it…otherwise I’ll just go looking through what I have here
myself.

The above may seem a bit far fetched…but I can’t think of anything else
to indicate whether this problem is emanating from SGP, Temma or
Shelyak. The problem has been there quite a long time (for sure when
this topic was raised:
Odd Dome Behavior while slewing to 2nd Target - Sequence Generator - Main Sequence Software).
It was an inconvenience before, now it really is having a huge impact
on how I want to run things.

Brendan.

**Additional note: Just now noticed/remembered that PS2 also has the Lat & Long info. **

AIUI the dome is controlled by SGP and nowhere else. This means that the bad dome position command is coming from SGP. The SGP logs don’t seem to show what dome position commands it is sending.

It is likely that the reason for the bad slew is that the mount is returning a bad position. This seems to sort itself out once the slew to the other side of the meridian has been done. Getting the position data that the mount is returning will be useful.

One thing that may help is to enable the ASCOM DriverAccess Trace. This will log every message sent and received by any application that is using the ASCOM DriverAccess component - which SGP does. This can be done in the Trace menu option of either to ASCOM chooser or the ASCOM diagnostics. No need to run the diagnostics.

This will, hopefully, generate several ASCOM.DriverAccess log files, for the scope, dome and camera. With a bit of luck these will show what is going on.

Is there some way this can be reproduced without wasting good imaging time? Difficult with something that relies on imaging and plate solving.

Chris

Thanks for looking in again on this Chris. I’ll enable the trace via the diagnostics page and see what is produced. (Unsure where that will be saved - but I am sure I will find it). I’ll do some testing as soon as I have stars to plate solve & sync with.

If I put slightly different latitude values in Temma, Shelyak and PS2…if we see one particular value showing up in the Temma log - might that indicate the source of the bad sync info? (The Latitude numbers that are being passed to tell the dome where to align itself are the easiest for me to track down).

Just one point re your last post: this does not just happen at meridian flip - it appears it can happen anywhere once a solve & sync is involved. (Sometimes I don’t notice it because the dome does not move too far…and thus a subsequent plate solve image is not affected).

Hopefully the Ascom Trace will tell all.

Brendan.

OK - a follow up tonight. Thanks to Chris for pointing me toward Ascom Driver Access Trace. I can see now what is happening - but I don’t know why.
Chris was right the first time around - the dome is doing what is being asked of it. Ken & Jared are also right - looks like the correct sync information is being passed to the mount.
Tonight, I connected mount, dome & camera. I unparked the mount and at that position did a solve & sync. The dome took off for a little spin - but came back again. I then solved a previous image (NGC 2359) and told the mount to center on that location. It did that OK but when verifying with the solve & sync on ‘arrival’, the dome moved off some & returned. However it did not move too far. the move did not block a second solve & sync that verified the move completed correctly after slight adjustment.

What appears to be happening - though not every sync as far as I know - is that the mount momentarily syncs to the Zenith even though the proper RA & Dec are being passed to the mount. Why this happens I don’t know. It appears to correct itself within seconds (on the test I did tonight anyway) - but the dome has already started moving and this is what is causing me problems. This would not be noticed working the mount without a dome (I guess) as the mount remains in position during this critical time.
Looks like the answer to my problem does not lie with SGP.
Chris - if you have time to look at the attached logs - whenever - just let me know if my reading of the information is correct. Any thoughts on why this might be happening?

Brendan.

Ascom Logs 23Jan16.zip (593.1 KB)

Yes, it’s pretty clear, the mount is reporting the position incorrectly for a few seconds after the sync command.

This is something you will need to take up with the Temma driver author or mount people. It isn’t something that can or should be fixed in SGP.

One thing that may be significant is that the initial Ra is 0.003 and the sync position is 23.994. The change happening over the 0 - 24 hour boundary may be relevant.

Chris

Thanks once again for all the time you put in on this Chris. As soon as I had a look through those logs - I knew it was not SGP…but until you pointed me in the right direction I had no idea. I am still not sure if it is hardware or software - but I’ll stay with it until I get it sorted.

Just a further update:

This is what I am getting back from the Temma driver author:

The Temma protocols require the following structure to sync:

1 - Set Sidereal time
2 – Send Zenith command
3 – Set Sidereal time again
4 – Send sync coordinates

AND

All Temma commands are dictated by the Takahashi protocols. So, the
Zenith command is required before the actual sync coordinates will be accepted
by the mount. Moreover, the latency of sending and receiving the commands
can be as much as 250 ms each. I’m not sure what I can do on my end since
I can only follow the command protocols.

So, looks like what we are seeing in the logs is as expected from Temma 2 (Takahashi). I cannot understand then that nobody else is experiencing the same problems…maybe they are not operating domes ???

Was that posted somewhere?

What I would recommend is that the sync command block until its good and not to change the ra and Dec that the scope is reporting to bad values. Sgp has no way of knowing when this happens.

Also what do you have your dome slave time set to? Increasing that may help to make this happen less often. And while not a fix it may take care of it for the vast majority of times.

Jared

IMO it should be up to the driver to implement sync without returning incorrect values. If this requires that it has to jump through hoops to do so then so be it.

There are several ways this could be done, one way would be to block all commands while the sequence of commands needed to do a sync is done.

Expecting the application to guess that because a sync was done recently the data can’t be relied on isn’t really acceptable.

Chris

I spoke to Chuck Faranda (Temma driver author) via the Tak Yahoo Group

See: https://groups.yahoo.com/neo/groups/UncensoredTakGroup/conversations/messages/64030

…and subsequent messages.

I don’t think the slave time (10 secs) has any bearing - the dome takes off immediately the sync is completed…or at least as soon as the mount reports the position (albeit the wrong position).

Brendan.