I’m running Beta 3.1.0.321. I just started to run a sequence and I immediately have dome sync issues. The initial slew goes to the target and the dome is in sync. The next slew, at the start of the centre routine, makes the dome go out of sync, it sits there for a short while and then goes back in sync to where it was first of all. Meanwhile the platesolve fails as it has taken a picture of the inside of the dome. The problem continues again and again. The safest solution is to unslave the dome, get the sequence started and then reslave the dome for it to catch up. All is fine until the next centring run…
Ken, thank you for your swift reply. I fully appreciate that you can not problem solve one user’s issues, but I do believe that I am not alone with this new problem. In my earlier thread about this problem another user replied:
“I am having the same dome sync issues as you describe. It ruined two nights of imaging last week as they happened while I was sleeping. I was able to see it happen two nights ago when I was attempting to start a session manually. I was able to go around it by disabling dome sync while I did the centering routine, and then re-syncing it. But that is not a solution.
This is a serious issue that I hope is addressed soon.
Eric”
Please let me know what else I can furnish you with to expedite a solution.
Jared is more familiar with dome aspects of SGPro… I pinged him with this. In the meantime… I didn’t see anything suspicious in the logs. It might be helpful if you able to reproduce the situation and provide SGPro logs with corresponding trace logs for the dome. I don’t know if that will be necessary, but it should be easy enough to toggle them on.
I have just reproduced the problem multiple times and have the dome trace log too. The Dome is clearly being sent to a particular wrong azimuth and then shortly afterwards being sent to the correct azimuth. The autocentre failed a couple of times in this run as a result.
Please tell me that it is a simple error that you can fix easily, or that it is something I have done wrong that I can fix easily!! It’s weird that it does sync up properly, but just has these moments of being sent a few tens of degrees away from the correct azimuth on a slew.
@PhotoGav I saw this behavior long ago, but thought we had addressed it. To clarify, is the dome in the correct position when the scope is just tracking or is it (briefly) in the correct position when a slew is issued to the scope?
Basically if you slew the scope does the dome go to some weird position initially and then go to the correct position within a minute or so?
The first position the Dome goes to is not correct - it is about 30 to 60 degrees off. About 10 to 15 seconds later it receives the message to go to the correct azimuth. It then continues to track in the correct place. On the next slew command it goes to the off sync azimuth followed by the in sync azimuth again.
I am strangely encouraged to hear that this is familiar behaviour to you. Older versions have worked ok, except for some issues which I put down to being near the meridian, but had flagged up. The problem has emerged significantly again in the last few beta releases.
Good luck hunting through the code. I hope you identify the issue easily. If I can help with testing pre-release versions, please just shout.
Jared, just to follow on from above… I think that the weird sync behaviour is tied in with the auto centre routine. I need to test this for certain, but I think it’s true to say that if I right click on a target and click slew to target, it will sync up properly immediately. However, if I click centre on target, or when running a sequence, the weird behaviour happens. I will go and test this as much as I can in daylight now and will report back my findings. Gav.
Jared you are correct - I had the problem at the time…but it turned out to be the Takahashi driver for the mount causing the problem…fixed with a new driver.
I have confirmed that the dome slew errors are connected to both ‘slew now’ and ‘center now’ commands.
I am in the dome now. All equipment connected and dome slaved. I command a slew to a target, the scope slews properly, the dome slews approximately 25 degrees off of that point initially. Then re-slews around 30 seconds later to the correct position and tracks normally.
When I command the ‘center now’ command the same behavior occurs. Target azimuth at 335, dome slews to 311, then 30 seconds later corrects to the right AZ. But by this time the plate solve image has been taken and it of course fails because it’s looking at the inside of the dome.
I have attached the logs of that series of events. There are lines in there that show the initial slew command to the correct location, but then a command from the Observatory function to adjust to the wrong location.DomeSync.txt (16.8 KB)
I too have just been in my dome having a play with all of this. Here is my order of events and what happened:
Connect everything
Unpark
Slave dome – Moves to correct sync point with mount in home position
Slew to M29 (in East) from Cartes du Ciel – Mount moves, dome moves to correct sync position
Slew to M3 (in West) from Cartes du Ciel – mount moves, dome moves to correct sync position
Slew to M29 manually from SGP – mount moves, dome moves to correct position
Centre on M27 manually from SGP – mount moves, dome moves to correct position
Park – everything parks correctly
Run sequence to ‘image’ M29 – dome starts moving, mount moves to target, dome stops in completely wrong position, then about 15 seconds later moves to correct position
Aborted sequence
Tried M29 sequence again – same as above
Aborted sequence
Unpark
Slew to M29 – mount moves to correct position
Slave Dome – Dome moves to correct position
My conclusion is that this is all going wrong when in a sequence…
Reading through the logs, it looks like it is doing this on Run Sequence: slaving the dome and sending it to sync with the mount in the home position. As it is on route to the mount home position, the instruction to slew to the target azimuth arrives from the Sequence Thread. This is an incorrect value. Then the Telescope Thread kicks in with its attempt at the dome sync azimuth for the active target and comes up with ‘no adjustment needed’, so the dome carries on its journey to the incorrect position. Then Dome Thread has a go and comes up with the correct azimuth value. The Dome starts its journey to the newly established correct value. The plate solve routine kicks in, thinking that all is settled with the Dome, when actually it is moving to the correct place - the plate solve image is of the inside of the Dome and it all fails.
If the Dome is synced manually while not in a sequence, the azimuth comes from the Dome Thread and all is perfect.
Jared - does that look like it could make sense and that the azimuth is calculated from different areas, one or more of which could be doing it incorrectly?
Thanks for your testing. My situation is worse, as it fails on regular ‘slew now’ commands also. The log of one such attempt is below. The target AZ is 36, it slewed back to the left to 321, waited 30s then corrected back to the correct AZ. ‘Center now’ has the same behavior, but with the added harm of ruining the center attempt. It is going to be crystal clear here the next two nights and it is very frustrating that this situation is functionally preventing me from allowing the sequence to run unattended as I would have to manually slew the dome, disable slave, do the centering, then renable slave so the dome tracks the target appropriately.
[10/23/19 18:57:08.388][DEBUG] [Main Thread] Slewing to target…
[10/23/19 18:57:08.397][DEBUG] [Telescope Thread] Slew telescope message received…
[10/23/19 18:57:08.397][DEBUG] [Telescope Thread] Telescope: Slewing to J2000 RA: 2.4482 (02h26m53.52s) Dec: 62.035 (62°02’06.00")
[10/23/19 18:57:08.397][DEBUG] [Telescope Thread] Telescope: Slew received J2000 coordinates, mount requires JNOW, converting…
[10/23/19 18:57:08.398][DEBUG] [Telescope Thread] Telescope: Slewing to JNOW RA: 2.47380325982841 Dec: 62.1219692379047
[10/23/19 18:57:08.398][DEBUG] [Telescope Thread] Telescope: Calling Observatory Slave Slew
[10/23/19 18:57:08.400][DEBUG] [Slew Monitor] Waiting for slew to complete…
[10/23/19 18:57:08.414][DEBUG] [Telescope Thread] Observatory: Calculating new position using:
[10/23/19 18:57:08.414][DEBUG] [Telescope Thread] Azimuth: 323.730438634818
[10/23/19 18:57:08.414][DEBUG] [Telescope Thread] Altitude: 43.8983399166111
[10/23/19 18:57:08.414][DEBUG] [Telescope Thread] Hour Angle: -99.3903478672878
[10/23/19 18:57:08.414][DEBUG] [Telescope Thread] Pier Side: West
[10/23/19 18:57:08.414][DEBUG] [Telescope Thread] Observatory: Adjustment needed, slewing to Azimuth: 322.100813579163
[10/23/19 18:57:08.415][DEBUG] [CP Update Thread] Error in control panel UI updater: Object reference not set to an instance of an object.
[10/23/19 18:57:18.516][DEBUG] [Telescope Thread] Scope reports it is done with synchronous slew, verifying…
[10/23/19 18:57:18.516][DEBUG] [Telescope Thread] Telescope: Observatory is reporting slewing
[10/23/19 18:57:19.518][DEBUG] [Telescope Thread] Telescope: Observatory is reporting slewing
[10/23/19 18:57:20.519][DEBUG] [Telescope Thread] Telescope: Observatory is reporting slewing
[10/23/19 18:57:21.519][DEBUG] [Telescope Thread] Telescope: Observatory is reporting slewing
[10/23/19 18:57:22.520][DEBUG] [Telescope Thread] Telescope: Observatory is reporting slewing
[10/23/19 18:57:23.520][DEBUG] [Telescope Thread] Telescope: Observatory is reporting slewing
[10/23/19 18:57:24.521][DEBUG] [Telescope Thread] Telescope: Observatory is reporting slewing
[10/23/19 18:57:25.522][DEBUG] [Telescope Thread] Telescope: Observatory is reporting slewing
[10/23/19 18:57:26.522][DEBUG] [Telescope Thread] Telescope: Observatory is reporting slewing
[10/23/19 18:57:27.522][DEBUG] [Telescope Thread] Telescope: Observatory is reporting slewing
[10/23/19 18:57:28.168][DEBUG] [PHD2 Listener Thread] PHD2 - No messages received from PHD2 for 1 minute, checking socket with status…
[10/23/19 18:57:28.168][DEBUG] [PHD2 Listener Thread] Checking PHD2 state…
[10/23/19 18:57:28.168][DEBUG] [PHD2 Listener Thread] PHD2 GetPhdStatus - Pre-Wait : Stopped
[10/23/19 18:57:28.168][DEBUG] [PHD2 Listener Thread] Sending to PHD2:
{“method”: “get_app_state”, “id”: 1001}
[10/23/19 18:57:28.168][DEBUG] [PHD2 Listener Thread] PHD2 GetPhdStatus - Post-Wait: Stopped
[10/23/19 18:57:28.523][DEBUG] [Telescope Thread] Telescope: Observatory is reporting slewing
[10/23/19 18:57:29.524][DEBUG] [Telescope Thread] Telescope: Observatory is reporting slewing
[10/23/19 18:57:30.524][DEBUG] [Telescope Thread] Telescope: Slewing has completed
[10/23/19 18:57:30.524][DEBUG] [Telescope Thread] Telescope: Settling for 5 seconds
[10/23/19 18:57:35.525][DEBUG] [Telescope Thread] Telescope: Settling has completed
[10/23/19 18:57:35.525][DEBUG] [Telescope Thread] Slew complete…
[10/23/19 18:58:15.124][DEBUG] [Main Thread] PopulateDataModel: Transferring view to the data model…
[10/23/19 18:58:15.133][DEBUG] [MF Update Thread] Performing serialize…
[10/23/19 18:58:15.143][DEBUG] [MF Update Thread] Serialization took 10 msec.
[10/23/19 18:58:28.302][DEBUG] [PHD2 Listener Thread] PHD2 - No messages received from PHD2 for 1 minute, checking socket with status…
[10/23/19 18:58:28.302][DEBUG] [PHD2 Listener Thread] Checking PHD2 state…
[10/23/19 18:58:28.302][DEBUG] [PHD2 Listener Thread] PHD2 GetPhdStatus - Pre-Wait : Stopped
[10/23/19 18:58:28.302][DEBUG] [PHD2 Listener Thread] Sending to PHD2:
{“method”: “get_app_state”, “id”: 1001}
That is highly frustrating. The weather is awful here at the moment so at least I’m not losing imaging time. You might consider going back a number of releases so that you can gather data reliably?
It is very interesting to note in your log that, similar to my findings, the Telescope Thread is issuing the wrong azimuth value, but the Dome Thread is giving the correct value. I look forward to hearing Jared’s opinion on this one.
It’s a CGE Pro and an 11’ Poly Dome using Max Dome 2 software. I installed it this spring and it has been working mostly flawlessly until these most recent beta version.
I got around the problem last night by disabling slave during centering. But also had to wake up at 0130 to do a manual meridian flip. After all these months of sleeping blissfully through the night that wasn’t fun!
Since this same issue came up several years ago in a previous version of SGP I hope that it can be identified and squashed soon so we can go back to normal ops.
Absolutely! Interesting re. kit as we are using completely different things, which points the finger at logic problems rather than kit problems.
I’m sure Jared ‘The Bug Killer’ will be along very soon to let us know that he has found the error, fixed it and made a new beta release that solves all our issues!
Sorry, I had typed up a reply last night but got distracted with tracking this down and forgot to hit the button :-/
Thanks for the clear reproduction steps! I’m able to make it happen with the sims and hope to address it shortly! It seems to be a race condition between the unpark and the slew. Basically the mount/dome unpark and then SGP slews the dome to the mount location, shortly there after the mount slews and the dome doesn’t catch up.
Thanks Jared. Sounds like you have a good idea of why it is going wrong. Do you think that the initial problem of everything unparking and being sent to various positions without getting there creates an issue that persists to mess up future slews and Dome syncs?
Out of interest, where does the azimuth calculation happen - is it in multiple places as the different Thread references might suggest or is it one routine that is simply called from different Threads?
It is only in in one place. But we only calculate azimuth if we can’t get it from the scope. Generally we get the azimuth from the scope unless we’re slewing, in which case we can’t ask the scope for the azimuth preemptively, so we have to calculate it. This calculation was incorrect in some states.
Beta is forthcoming…but we’re at a star party and the internet is “non-ideal”.