OK. Well, the good part is that your logs are remarkably consistent. That’s the first step to a good trace. They have led me to a particular area of code that is possibly problematic and I have refactored a part of it that I have not liked for a very long time… maybe it will make a difference, maybe not… either way there will be more logging in this area (from your description it seems you can reproduce it pretty easily?).
I replicated your sequence, hooked up to the SBIG simulator and took hundreds of 1 second flat images without issue… I’m working blind so any feedback is much appreciated.
OK @jmacon . Some minor stuff has changed in this build, but it’s nothing likely to fix any reported issue. It does, however, contain some more interesting trace logging. This is not an installer. Please replace the exe in your install directory with this:
Hi Ken,
Here are some logs for this latest version 13.
The first log shows a failure saving the 3rd image.
In the 2nd log it ran ok for about 10 images and I paused the run to change the delay from 2 to 0. On resumption it failed immediately.
The next couple of runs failed quickly.
I did not touch the keyboard during these runs, so no threads besides the one doing the image download had any work to do. Also, there is no connection to mount, focuser, plate solver, weather monitor, or dome.
I have included the sgf I have been using. It might make a difference in your simulation runs.
thanks for all your hard work in tracking this down.
Jerry
@Ken, I don’t know why I didn’t think of this earlier. You have a Canon 6D and I have two of them. This fails with all 3 of my camera brands: SBIG, Canon, Nikon.
You should be able to reproduce this with your hardware.
Here are my just produced logs:
The first 2 are the SBIG, but 1 ends differently than the others have.
The next 2 are using my Canon 6D.
I have included my sgf for the Canon run.
My hardware:
Windows 8.1 Pro
8 month old i5-4660 3.2 ghz quad Intel pc. 8gbyte ram. Dedicated to running SGP and associated programs. In this case, nothing much else running since I am taking flats. No filter, mount, plate solver, safety monitor, guider, dome.
Canon 6D connected to pc through USB 3.0 7-port powered hub, thoroughly tested over several months against 4 or 5 other hubs I have. The SBIG connects through ethernet, the most reliable possible camera connection.
If you can’t reproduce this problem on your hardware, I would conclude that maybe it has something to do with Win 8.1 vs Win 10 (which I think you are running under).
One other possibility might be some bad interaction with other software running on the pc, such as anti-virus software.
This idea is not supported by the fact that I could never get this failure to occur with 2.5.0.10 but does happen easily with both earlier and later versions.
Right now, it is in Jared’s possession while he addresses some mirror lock and live view issues. We can certainly coordinate this, just not immediately.
Are you sure about this? The logs seem to indicate saving is going fine, but there is a failure to display the resultant bitmap preview.
This is both interesting and disheartening as it seems like there are 2 separate issues:
An issue with cross-thread rendering (SBIG)
An issue where RAW data extraction fails in a bad way (hanging) (Canon)
Not sure about the RAW data one. That said… the one where image fails to render the bitmap preview (SBIG) is less an issue of reproduction at this point and more of a discovery process as to why. You could be right… there may be subtle ways in which the .NET runtimes function by OS. Not sure… Anyhow, I have 2 ideas. The first one is here in 2.5.0.14 (please continue with the SBIG so we can look at just one issue at a time):
Two final thoughts notes:
Your SBIG drivers seem a bit out of date. The current release is 4.9 build 1 (this is just FYI… as it does not seem to be a contributor here)
This seems like a pretty major issue and I am trying to fathom why more people don’t see it (it is certainly present beyond your reports… but we only have 3 or 4 ppl right now). It leaves me wondering if the issue is code or environment related.
Hello all,
Downloaded the 2.5.0.14 .EXE file and thought I’d have a go so I set up a simple sequence with simulators for x60 LIGHT exposures of 5s each. Although the crash didn’t come something rather interesting overwhelmed the Log so I thought I’d post it.
Log File (its huge so you might have to wait a while, we are talking 43Mb)…Hmmm, upload failed probably cos it was too big !
@Ken, after extensive testing of 2.5.0.14, I can report the following results showing a major change in behavior:
With all other versions (excepting 2.5.0.10) it ALWAYS crashed within the first 2 or 3 images for the initial several reruns of a sequence to take lights. After several crashed reruns, it would finally get into a mode where it would finish my 240 images, 40 per filter. I can also report that I had 1 crash this morning running 2.5.0.10 early in a sequence of 4 minute darks for my Canon 6D. I had thought that version 10 had fixed this issue, apparently not completely.
This version is majorly improved: it is now VERY DIFFICULT to trigger a failure. But I have had 3 crashes in a couple of hours of testing and many runs. Logs attached. And this improvement is true for both the SBIG and Canon 6D cameras.
Additionally, for a sanity check, just in case some unknown factor had suddenly coincidentally caused this change in behavior, I swapped back and forth between versions 14 and 12 a couple of times. Sure enough, version 12 continues to crash quickly and version 14 does not.
Also, no long logs with the error @pscammp has reported above.
@jmacon Thanks for the prompt feedback. That is good news. I have one more idea that I’m hoping might eradicate this issue for good. I’ll try to get that out ASAP. Of the 3 logs you sent, only two hung for the reasons discussed in this thread, the 3rd was an unceremonious failure of the SBIG driver itself. Stay tuned…
@pscammp The issue you experienced is not related to this thread. That said it has been around for a while and your particular testing methodology exposed it. This has also been fixed.
@jmacon Just want to clarify the terminology of “crash” in the context of 2.5.0.14. When you use this term, do you mean that the application completely shuts down by itself? or that it hangs, becomes unresponsive and you need to take external measures to shut it down?
@Ken, it crashes. Pops up the app terminate message from windows, then terminates after closing the message box. That has always been the behavior with all the releases when taking lights and darks.
And on the 1 log of the last 3 I sent you I did have an SBIG driver failure. When looking through the many logs I generated yesterday for testing I only remembered 2 crashes, but got a bit confused with all the logs.
@Ken, here are 4 logs for my runs doing lights with 2.5.0.15:
Same behavior as with prior releases, crashes after taking between 10 and 150 images with exposures around 0 to 2 seconds. Longer exposures of 10 to 30 seconds did not crash for me, but I did not let them go that many reps.
A possible slight change in behavior is it seems to take more reps before it crashes.
I also need to clarify a point here: this does not just happen taking quick back to back images such as lights or bias frames. I have had a number of crashes taking 4 minute darks, so I expect this is going to show up in a normal imaging session.
@Ken, release 15 may have a new issue, or perhaps the latest SBIG drivers are weird.
Part way through my earlier testing (4 logs sent above), I upgraded my SBIG drivers to the latest, per your suggestion. Thanks for the heads up.
Some of those crashes were with the new drivers.
Here is a log and screen images of this issue:
Went to lunch, farmers market, etc., on return started testing some more. Could not get SBIG to fail, until this new issue that appeared on the run with log and screen shots attached. This same sequence but with only the L and 3 nb filters ran completely perfectly one time before this run, so I was really scratching my head. So I ran it again, adding the R,G and B filters.
This time it got through 222 L 1 second frames, 40 R .23 sec frames, then started the weird duping of images. By the time it finished the B frames and went into the Ha frames, it was taking 3 minutes/frame to write the Ha frames, but they are labeled B. However, checking the filter, the Ha filter was actually (and correctly) loaded, but it was labeling the images as B.
I also ran about 5 runs of .2 second lights, 80 frames each, with the Canon 6D. All ran perfectly. Earlier releases would have crashed numerous times.
OK. Thx. I am going to revert this to code the version I used in 2.5.0.14 (as opposed to 15). It seems like it might have been more stable overall (not perfect). I am officially out of ideas. Will need time to experiement and research at this point.
It would probably be best if jmacon ships his camera (and possibly laptop as well) to SGP so that SGP can do better testing. There’s too much guess work going on.
Or try use TeamViewer so that SGP can see what’s going on directly.
@topboxman, earlier here I offered Ken to log on to my obs and run my rig from his computer, which he says would not help that much, since I do not have his development environment installed on my pc there.
However, the best possible next step is for him to recover his Canon 6D which Jared is now working with (which easily produces this problem for me) and debug into it. I am fairly sure he will be able to trigger this as easily as I do, if not, we will learn a lot from that.
In the interim I think it is a good next step to go with the ver 14 code, which for me has been quite stable. I ran all night before last with it with no issues.
@jmacon Installed with ASCOM is a camera simulator (and CFW sim). I am curious if you can reproduce the crash with this camera. This would also tell us a lot. I’ll get the 6D from Jared this week.