Image output Duplicating and Skipping problem

I started an all night sequence early this evening to create 320 Bias subs and a full nights worth of 240 second Dark subs on both my SBIG8300M and my Canon 6D cameras.
The good news: no crashes on either one (so far) after 7 hours of first the Bias frames and then non-stop darks.
The not so good news: the SBIG8300M bias directory looks like this for the first 30 frames:


Approximately 5 out every 30 frames are a duplication of the prior frame. Total image count at the bottom shows 319 images, but the sequence dialog showed exactly 320 frames completed, as it should. So around 50 were duplicated and one was skipped.

And here is an image of the last 30 frames of the Canon 6D bias directory:


No duplicated frames, but there are 23 missing frames, the total count at the bottom showing 297 images and the sequence dialog showed the correct 320 processed frames.
Note that frame 0292 is missing.
And there is a consistent 8 seconds between all the listed frames.

The SBIG bias run used 0 delay seconds between frames. The 6D run used 1 second delay between frames.
I set the delay to 1 second for the SBIG and ran 80 more frames…same result. Duplications.

I noticed the same frame duplication last night. I assumed that it had to due with marking frames as “Bad” and then checking the box to decrement the image count, but unless you marked some bias frames as “Bad” my assumption is apparently incorrect.

Tim

Update to initial post:
Last nights marathon Bias/Dark collection on 2 cameras ended with no crashes on either one. I love it!

Canon 6D Darks took 120x240s with no issues, perfect run.

SBIG 8300 said to took the 120x240s that I requested, but actually took 121, having inserted a duplicate (perhaps) of which one? Hard to tell from the timeline.

Odd indeed. Do you have logs for either of the runs corresponding to the screen shots? They might show something that caused a frame restart (or possibly some weird entry into the new-ish “calibration frame mode”.

In the screen shot above there is a much larger time gap for the duplicate frame (0055) than the others. Did you stop and restart the sequence here or is the just the way it folded out?

Ken, I have the logs for all these runs on my obs pc. Unfortunately I am in Colorado now and for the first time all winter my obs pc is not visible to the internet. There have been snow storms there for a week and maybe caused some kind of service interruption. Fortunately I am going down tomorrow and will fix it, then I can upload the logs, and the few images before 0055 to do a file compare and see if it is a dup or not. I have not been there for 10 weeks so I have a lot of hardware shuffling to do and a full rc12 collimation. Bad luck, I had just started to download those 4 images when I discovered it was down.

The dark run for the SBIG I stopped part way through the Darks so I could add more Bias frames since I wanted 320 and about 60 of those were dups. So I restarted the SBIG run to finish the extra 80 bias then continue to finish the 120 darks. That was the only break in the run and occurred before I went to bed sometime around 11:00 pm. No irregularities at that time. The extra 0055_L-1 occurred at 3:08:48 AM on its own authority.

The SBIG apparent Bias dups I think are really dups of the prior since they closely follow with 0 or 1 second separation. I will double check that too.

And a key difference here is for SBIG, there are about 1 of 6 dups for Bias frames (very short exposures), and 1 of 120 for Dark frames (long exposures). Sort of looks like a timing issue and/or thread issue.

Here are the logs:

The following zip contains the clues to why there was 1 apparent dup in the set of Darks.
Turns out 0055_L is actually one of the Bias frames, or so its header says, using file compare.

This shows us a completely different problem, caused by this sequence of actions:

  1. I ran the Bias frames early in the evening before going to bed (target #1).
  2. Target #2 was the Darks which got a good start before I went to bed.
  3. I got up at ~2:50 and checked on it. Noticed so many dups in the Bias run so…
  4. Paused the run, cleared finish flag in #1 and extended the count an additional 80 frames, then restarted the sequence.
  5. It restarted by continuing #2. I thought it should restart with #1 to finish my additional 80 Bias frames.
  6. So I pause it again then disable #2 then restart. Proceeds to work on the new Bias frames. I turn #2 back on and go to bed.
  7. It finishes target #1 (Bias) then finishes target #2 (Dark).
    Dropbox - File Deleted

So you are suspecting that the data is accurately captured, but the naming of the files is wrong which makes it look like duplicates?

Not following this (meaning I don’t see another problem described here).

This is by design. SGPro uses “sticky” targets and events. Pausing and restarting will resume wherever you left off. If you want to change that, no need to disable events, just click on the event “gear” and “set as current event”.

Is that the new problem you are writing about? Or am I just being dense and you describing something else?

I will go through the logs shortly and see what’s going on.

Also… your file naming pattern looks odd to me:

%bn%bn_t%es_%bi_%fz_%su

and

%bn%bn_%mm%dy_t%es_%bi_%su_%fz

Looks like frame type has had the “f” removed. This could cause a lot of your frames to have duplicate names and create the appearance of duplicate frames all by itself…

Any clarifications are appreciated.

Yes,[quote=“Ken, post:7, topic:3426”]
Not following this (meaning I don’t see another problem described here).
[/quote]
Yes. This is the completely different problem.[quote=“Ken, post:7, topic:3426”]
Looks like frame type has had the “f” removed. This could cause a lot of your frames to have duplicate names and create the appearance of duplicate frames all by itself…
[/quote]
You are referring to the _t%es parameter. I am not trying to include the frame type. It produces _t240, a nice concise display of the exposure. There is never any possibility of duplicate names because I never mix different frames types in the same target. Each target name includes the frame type which is the lead part of the file name.

I see. I’ll take a look.

Also, just a curiosity… do you find dark scaling to be detrimental to your images? For my 8300 CCD, I am getting good results, just letting PI scale my bias master in place of darks (that’s how good it seems to be)

I think this is a timing issue (I cannot reproduce), but I did fix what I believe is causing the problem…

I was always using a dark master until about 6 weeks ago when I redid some targets with no dark master. The results seemed much the same to me. I plan to do some SNR analysis on these, but so far it looks like just using the master bias may be a good way to go.
Are you actually getting better images not using darks?

Not better… just no difference. And to be clear, I do use darks, I just let PI scale my bias as my master dark.