That is an interesting point. Without any proof, I suspect it has more to do with the full well depth rather than the ADC bit-depth - as the resolution limit in a perfect sensor are the electrons themselves?
@buzz The ADU depth is what you get as data. Any extra information from full well is lost to quantization errors.
@jon.rista Stark is showing that stacking helps average out quantization errors Quantization errors are one of the sources of random errors that stacking helps average - that is well understood. However, that is not creating new signal (information) it is just noise. You can’t create new information bits out of random noise despite Stark’s suggestions. If I take lots of pictures of a TV that is displaying random noise and stack them do I get a meaningful picture? With enough images the noise would be reduced to some neutral gray but no new information appears.
So again I ask, what do you think is the upper limit of bit depth obtainable from stacking? Let’s talk actual numbers.
I think you have latched onto “bit” too tightly.
We are talking about discrete steps of information. The number of discrete steps of information that can be discerned in an image is relative to the noise in the image. Reduce the scale noise of the noise relative to the range of the signal, increase the number of steps of information.
If you use integer accumulators, then the improvement in the number of steps of information is limited by the bit depth of the accumulator. No one does this. We use 32-bit or 64-bit floating point accumulators these days, which reduced any error introduced during the integration process to effectively meaningless levels.
The number of discrete steps of information in a single sub may be 2^12, but if you stack a bunch of subs, you reduce the noise, this creating more room for additional discrete steps of information to be discerned where previously you could not. Where an integer might only be able to represent 128 and 129, a floating point could represent 128.2, 128.5, 128.6, 128.9, 129.1, 129.5, etc. In fact, floating point could represent thousands or tens of thousands or more values between 128 and 129, and as you average the chunky 12-bit data, you can fill in more and more of the cap between 128 and 129 across the pixels of the image.
Assuming you handle the data properly, I do not believe there is a limit here. The only potential limiter, in fact, is FPN, which I believe you are totally ignoring. FPN is the only potential asymptote here, as it is effectively a signal itself, from the hardware, that is revealed through stacking. FTR, EVERY camera has FPN, CCD and CMOS alike. The difference is that CMOS may have more non-random pattern, while CCD tends to have more spatially random FPN. In either case, they both have it, and it can and will become a limiting factor if it is not dealt with.
If you properly calibrate, which will remove the vast majority of FPN, then you can stack significant numbers of frames before running into any practical limits. Calibration is imperfect, and there can be remnants of FPN left behind. In fact, the pattern of noise in a calibration master is a form of FPN itself, and that pattern, if you stack enough data, will ultimately become a limiting factor.
Remnant FPN can be dealt with by dithering. Dither your data enough, and you effectively impart a temporal component (through the timespan of the stack) to patterns that were otherwide fixed in time. Once remnant FPN is randomized, then it is all just temporal noise, and there are no effective limits on how much you can stack. The more you stack, the more you reduce the amount of noise, and the more you increase the number of discrete steps of information that can be separated in the resulting integrated image.
You should look into the Dragonfly Project. This is a scientific project, utilizing an array of 50 scopes each with their own KAF-8300 camera. They routinely stack tens of thousands of frames (50-60k subs or so, I think more recently they have been pushing 80k), and have noted no systematic limitations on how much data can be stacked or how much information can continue to be extracted. They are able to increase the information resolution WELL beyond the limitations of the camera, as they are detecting objects 32-33mag/sq" surface brightness, with is over 10,000 times fainter than an airglow limited sky. This is a realworld application with proper scientific rigor that demonstrates that you can indeed keep stacking, and stacking, and stacking, and continue to extract more and more information. Stacking 60k subs, the effective bit depth of the DragonFly data would be around 19.5 bits, assuming the read noise of the KAF-8300 cameras is around 8e-.
There are other scientific papers out there that describe the benefits of stacking, not just on SNR, but also on “bit depth” or “dynamic range”, both of which really just refer to “discrete steps of information”. Don’t get hung up on “bits”, it’s just a descriptive term. We are talking about discrete steps of information. Stacking definitely increases that…assuming you use high precision floating point accumulators.
1 Like
@jon.rista Let’s examine Stark’s conclusions a bit. He says “Thus, given the noise in the sampling process, a 100-200 stack length of 8-bit images yields quantization errors on par with a 12-bit image. Further, it takes very few images to yield real gains in bit depth.” The first part of this quote is quite true. He is reducing the quantization errors. The second part of the statement is where he gets into trouble. Reducing the quantization error improves the accuracy of the 8 bits but it tells you nothing about the 9th bit.
Your argument about bits verses discrete steps of information is talking about the same concepts. But to follow that thought I again ask, what do you think is limit of the discrete steps of information possible with stacking? You don’t seem to have an answer except to say I must be wrong. If you have some source that demonstrates how one creates new information from noise, we would all like to see it.
@dts350z What is your point with these references? They are both talking about reducing noise by stacking. No disagreement about that.
@jon.rista Your reference to the Dragonfly Project is interesting but they are not creating new information from nothing. Stacking the same exposures from multiple cameras just reduces noise - it does not create new information. HDR on the other hand does create new information and is valid way to increase image depth.
They all had discussion of the limits of stacking, which you asked about.
Let’s pretend we are a computer and try stacking some images. Suppose one of the pixels of a perfect camera has a value of 100. Let’s stack 5 images so we get (100+100+100+100+100) / 5 = 100. Now let’s stack 5 images with any kind of noise (101+98+102+103+99) / 5 = 100.6 So what is the .6 part of this result? Is it new information about the pixel? No, it is residue noise that did not get averaged out.
What is messing with your intuition on this stacking concept is that you think that adding up the photons creates new information. However, each image contains the same base noiseless information so stacking produces nothing new except residue noise. The depth of the image is not increased - it’s just noise. If you are still doubting, go through the sample above again and see it in action. After you stack, you have the base image plus some residue noise.
What makes you think that the exact same number of photons is going to hit a given pixel on each exposure?
Once you realize that this assumption is incorrect, what the rest of us are saying might make more sense.
I did not make that assumption. The variations you refer to are a form of random noise. My example is of course simplified. Every real pixel consists of some base value plus noise. Looking at a single image pixel you can’t determine what is the base and what is the noise. That’s why you stack because it averages out the noise to reduce it. After stacking, each pixel is some base value plus residue noise and again you can’t tell which is which. But none of this additional detail invalidates the example. Stacking is not enhancing the base image - it is reducing the noise component. Don’t get too hung up about the concept of a “base Image”. It’s not something that you can extract from an image but it is there nevertheless.
I’m sorry, but this is fundamentally incorrect. The 0.6 is not residual noise, it represents a increase in the precision of the information. Further, a perfect camera is simply a camera that has no noise of its own…that does not mean the signal is noiseless, though. The photons themselves have intrinsic noise. They are discrete events that will arrive at each pixel randomly and in an unpredictable manner. So even with 0 read noise and 0 dark current, you would never get 100 in every sample of a value for a given pixel, nor would you get 100 in every pixel spatially either. For one, you don’t know ahead of time what the correct signal is…you don’t know that it is supposed to be 100, not with a real-world signal. Noise is INTRINSIC to the photon signal. It takes many measurements (many observations) of an intrinsically noisy signal to determine what the signal actually is. It is a statistical process. Photon shot noise in fact follows a Poisson distribution, so those statistics apply when measuring and stacking.
I don’t know how else to explain it to you, but stacking COMBINES the information from each sub. Every sub has unique information (assuming you are stacking distinct subs, and not stacking the same sub over and over). There is not a “real signal” buried somewhere within the noise. There is a real and distinct signal in each and every sub, and stacking combines the real and distinct signals from each sub to produce a new stronger signal that represents all of the information from each sub
Again, sorry, but fundamentally incorrect. Stacking is no different than taking a longer exposure, with the sole exception that stacking includes additional read noise from each sub. Assuming a camera that has zero read noise, then stacking is literally no different than just taking a longer exposure (from an information standpoint). If a longer single exposure contains more information than a single short sub, then a stack of shorter subs of the same total time also contains more information than a single short sub. That means that every single photon, regardless of whether they are captured in a single sub, or across many subs, most definitely represents MORE information. (It is not “new” information, but it is definitely MORE information.) The total amount of information in the stack is most definitely greater than the amount of information in a single sub. Stacking does this:
(Sobject1 + Sobject2 + … + SobjectN) / SQRT((Sobject1 + Sdark1 + Nread1^2) + (Sobject2 + Sdark2 + Nread^2) + … + (SobjectN + SdarkN + NreadN^2))
Note what is happening here. You are ADDING the signals from each sub linearly. That is combining information, thus the amount of information in the final integration must be greater than the amount of information in the individual subs. You are also adding the noise from each sub, however the noise is added in quadrature. So while the signal grows fast, the noise grows slowly.
This difference in the rates of signal vs. noise growth is why SNR increases, but it also means that the relative scale of the noise to the maximum range of the signal changes as well, which means we increase the “resolution” of the information in the final integration as well.
Anyway, I don’t know how else to explain it to you… We are just kind of going round about now without really getting anywhere.
1 Like
No. Every pixel contains a noisy value. There is no base value, the signal is not a known quantity ahead of time, nor is it just there waiting to be revealed. Noise is not some separate and distinct thing that is just “on top” of the signal, obscuring it.
Signal and noise are two sides of the same coin. You don’t wash away noise to reveal a signal. You integrate noisy signals to make the signal stronger relative to the noise. Signal and noise are inseparable. Noise can never reach zero, and fundamentally there is no upper limit on the magnitude of the signal.
Wrong. There is no base image. There is only a noisy signal. Stacking is just integrating signal. Signals contain noise, period. There is no such thing as a noiseless signal in the real world.
We “integrate” signals. Doesn’t matter if you just expose longer, or stack, same difference in the end. You accumulate more and more information. That information is noisy. The magnitude of the information grows linearly as you stack, however the noise grows in quadrature. In literal terms, both grow, always, you are always combining information. SNR grows because the magnitude of the signal gets larger faster than the noise gets larger as you continue to integrate information.
Averaging is simply a way of modeling the information. Summing is just another way. Fundamentally, though, integrating a signal is always combining more and more information, and more and more noise. The ratio of signal to noise changes, the relative scale of the noise to the maximum range of the signal changes, the maximum range of the signal grows, information quantity grows, precision grows. There is no asymptote here.
…
Well, I think we’ve thoroughly derailed this thread. Apologies. I’m gonna move on.
2 Likes
@jon.rista “The magnitude of the information grows linearly as you stack” Guess you should call NASA and let them know about your new discovery. I’m sure they will pleased to know all they have to do is stack more Hubble exposures and all will be revealed. Then for the next telescope launch they can use 10 two bit cameras and just stack more. Should save them a lot of money.
Here goes:
- when digitizing an analog signal, a higher bit depth reduces the sampled error and gives a finer resolution, each extra bit, halving the error state
- in the case of astrophotography, there is a slight confusion since the signal is not analog,
- a single exposure resolution is ultimately limited to the well depth number of pixel states
- integrated over a number exposures, however, the random nature of light itself creates an illusion of an analog signal and certainly each accumulated pixel has more states than the ADC
- If the effect of stacking, did not improve the perceived tonal resolution, as increasing the ADC bit depth does, then an experiment would easily confirm this looking for missing bits in a single exposure (12 bit ADC scaled to 16-bit) vs. a stacked set of images.
The answer to the upper limit is not something that I can find an immediate reference to. That is not the same thing as discounting it. It is the same argument to the SNR limit, which is asymptotic to the sqrt(signal).
If I took a flat frame in the middle of the range, its shot noise is considerably higher than any sensor noise. The photons hit the sensor randomly but at an average rate that is perfectly linear to the light intensity. Assuming a gaussian noise distribution, I think the ultimate effective bit-depth by stacking is the same as the accumulator bit depth used during stacking. To make the most of a 32-bit FITS file it would then require over 1m samples to shift 12-bits to 32.
This is the precise equivalent to signal oversampling in ADCs. Here an ADC samples well above the signal (and noise) Nyquist frequency and then the output is low pass filtered. In our case, the integration is the low pass filter. So, for example 256x oversampling causes a 16-bit ADC to behave as a 24-bit.
2 Likes
Let’s look at the math behind my simple example. We will stack n single pixels - all the other pixels in the frame are treated the same, There is no inter pixel interaction. Each pixel in the stack is S[i] + N[i]. The signal of the i th element plus the noise of the i th element. (In the follow steps Sum() means summing each element from i=1 to i= n).
Let’s create the mean of all the signal values.
B = Sum(S[i])/n.
Now define D[i] = S[i] - B. The difference between each signal value and the mean.
So with substitution we have each pixel as:
B + D[i] + N[i]
Each pixel is a base mean noiseless value plus the difference from the mean plus noise.
Now let’s stack these pixels and we get:
Sum(B + D[i] + N[i]) / n
which reduces to
B + Sum(D[i])/n + Sum(N[i])/n
Because of the way we defined D[i] the second term of this equation is zero. So we are left with
B + Sum(N[i])/n
This is the base noiseless mean value plus the average of the noise. As has been said before, the residue average noise term converges to zero by the square root of n. So using simple algebra we can see that stacking does not change the base noiseless mean image.
Doing stacking on a computer creates some rounding errors but those errors degrade the result not enhance it.
If you wish to fault this result, please do so by pointing out some incorrect math step if any. These are general equations that only assume that the pixel has some useful signal in it. Any arguments about real world noise, cameras, etc. don’t mater because nothing else was assumed. You can’t argue that each pixel is not a sum of signal and noise.
I’m going to point out that while my equations are correct. It does not answer the question of whether the mean of the signal values can be used to predict lower digits. This is a probability question. If I have two value 2 & 3, the mean is 2.5. What is the probability that .5 is correct? There is a 1 in 10 chance that it is correct. If I now find the mean of 100 integers to be 2.57, I have a 1 in 100 chance that .57 is correct. What is the chance that the .5 is correct? It is still 1 in 10. The chance of .00 through .09 is 1 in 10, the chance of .10 through .19 is 1 in 10 and so forth. So increasing my count has not improved the probability that I can predict the next digit. So adding more samples is not increasing my chance of knowing the next digit. This analysis assumes the lower digits are random. Since we are dealing with noisy signals it is not a stretch to assume the lower digits below the ADU floor are nearly random.
Moving on too -
In extreme synthesis, the entire process is flawed because the pixel value in each exposure is a random variable. In statistics you just can’t add up random variables like they were simple “numbers”. Basically, you may only talk about expected values, variances and distributions of combinations of random variables (the stacking process being a linear combination).
One thing you could do is to assume that signal is following a Poisson distribution, that noise follows a gaussian distribution with mean = 0 (so assuming that FPN=0) and use the Central Limit Theorem to determine that the stacked image (which, by the way, is again a RANDOM VARIABLE) may be considered to have a Normal (i.e. Gaussian) distribution with a sufficiently high value of n (typically, this is verified for n > 30 no matter which statistical distribution the samples - i.e. the single exposures - follow). That would lead to a conclusione that is similar to yours (but not identical), i.e. that the stacked average value has a mean equal to the lambda parameter of the Poisson distribution of the single exposures (which you would call improperly the “real value” B), while the standard distribution is equal to the standard distribution of the single exposures divided by the square root of n.
Translated in simple words, the “bell-shaped” curve representing the distribution of the stacked average gets narrower and narrower around a mean equal to lambda (the expected value of number of electrons per exposure) as n grows. We can say that the limit of the stacked average for n growing to infinity is actually lambda. A side effect of the Central Limit Theorem is that id doesn’t matter if the exposures are discreet variables, for a large value of n we can approximate better and better the distribution of the stacked average to a real number.
But even if your proof was entirely correct, for n = infinity your calculation gives exactly B, the noiseless, “real” value that you wanted to measure through multiple exposures: you reached infinite precision. Ironically, that’s exactly the contrary of what you wanted to proof.
I just can’t comment on this because I don’t understand what you’re saying. That’s probably my fault, my English knowledge is limited.
I raise white flag. No reason to go any further. I leave too. @jon.rista wait for me, let’s have a fresh beer.
Then you’re invited too, Chris!
1 Like