For cases where your individual sub exposures are long (5 or 10 minutes and up) this does not make a big difference. However, with the newer low-read noise CMOS cameras, it is sometimes desirable to collect hundreds of very short exposures (600 x 30 seconds for example). With so many subs in a given imaging session, a 10 second delay for downloading + saving the last frame begins to add up to a significant fraction of the total imaging time in an outing.
There might be the chance of having the download status of cameras in the API well published (is already as status there, but GUI shows download whereas API still says integrating)
I’m pretty sure this is not possible in CCD cameras, maybe it is in CMOS cameras?
Why can’t it work in CCD cameras? Because “download” time isn’t actually just the download time, part of that time is also the sensor readout time which is slow for CCD sensors unless there’s several readout channels.
I see this problem all the time with my ASI1600MM and laptop.
Maxim 4.62 and Sharpcap can capture bias frames at about 2seconds per frame.
SGP 3.0.2.91 using ASCOM takes 6 seconds per frame!
Using the ZWO direct driver it take around 3.5 seconds per frame.
I would think the code would offload saving/image analysis to another thread while getting the next image integration started so it would be almost instantaneous but it appears to be very serialized.
Just to note – no camera can download the current image and start taking a new image at the same time. The current image must be transferred off the chip before a new image can start. Currently that means transferring the data all the way to the PC. Next generation cameras may / should be designed to have internal memory chips. The data could then be transferred from the imaging chip to the internal memory very quickly and the chip would then be ready to start a new pix. The data in the memory buffer could then be downloaded to the PC at the much slower rate while the new image was being exposed.
This might cut 10 to 30 seconds off the time lost between images – speed costs money; how fast do you want to go?
BTW – the new Scientific CMOS based cameras from FLI can download a 16 megapixel image (a KAF-16803 size chip) in one tenth of a second. It only costs $20K.
The new Mark II cameras from Moravian can download a KAF-8300 image in about 3 seconds in high quality mode. They are not $20K.
Hi Charlie, those “Next Generation” cameras already exist That’s exactly what I’m talking about.
The ASI1600MM-Pro, QHY163 and Atik Horizon (and other current generation mono-CMOS astro cams) all have DDR memory buffers on board. The image is read off the chip and into the buffer very quickly. The competitor program NINA already can start the next sub frame while download/save operations are performed on the previous frame. NINA also will perform dither operations while downloading/saving the current frame.
As it is currently, SGP performs download and save operations prior to starting the next sub. This wastes a lot of time, especially the save operation for people running low power PCs in their imaging rigs with traditional 2.5" spinning disk HDs. For me, even over USB3, download and save together take over 20 seconds. If you want to do hundreds of subs in a night, this is an enormous inefficiency.
Have you read about some of the new imaging techniques emerging with these new CMOS based astronomy cameras? With a camera like the ASI1600, you can get above the noise floor in as little as 30 seconds in some cases (conditions depending of course). A lot of people are finding that a very viable imaging technique is to take hundreds of very short subs (even over 1000). You don’t have the deleterious effects of build up in read-noise that would occur with traditional CCD camera (it still happens just with many many more subs). There are some advantages of this method, including less criticality placed on the mounts guiding capabilities and better statistical pixel rejection in pre-processing. I encourage anyone interested to go read the many threads on Cloudy Nights about this subject.
For use-cases like these, even 10 seconds between frames is a lot of wasted time when you’re doing hundreds of subs a night. As I said, this feature is already implemented in a competitor imaging suite and it would be nice to see SGP keep pace as I like it better.
Slow downloads and even slow saves to disk have been a problem with SGP for a long time (since I started using SGP in 2016). I’m pretty sure I communicated this problem to the SGP team almost two years ago and it’s still a problem, at least with v2.6. Now it looks like they have the same issue with the new v3.
One of the problems is that it looks like they don’t even buffer the image so that it can be written to storage using a few large writes. Instead it seems that they write to disk using very small memory buffers (a few KB per write?) which really slows down the i/o. What they need to do is buffer the download until they have a decent amount to write and then write that large buffer to disk using either a separate thread or just with a fully asynchronous disk write. In fact, given that all PCs today have GBs of DRAM memory there is really no reason why they shouldn’t buffer the entire download into memory before they write anything to disk.
You can watch the disk writes in the Windows’ performance monitor and SGP never even reaches 10MB/second even with a fast SSD. It almost looks like they try to interleave the download from the camera with the disk i/o in a completely serialized manner and all of that is done in the same (main?) thread.
Of course, they may already be trying to do some of this in the current release, but if that is the case then it certainly isn’t working very well (or as well as it could).
If you look at the ASCOM standard, you get the image all at once - not in pieces. That’s not saying the ASCOM driver is not downloading in chunks but SGP gets it all at once.
I’m pretty sure that my camera under the v3 version of SGP uses the native driver from ZWO, not ASCOM. I think this is one of the changes between v2 and v3, and I’m pretty sure that I enabled the native driver under SGP v3.
However, none of this explains why SGP is so slow to write the image to disk. There seem to be two different phases for the download and saving to disk. You can watch the progress bar and within a second or so the “Download” progress moves fairly quickly to 100% and then there is a short pause and the progress bar moves back to zero and then slowly moves toward 100% once again. I don’t know what that cycle represents, but it may be the download from the camera followed by the progress for the write to disk. Either that or the first progress bar just represents the time that SGP waits for the download to begin.
But, if that is true it shouldn’t take SGP several seconds to write a 40MB data file to an SSD (unless they are writing it in very small pieces). The latter seems to be what is happening since you can see the rate of the disk i/o go up and down for several seconds as each file is written to disk. Also, I have the image history turned OFF and I’m running on a Core i3 with 20GB of DRAM.
I have a solid state drive so the write is pretty fast for me. I don’t think SGP is the problem which not to say they couldn’t make improvements. I’d look at your hardware configuration and/or whatever other software is running.
That’s exactly what I’m talking about.