I have been following the latest developments on CMOS cameras with interest and the concept of lucky imaging in general. It certainly seems that there is a new direction coming astrophotographer’s way, using high speed cameras and plenty of fast, cheap storage on USB3. This is going to require some top-down thinking with acquisition packages.
I’m puzzled though. When there are still some general implementation wrinkles to be ironed out and some outstanding fixes, why is ZWO been singled out for special attention in a beta? I can see that there are a few similar manufacturers, using cooled CMOS chips, all with attractive prices and with a range of mechanical and software quality. Surely, when a manufacturer’s device drivers are so-so, it is often an indicator of other things and I wonder about the decision to spend precious development time supporting it. This could be a TemperHum on steroids.
I can see that there are a few similar manufacturers, using cooled CMOS chips, all with attractive prices and with a range of mechanical and software quality.
Really? I do not seem to know them. There are not many companies that offer a comparable product line that broad and up-to-date like ZWO does. Their only competitor is QHY. Any more companies? OK, Atik offers one product (with Panasonic MN34230). However ZWO (and to a lesser extent QHY) has several cameras with current Sony CMOS sensors in their portfolio.
ZWO recognized early that the new CMOS sensors have a field of application not only in planetary photography, Electronically Assisted Astronomy and lucky imaging but also in classical deep sky photography (time exposure → cooled devices). They react rapidly to new sensors. Meanwhile cooled CMOS cameras by ZWO in different formats (from 4/3, APS-C to full frame) are widespread among astrophotographers, and (perhaps more important) their percentage is rapidly growing. This can be inferred from the numerous threads in different forums dedicated to astrophotography. The developers seem to use ZWO cameras too at times.
I’m puzzled though. When there are still some general implementation wrinkles to be ironed out and some outstanding fixes, why is ZWO been singled out for special attention in a beta?
So the answer is obvious: it is the widespread use of ZWO cameras that led to special care about their products.
As to the need of spending precious development time: I don’t know the driver issues and cannot judge whether the effort is justified. Essentially, here I agree with you, it is the duty of the maufacturer to deliver a flawlessly working driver for the device. Yet my cooled CMOS camera is shipping just now. Will test the ZWO ASCOM driver as well as SGP’s native support in v3 Beta when the time comes.
This could be a TemperHum on steroids.
I don’t think that ZWO cameras are a flash in the pan like the TemperHum.
Because the cost to support the ASCOM implementation was getting sufficiently high that it made sense to write a native implementation. Essentially, for whatever reason, the ZWO Ascom driver and SGP don’t play nice together. We brought in the driver so we could actually remove some controls from the user which seem problematic (like USB limit).
The truth is that ZWO integration has been mostly done for a few months, but you’re just now seeing it. We didn’t “drop everything” to work on this for 3.0, we just added it in as it was ready and the Beta was a good time to do that. Same with INDI which is “mostly done” and will probably be seeing light in a few months once we move some things around in the devices that had previously only relied on ASCOM (focusers primarily)
The QHY ASCOM drivers work just fine so we don’t really have a reason to pull in native support for QHY. If we end up exposing offset later on then it may make sense to do so.
Thanks Jared. I’m broadly familiar with writing embedded code and ASCOM drivers. I found the driver much easier and so it worries me that if their driver has issues, it is an indicator of the internal quality. Time will tell and as bad news travels faster than the speed of light, we will soon see.
If you, or anyone else, has a ZWO camera and a little time to spare I’d be interested in seeing what the results of the ASCOM Conform application is with this camera. This could give good information on what is going on.
I understand this reasoning, but there is another fairly compelling reason to add native support for QHY cameras - download speed. With these wonderful CMOS cameras, people are choosing much shorter exposure lengths and while the cameras tend to download images much faster than CCD cameras, they download a lot faster natively compared to via the ASCOM driver. My QHY163M will download a full frame almost instantly in SharpCap using native support, but may take 4 seconds or so in SGP using the ASCOM driver. 4 seconds may not sound like a lot, but if one is doing “lucky imaging” with exposure of a few seconds, it becomes a huge overhead.
Also, and perhaps a topic for another feature thread, I wonder if there is a way to disable settling in PHD2 when not dithering. This ends up being a source of additional overhead of precious seconds when using “dither every X frames”. For some reason (maybe because of potential shake from a shutter closing?) PHD2 settles guiding even when no dithering occurs. I presume this is because SGP tells the PHD2 server that an image is complete. If there is no dithering on a shutterless camera, there should be no need to settle guiding between frames. This can also take several seconds adding significantly to time wasted when lucky imaging. So I wonder if there is a way to disable this?
I have worked a fair amount with the zwo api to control an asi1600 camera - and here are some numbers comparing exposure/download times to sgp.
If I use the frame and focus tool with the asi1600 over usb 3.0 - the exposure and download time for 1ms exposures is just over 5 seconds with image history enabled - and just over 2 seconds with it disabled.
So a big part of the delay people may be seeing is due to the image analysis and star measurement that are part of the image history tool.
But 2 seconds is still a fair amount of overhead.
If I do sustained video recording I can get just over 8 fps with full frame 16-bit images - which agrees with what sharpcap can do. So there is no magic in sharpcap that goes beyond what the zwo api can achieve.
If I do a single expose/download with my code, it takes 0.21 seconds to start and complete the single exposure, then 0.08 seconds to download it, then 0.11s to save it to fits on an ssd drive.
If I do a 10s exposure, there is 0.18s overhead to take the exposure, then 0.05s to download it, then 0.09s to save it.
So there is some fluctuation in these numbers, but about 0.2s overhead to setup and take the exposure and around 0.2s to download and save it to fits.
In video mode there is much less overhead. The ideal way to do lucky imaging would be to run in video mode for a time, then stop and dither, then continue video mode. The win for lucky imaging in that case compared to the current overhead would be huge. But sgp isn’t really designed for lucky imaging in the first place. And I don’t think sharpcap is set up for dithering - I’m not sure.
But if sgp is already working directly with the zwo api, and if a fair number of people want to operate in lucky imaging mode with occasional dithering - there is a huge win possible.
There’s a separate ASCOM Video interface that’s designed for video cameras. I’ve not been involved in it at all.
From what Jared said it was the difficulty of integrating with the ZWO camera that was the problem that caused them to do their own driver, not the download speed. It’s that which I would like to get some information about.
Downloading images at a higher rate may increase the noise because you have to have a shorter time constant on the analogue amplifier for the ADC.
By “video mode” I just mean with the camera in free-running output of continuous exposures - which is one of the modes of the zwo API. You just set the exposure and start it capturing video - then catch and store the frames as they come in in a separate thread. There is a separate “high speed” switch - but the rates I describe above aren’t high speed and aren’t compromised at all in terms of quality.
I don’t think SGP is set up to operate in that mode and it’s understandable - since the main focus is sequenced long exposure individual images. But I think if you really wanted to operate the camera at its full potential throughput you would need to use this mode.
This doesn’t take much code at all to do - and the only problem I had was getting some of the bandwidth and timeout values set so that it didn’t hang on download.
I should mention this is one of the early asi1600 cameras, and with sgp I am still at version 2.6.0.23. I don’t know if any of the timing stuff has changed.
If I run the camera continuously in frame and focus mode, each exposure looks like this in the log - shown below. The Start… marks the start of the exposure process and “Done” happens only about 0.2s later - which is about right for a single exposure of 1ms. But then there is all this other stuff going on - some of which I think is disabled under the covers. The log says that “find stars took 2064ms” - but find stars is disabled and I don’t think that routine is actually running. So I don’t know what is happening in all that time.
But it’s pretty clear this is set up for individual exposures - with a fair amount of internal overhead that is taking a lot of time. So even if it didn’t operate in video mode, it seems like the overhead could be reduced. In video mode the max rate is about 8 fps, and in individual exposure mode it is about 1/(0.4s) = 2.5 fps. So even in individual exposure mode - there is potential to greatly speed up lucky imaging if the overhead can be reduced.
I also include the requested Conform file for the camera. It has 5 errors. I don’t know conform very well - I hope I ran it correctly.
Frank
[12/24/17 09:58:46.722][DEBUG] [Camera Thread] SaveFileAscom: Start…
[12/24/17 09:58:46.722][DEBUG] [Camera Thread] SaveFileAscom: Checking image data…
[12/24/17 09:58:46.722][DEBUG] [Camera Thread] SaveFileAscom: Create normal preview bitmap…
[12/24/17 09:58:46.729][DEBUG] [Camera Thread] SaveFileAscom: Locking preview bits…
[12/24/17 09:58:46.729][DEBUG] [Camera Thread] SaveFileAscom: Starting byte traversal…
[12/24/17 09:58:46.917][DEBUG] [Camera Thread] SaveFileAscom: Unlocking preview bits…
[12/24/17 09:58:46.917][DEBUG] [Camera Thread] SaveFileAscom: Done
[12/24/17 09:58:46.918][DEBUG] [Camera Thread] Internal Interface: Set Preview…
[12/24/17 09:58:46.919][DEBUG] [Camera Thread] Display image preview using asynch task…
[12/24/17 09:58:47.198][DEBUG] [Main Thread] --> Find stars (normal)
[12/24/17 09:58:47.301][DEBUG] [Main Thread] AF frame was too large… downsample = 0.5…
[12/24/17 09:58:49.210][DEBUG] [Main Thread] Star detection using min star size of 2px…
[12/24/17 09:58:49.210][DEBUG] [Main Thread] Star detection using max star size of 40px…
[12/24/17 09:58:49.262][DEBUG] [Main Thread] Find stars took: 2064 ms…
[12/24/17 09:58:49.387][DEBUG] [Camera Thread] SGM_FRAME_AND_FOCUS complete…
[12/24/17 09:58:57.193][DEBUG] [Camera Thread] SGM_FRAME_AND_FOCUS message received…
[12/24/17 09:58:57.199][DEBUG] [Camera Thread] ASCOM camera: frame and focus…
[12/24/17 09:58:57.200][DEBUG] [Camera Thread] SetAscomNormalSpeed…
[12/24/17 09:58:57.200][DEBUG] [Camera Thread] Cannot set readout speed, not supported by camera…
[12/24/17 09:58:58.838][DEBUG] [Camera Thread] SaveFileAscom: Start…
This is touching on what was in the back of my mind and my earlier post… that what was being required falls between traditional video modes and single frame capture mode. I don’t know the particulars of this camera but I do know that many video modes compress output to save bandwidth with keyframes every so often. It seems that there is a need for rapid full-frame download, with minimum overhead and equally quick save. I notice that several of the CMOS cameras are now putting in large buffer storage within the camera to prevent readout noise issues.
For the asi1600 and most monochrome astro or scientific cameras - the video output is raw and uncompressed. So there is no downside to doing video output from the camera at 8 fps or so. And with 1 second exposures it would be about 1 fps - with plenty of time to store the raw exposures.
The 5 errors and 3 issues aren’t critical, more relatively minor issues to do with details of the specification. None of these are difficult to fix, especially as Conform tells you exactly what the problem is.
I wouldn’t expect these to cause enough of a problem to make the driver unusable. Conform is starting an exposure, waiting for it to complete, and reading the image array. The array it gets is the expected size.
One thing is that the optional properties LastExposureDuration and LastExposureStartTime are not implemented. The driver throws the correct error message.
This could be what’s happening, Ken and Jared have a tendency to ask for more than ASCOM compliance and may require that these properties are implemented. If that’s the case writing and maintaining their own support code seems a little OTT, these property reads could easily use the try - catch pattern and if the property is not implemented uses local estimates. This would work for all drivers.