ASTAP can provide the median HFD value of an image using the command-line. If command-line parameter -analyse is used both the median HFD and number of stars used are reported in the program exit code. But I assume that the SGP developers (I’m not) are more interested in improving there own star detection.
Focusing is a hot topic and one part of developers is using parabolic curve fitting and the other part is using hyperbolic curve fitting or something else like V-shaped fitting. The focus curve shape is also depending on the HFD/HFR detection performance/consistance.
Your Excel spreadsheet will be interesting. I think you have to drop it somewhere else and provide the link.
Thanks for your reply. Using a few of the SGP AF Packs that I have available, here is my comparison of the Star Counts and HFD values obtained from ASTAP versus the corresponding Star Counts and HFR values obtained from SGP.
My observation is that with images from my set-up at least, ASTAP is considerably more effective at star detection when using an in-focus image than SGP and majorly more so when the image is somewhat out of focus. I have played around with SGP’s Min Star Size parameter in order to obtain the highest Star Count values that I can.
To my mind, the more stars that can be included within the AF calculation then the more robust I would expect the overall result to be.
I would be interested to know what Focus Positions would be determined by SGPs curve fitting algorithms if ASTAP’s star analysis was used in place of SGP’s values but this step clearly would need access to the curve fitting routines. I have no real issue with SGP’s AF calculation when a decent number of stars are detected but less convinced that a good focus position is deternined with only a handful of results. Maybe SGP’s star detection routines need only a minor tweak to improve sensitivity.
The images used in my analysis were shpt with L and R filters both with 2x2 binning.
Interesting idea. We do something similar with allowing PinPoint to generate the focus metric. I’ll look into using ASTAP for this as well. I wasn’t aware it had that capability.
I recall reading that SGP AF has issues when trying to focus on globular cluster. As an after thought I just hada look at an M13 image in ASTAP and the HFD / #Star results look very typical for the results that I get from a non-globular image.
Han can no doubt comment more effectively on strengths and possible pitfalls with using ASTAP for deterning metrics.
Detecting less stars is not a bad sign. You do not want to include very faint stars for HFD measurement. You have to set a minimum SNR value.
What is more interesting, I have calculated from your measurements the mean HFD/HFR ratio and it is about 2.3 but it should be 2.0. It is possible that SGP misses some star flux. It probably doesn’t matter for focusing but it should be stable but it is not. The ratio varies between about 2.1 and 2.5 or even a little worse. Best way to prevent outliers is to take the median HFR value of the measured stars each time rather then mean value. but I assume that is already done.
What a brilliant idea Mike! I image at 2.8 to 4 metres focal length and often have limited stars in the field of view, and of those, SGP appears to only detect about a third when in good focus, and even less (sometimes zero) at either end of the focus run, resulting in quite noisy focus curves on some targets. At 4m focal length, stars tend to be quite soft and HFDs are quite large, so I assume that is why SGP rejects them. This is despite the fact many of the undetected stars are quite obvious and with good SNR. I’m under no illusion as to how hard it must be to design a star detection routine that attempts to simultaneously minimise both false positives (detecting non-stars as stars) and false negatives (not detecting stars), so the above is not intended as criticism. If ASTAP can be used to measure the HFD or FWHM of more stars in the FOV at long focal lengths then that would be a very welcome advance.
I absolutely agree that anomalous detections are of no value. I assume however that the detections that ASTAP makes are generally very sound. Based on the (admittedly relatively few) SGP AF Packs that I currently available I see that at close to best focus ASTAP detects around twice as many stars as SGP, and when the focuser is at a displacement where HFR is circa 2.5 times the HFR at best focus then ASTAP detects circa five times more stars than SGP. I have always understood that in statistics a wider sample base is generally better to a small one.
I’m not a statistician but I accept your point about using the median HFR value to eliminate potential outliers but surely for this to be reliable it is preferable to have a decent number of detections. At the extremities of the AF curve when I’m using RGB filters, I often see only a handful of detections. I feel more comfortable with the idea of taking a median HFR value from a set of 10-15 detections rather than 5.
It occurs to me also that as ASTAP utilises a star database for platesolving it likely has the potential to look up its candidate star detections and eliminate any that seem false or anomalously bright (galaxies, globulars and hot pixels)? If feasible this approach might complement or replace the simple SNR cut you mention to give more reliable detections.
Thanks Ross, my point exactly. As I use a 80/480 f/6 scope and have a FOV of c1.5 x 1.2 deg I imagine your focusing issues are much more taxing than mine. Looking at my AF packs I see that I have a good focus zone of c 20 focuser steps and I am happy that SGP generally puts me reliably somewhere near the centre of this range especially with L filter but somewhat less so with RGB. In addition to the sparsity of stars on the extremities of the AF curve I find that if I run AF several times in succession I do get quite a spread of results which concerns me somewhat. None of this is a disaster to me but makes me think there is room for improvement. Let’s hope Jared and the other devs agree!
P.S. Have you tried examining any of your AF images with ASTAP or equivalent to see how it/they perform?
Thanks Jared for the appreciative response. I’ve referenced ASTAP in this post as I have it installed for platesolving and its CCD Inspector feature helped supported my arguments. So far for me it has proved an excellent piece of software. For a rapid prototype at least it seems to offer the capabilities needed. I assume however that there are Scientific Image Analysis Libraries etc out there that also contain well researched routines that might perform equally well in place of ASTAP?
Just checking that this ASTAP solution would allow filtering out very tiny galaxies and very close double stars when calculating the focus V curve? If yes, this would be a huge addition to SGP. (-:
I’ve created a test version of SGP that uses ASTAP as a metric provided for auto focus and sent it to a couple of imaging buddies. The ASTAP version worked great on my system but a couple of them had issues with the HFD coming back very low on images that should have had very high HFDs thus throwing things off. I’ve sent han those images and some additional information to look into.
If you have some auto focus packs of runs that were problematic I would be interested in checking them out against the ASTAP metric.
Here are all the AF packs I have currently available. None especially problematic but interested to know if a) wings of curve get more values b) much change to the final focus position.
The current setup in ASTAP is such that it can’t measure above around HFD=14 so HFR=7. That is related to the measuring rectangle window used. Beyond that value the measurement will break down more or less completely because the star donut is bigger then the measuring window/rectangle.
Here are some AF packs that have poor autofocus results, plus a few with good results. Four metre focal length and native pixel scale of about 0.193", so around 0.77" per pixel @ Bin x4:
Some of the poor results may be due to windy conditions (the Meade 16" tends to be a bit of a sail), but most will be due to poor seeing and limited stars I suspect. On nights of very good seeing conditions, HFR tends to get down to just below 2 (at bin x4). On bad nights it can be up around 4, and the focus curve becomes quite flat. I can’t increase the tail HFR much more (to obtain a deeper curve on nights with such poor seeing) because sometimes no stars are detected at the tails and AF fails.
By the way, would it be possible to include the RA and Dec in the Fits header of the autofocus images? These images make great trial images for testing/tuning plate solvers like ASTAP, but without hints the blind solves can take a lot of time.
We’re going through a period of poor weather here in the UK so rather than be totally idle I’ve had a think about how one might test whether this candidate change is actually an imrovement.
I’m thinking:
a) If the star detection method can be a switchable option (ie current SGP AF and ASTAP AF) then for a chosen filter (ie L) one might run several AF runs in quick succession so as to minimise impacts of changing temperature and atmospheric conditions. Hopefully one would see a tighter bunching of results with one or other option which would demonstrate greater consistency.
b) As for a) run a sequence of AF runs for a chosen filter but using only a small step size so as not to deviate too far from the anticipated best focus. Hopefully this test would still see a shallow U shaped AF curve with ideally the previously identified best focus point still showing as the lowest point. The better AF method would be the one with the starting AF focus point showing the least deviation from the lowest point of the curve.
c) The ASTAP analysis method seems able for a given AF image to make considerably more star detections than the current SGP method. The value of these extra detections is not yet proven - I accept a point that Han made earlier that the quality of the detections is a least as important as the volume. However I’m currently of the opinion that perhaps 10-15 detections at the extremities of the AF curve would be about right but if sufficient detections can be made with a shorter exposure time then I think this would also be a good thing.
@Ross_Walker , I’ve taken a look at two of the good AF run results you posted and I see that ASTAP generates c5x more detections from each image compared with current SGP.
@Ross_Walker, @Jared , I’ve only had a quick look at your poor AF runs but from what I see I think a change in star detection method alone will not benefit runs made during say gusty conditions. A single poor exposure may well be covered up by the curve fitting routine but probably not much more than this. The answer here I think might be be to have (as an option) SGP to make say three exposures at each AF step and then to stack these exposures before doing the star analysis. ASTAP appears on its UI to have the capability to do this but I suggest the idea needs some further investigation before proceeding further. This option of making more than one exposure per AF point might benefit imagers working in poor conditions/locations and also those with high-end equipment seeking the ultimate from their work.
I’m not familiar with PWI3 but assume that it adopts a similar approah to focusing (ie AF Curve fitting) as SGP. I’m wondering if the method they use is proprietary or described in publicly available material. I’m not sure about copyright but if it is described in publicly available material then maybe the method could be incorporated with SGP if it is demonstrably superior and benefits the wider astroimaging user community. Do you have time to collate some material (AF images etc) that could be analysed and compared PWI3 versus SGP)?
You assert that SGPs focus point is frequently out by c 100 microns. My focuser set-up by comparison does c6000 steps per inch of travel, equivalent to 25400/6000 or approx 4 microns per step, so a 100 micron error would to me be c25 steps. Looking at my AF curves, this level of focus error would be noticable in HFD measurements and I think would certainly be rather annoying if it happened on more than say 5% of occasions.
You suggest that this problem is due to the inclusion of ‘double stars, small galaxies and other non-star objects’. I’m not sure that this ‘feature’ would be so much of a problem when taking a median HFR/HFD value on a larger population of detections as will hopefully be provided by ASTAP. To illustrate my point I noticed a small number of such occurences when looking at the data provided by Ross:
Planetary neb contributes a single HFD value out of 18
In the past I had in ASTAP a stricter algorithm for oval stars and galaxies but it excluded too much if the guiding was not perfect or aberration near the edges. Your PN is round making it even more difficult to exclude on oval shape.
Stars are normally more abundant then galaxies so statistics help.
Sigma clip exclusion could could work but then you have to do a few repeat till the values are stable. So exclude outliers in the first round and do again a mean and standard deviation calculation. But I think this work less well for a small number of values.
For focusing the best approach is taking the median of the collected HFR values for each focuser position. So you first sort the values on value. Then take the middle value. So if you get something like
0.1, 7.0, 7.1, 7. 1, 7.2 , 7.2, 7.2, 7.3, 22.0 the median is 7.2
or if you have even number of values:
0.1, 7.0, 7.1, 7. 1, 7.2 , 7.2 , 7.3, 22.0 then the median is (7. 1+ 7.2)=7.15
With ASTAP it is possible to create artificial star images (tab Pixel math 2) containing Gaussian shaped stars against a noise background. This was great for testing. I have made one with a HFD=3 /HFR 1.5 and loaded it in SGP and ASTAP. The SGP reported values are not so stable even for high flux values and drop out earlier a low intensities. There seems a fixed ratio difference in the reported values of 1.7 / 1.5.
You can see ASTAP struggling at the very low flux intensities but still reporting a little lower values up to the 14th row. SGP is starts to fail at the 7th row at an SNR of about 20.
