SGP AutoFocus LogViewer

Here is an improved result for @bvalente. I adjusted the algorithm so most runs end up using the curve that includes all focus data points. Only curves with a worse than usual data point will exclude that data point.

I am very interested in whether or not your statement that R^2 is not a good quality measure for our focus curves is true or not. Some of the arguments this paper presents are not relevant for us.

What I have seen so far is that decent usable focus curves are producing R^2 values greater than 0.97, whereas the bad ones usually have a value less than 0.8. It seems to work really well as a quality measure to be used as the trigger for whether or not to do a re-run. I have examined about 50 focus curves from two different users.

Why don’t you send me some log files with some examples of what you are talking about. If it would improve my routine I am all for adding it.

It would be a big help if you could send me specific examples of these two conditions.

The point isn’t so much that R^2 is misleading - the main thing is that it doesn’t tell you what you really need to know - which is how confidently the focus is known - in terms of steps. You might use very large steps and get a very good v curve that matches a parabola well - but the expected error in the predicted focus is very large.

What I’m talking about isn’t earth shaking and it is standard procedure when fitting data to determine an underlying parameter. The fit not only gives you an answer, but it also gives uncertainties in the parameters. You can then use those uncertainties to calculate the uncertainty in what you are measuring.

If I say one focus value is 100 with R^2 of 0.9 and another is 120 with R^2 of 0.8 - what do I conclude? Compare that with one value giving 100 +/- 50 and another giving 120 +/- 5. The +/- comes directly out of the fit and carries much more info for comparing different curves and parameters.

You can do the calculation with your own data - I describe how to on the page I reference.

Frank

I just read through your paper on how to include error bars. This would clearly make the whole routine much more complicated. Let’s assume this were implemented. Is this going to change the recommended focus position, or just give us a new measure of the expected position error compared to the ideal?

If you would like to plug this into my routine, I will be happy to send it to you. It is coded in C++ under VS2019. I would have to see some proof that this would actually be useful, to encourage me to undertake it myself. The simplest proof would be for you to send me some focus runs where you have applied this, and I will do my simple Quadratic routine and we will be able to see what differences there are.

I don’t think the implementation is very involved - and it just needs matrix multiplication and inversion - which are directly available in python.

The parabolic fit is just a means to an end - and the goal is to find the focus value as calculated from the coefficients of the parabola. The R^2 gives you an idea of how well the model explains the data - but it doesn’t directly tell you how uncertain the focus value result is.

I’m confident that if only the R^2 were available, people would be comparing R^2 values and trying to optimize it. But it doesn’t tell you how good the actual focus result is - and it doesn’t tell you if a change of focus of X steps is significant or not. I would think that is sufficient proof right there of the benefits of the approach.

I’m mainly addressing people who are actually implementing autofocus routines - in particular if SGP were to do it. If you don’t think it’s worth it for your viewer that’s ok - but it isn’t a lot of math - and error bars are directly interpretable whereas R^2 is not.

Frank

@freestar8n I think there is a misunderstanding here. I will not include any quadratic fitting directly into the Logviewer at this point. The goal of the viewer is to show what SGP does during an automated session in terms of autofocus, and not to advocate changes in SGP. However, I support the efforts of @bulrichl and @jmacon to provide evidence that a quadratic fitting can improve SGP, by adapting my program so that they can more efficiently analyse large sets of logfiles using their home made AF routine. Basically my application would save lists of extracted focus curves that can be read by their application. For now that happens outside the scope of the “official” release of the Logviewer, but will probably be included in next release.

I think this thread has maybe gone a bit astray with respect to the Logviewer, and if you don’t mind I suggest that we move this discussion into the already existing thread on quadratic fitting started by @bulrichl, here: Proposal of a "Quadratic Fit" Auto Focus evaluation method - #18 by bulrichl - Auto Focus - Main Sequence Software

Thanks, Mikael

1 Like

Thanks again for this software! I have used it a lot lately to help people diagnose autofocus and calculate temperature compensation :+1:
I wonder it would be possible to add an option to differentitate different filters when calculating linear regression?

Thanks Xplode! Good to hear it is useful.

This is planned for v1.2 yes, along with the ability to load multiple log files to get more data for the regression, if I can get to work without re-coding the thing completely. The timeline is probably 1-2 months for v1.2, I guess.

I am also about to upload a slightly improved v1.11 this week. It has one new feature for the Pos vs Temp graph, where one can now click on a point to remove it from the linear regression. This allows to exclude outliners. Could also be used too remove a particular entry with another filter, if you can identify that point in the graph. I will post a message here when I upload that v1.11.

Mikael

so badass. Thanks again for a great utility :slight_smile:
Brian

Just being able to exlude points will definitely help a lot :+1:

Here is version 1.11 of the AF Logviewer: SGP AutoFocus LogViewer download | SourceForge.net

Main changes include the ability to exlude/include points in the Position vs Temperature graph, “Target” shown in Summary output, and greatly reduced logfile loading time.

Mikael

Detailed changelog:
Version 1.11

  • Fix: several minor bugs were fixed
  • Fix: consecutive focus runs manually triggered by the “Rerun” button would be grouped into one big AF Run. These are now displayed as separate, individual AF Runs.
  • Fix: loading times could be many seconds for large log files. Now the loading time is FAST.
  • Change: the current target is now tracked through a sequence, and a “Target” column has been added to the Summary output. If the AF is triggered manually without running a sequence the entry will be empty.
  • Change: data points can now be excluded by simply clicking on data points in the “Position vs Temperature” graph. The linear regression curve will be re-calculated and re-displayed in the form. This allows for excluding outliners. To reinclude a data point one can click on it again.
  • Change: added a column showing potential warning and rerun messages. Possible messages are:
    Warning = HFR validation frame warning
    Rerun = manually triggered rerun
1 Like

I’m trying the quadratic fit focus routine tonight, i see some changed has to be done so the logviewer will handle the logfiles correctly

Here’s an early logfile from a couple autofocus runs
sg_logfile_20191012111055.txt (710.8 KB)

Before anyone goes crazy here… We have a debug log for AF that is turned off for the release. It is all JSON and would be way less fragile than bit-bangin around the main log file. I’ll post an example shortly. It is certainly not difficult to leave it on for the release.

Know what’d be even awesomer is if you could expand your program to figure out all the offsets for each filter and allow a selection to plop that data into the profile of our choice. Also I am sure you could calculate the temperature curve needed for temp compensation. Seems to me the hard work is already done.

Looks like an amazing app. Going to download it now.

@Xplode Please download and install v1.11 again. The bug should now be corrected.

@Ken That sounds great! While this was a minor thing that I could correct quickly, having access to your debug JSON data would make it more robust for sure.

@mikaelA

Here is an AF debug log. It will be on in the next beta (and forward). Let me know if it would be helpful to see other data in there (or if there is bogus data in there).

1 Like

Thanks for sharing this Ken!

That looks great for extracting relevant AF data, much more robust! I have to think this through a bit, particularly how to make sure the my program is backward compatible. I am thinking that it should be able to open older standard logs, and then these debug ones starting from SGP v3.1 when it is released. It is simple I guess to detect that it is opening a standard logfile, as there is an identifier in the beginning telling the SGP version. To see that it is a debug file is less evident to me. Could you add a header line preceding the JSON data indicating an AF debug file and the SGP version generating it? Or if you have a better suggestion maybe?

As for the JSON data itself, could you please add the number of detected stars for each exposure in the “Datapoints” data? Also, it would be nice to know if a particular AF run is part of a running sequence. Maybe add a FocusTarget entry with a string of the target name?

Realistically I will have take some time to digest this and figure out how to best switch to this new format, but for future compatability this is surely the way to go!

Thanks, Mikael

@mikaelA

I think this would work for you:

var isAfLogFile = File.ReadLines("theChosenFile.txt").First().StartsWith("{");

Yes

Easy enough

same

Some of the latest beta releases crashed the Logviewer. We have now tested the beta up to beta 307. So if you experience problems please re-download v1.11 from Sourceforge.

Mikael

Could you add the ability to include focus runs that weren’t complete/verified to be good in SGP AutoFocus Viewer?
Being able to add runs that weren’t good would make it easier to help diagnose focus runs from people not getting consistent good results.

1 Like