overhaul ETA calculation

[ipv6guru]

The Progress and Expectation classes are overdue for a rewrite. They need to move to a scheme that I'll call Reverse Temporal Mapping because it sounds clever.

The idea is that each StepProgress? metric (an axis along which progress can be measured, like bytes of output or test cases run) should be sampled periodically during the Step. Nominally we'd like about 100 samples total, but if we have no idea how long the step will run then we can just sample every 10 seconds or something.

Those samples form a graph of progress-versus-time. We then invert the graph to give us one of time-versus-progress. This is used to create an interpolation table that will map from progress values into elapsed seconds. During the next build, we'll take the current progress value and use this table to estimate how many seconds are left to go.

The tables should be averaged over multiple builds, to provide some smoothing. The ETA values from the various metrics should be combined with some sort of weighted average (some metrics may be more accurate than others).

We also need to handle overdue builds better. It may be useful to report both the ETA and the percentage complete (since a slow build that still produces the same output as before will show <%100 complete even when the build is overdue).

It would also be useful to add some intstrumentation that lets us check the accurary of the ETA code, perhaps a table to show how the ETA changes as the build runs, comparing it to the actual elapsed time. The ETA should get more accurate as the build nears completion: it would be nice to see a graph of this convergence. Submitted: Brian Warner ( warner ) - 2005-05-18 05:19

[marcusl]

I did this for a loading-bar in an application once, simply tracking where in the script it was at the moment and recording the time. It works pretty well unless the output/progress you're tracking is changing alot.

[dustin]

We should think about how this works in the statusdb world..