The harder category of bugs are when it is related to bad numbers generated from the more complex algorithms in ConnectStats. This is what is now happening with the best rolling plots.
The rolling best curve are one of my favorite feature in ConnectStats, they provide insight I don’t see in many other services. It’s quite common to see a power curve, but I feel the concept extended to heart rate or speed help give people a good sense of the characteristic of a workout. While the concept is simple, it actually can be a bit tricky to implement (at least for me it was). The current version of the app shows quite a few quirks that are obviously wrong, like the below.
These type of issues are tricky to debug because they appear in applying more complex algorithms on large numbers of data points. In this article I’ll review the approach of my algorithm, what some of the problems were. In a following article I’ll talk the techniques I used to debug and then about fixing and extending the historical plots in the Stats Summary page.
The first step in the approach: resampling
In order to compute the best rolling plots, the app goes over each points of the activity and keep track of all the averages reached for all the previous periods. For instance, at the point reached after 3 minutes, the app needs to calculate the average over the last 3:00, the last 2:50, 2:40, 2:30, down to the last 10 seconds. The algorithm uses a 10 seconds window to simplify, otherwise you can quickly see that for an activity of an hour or longer, it would quickly add up to a lot of numbers and calculations.
The first step to the algorithm is to sample the original activity at the 10 seconds interval. It sounds simple but in practice you need to make sure you interpolate between points correctly, and you account properly for pauses or large gaps in the data.
One issue is if you pause the timer. You may end up with a gap in the data. In the past from the data obtained from the garmin website, the app didn’t have the information about whether the timer was paused. So it had to guess based on the size of the gap, which was error prone.
One of the improvement in the latest version was to use the information in the new ConnectStats service. In these files the information on the timer is available, so now the new logic will try to properly account for pauses.
Of course, beside the good reason above, I also found some silly bugs in the accounting of all the indices while doing the interpolations of the re-sampling. This was mostly an issue for the very short interval, as I was allocating the speed to the wrong distance. If you make an error of one index it can make a difference for the short distances. Looking at a specific example say the best time reported in the GPS file after 300 meters was 64.89 seconds. Is that 64.89 over the 300 meters or over the distance up to the next point in the gps file (310 meters)? That result of a difference of speed of 3:29 min/km instead of 3:36 min/km. Definitely noticeable…
The next step: computing the best rolling logic
The core of the logic is to compute all the rolling sum or average of the values for each point in the activities. Then for each rolling sum or average it will take the minimum or the maximum to find the best value. You can see it illustrated below in the case of speed with the latest algorithm. The app will have to calculate rolling20 … rolling 9430, a 943 x 943 diagonal matrix for an activity of 9430 meters or almost 10 kilometres…
The first improvement in the latest version, is that to compute the speed, instead of taking the speed between each gps point, which can be quite noisy, the app now compute the minimum total time for each distance and then converts it to a speed at the end. This seem to be more stable, and has the benefits that the final value for the total distance is by construction the average speed over the activity.
With this improvement, the curve became less noisy, but it still ended up having two issues, too high values for very short periods and non-monotonic periods.
Short periods
Let’s focus on one such issue. In one of my activities, the method found that my best time in that activity for 200 meters was 38.86 seconds or 5.4 meter/seconds, which correspond at a 3:04 min/kilometer, which is very unrealistic for me.
As you can see above, while I compute the best time for a distance, I keep track of the point in the overall activity at which this timing happened. Here the distance of the best point was reached at the 2600 meters in the activity. So let’s go back to the original fit file, using FitFileExplorer a utility I wrote, available on the Mac App Store and included in the ConnectStats open source repo, to help me see what’s going on…
As you can see at around 2.6km the time stamp from the gps is 5:53:24 UTC, and if I look 200 meters later the time stamp is 5:54:02 or 38 seconds later… So the GPS definitely recorded a 200 meters stretch in 38 seconds at an unrealistic speed and thus, my logic was correct…
I would probably have to implement some smoothing or correction for the gps track and eliminate such points, but I am not really sure how for now, so I did some brute force fix for this one… I look at the maximum speed recorder by the watch and I remove any point that is faster than the maximum speed… Simple, dumb but it kinda works for now…
Lack of monotonicity
The next challenge is to understand why the best rolling curve is not always monotonic. You would expect that for a slightly shorter distance the best speed you achieved is always faster than the best speed you achieved on a longer distance and therefore the best speed graph should be decreasing. Unfortunately, if I zoom on some part of the graphs it looks something like this
I have to admit, this is puzzling me. I can see why mechanically from the data on the gps file this happens. Below is an example where the speed increased between 290 meters and 300 meters.
I can see the points in the GPS files that implies this change. But I am just not sure how to fix the algorithm yet. The best time in second is monotonic, but it implies slightly higher speed. I will need to think more on that one. But for now, as above, I do something brute force, which is I force monotonicity after the fact…
You can see the code for the current implementation by looking at the function
-(GCStatsDataSerieWithUnit*)calculatedRollingBestForSpeed:(GCCalculactedCachedTrackInfo *)infoLesser of two evils?
One may ask why I decided to use the approach of looking at the time by distance to compute the speed instead of using the speed field from the recording device. The issue I found with the speed field is that it gives very bad overall statistics. For example in the activity I used for all the above analysis, the total distance was 9.44 km, and it took me 47 minutes and 37 seconds, which implies a pace of 5:03 min/km. Now if you look at the average (properly weighted) of the speed fields on the points you end up with the speed of 5:40 min/km! Interestingly the minimum speed was then correct, so you end up with a curve that starts at the right best speed (4:20 min/km) but ends up at the wrong speed (5:40 min/km) while the average was 5:03!
So I decided for now to keep the computation using the time per distance, and the fixes above, which gives me in this sample activity the following curve, which has more “flattening” accounting for bad monotonicity but has the property of starting and ending at the right place (best pace to average pace)
Maybe I could make it an option in the app to choose the approach…
Next fix the logic for historical and summary view
I’ll push a new version with these changes, because the curves are definitely better now. But I will next have to fix the summary screen. I’ll write more details about it later, once I fix it…
In addition, I ended up using a combination of Xcode and python to debug and get to the bottom of it, I’ll write an article about it as well, as it may help others.