Tag Archives: ConnectStats

Performance Analysis

Posted on by
Reply

In the version 1.20, ConnectStats supports a first version of long term (fitness) versus short term (fatigue) performance analysis. This is a bit rudimentary for now, and hopefully will improve over time.

The performance Index

The analysis is based on two fields, a summable field like distance, time or elevation gain and a second field to rescale it like heart rate, power, etc.

The analysis is based on an index built using this scalable field and summable field.

To access the analysis you need to select from the statistics field view, a field. If the field you select is summable (Distance, Time, Elevation Gain) it will use it as the summable field and choose Heart Rate as the scalable field. If you select a non summable field, it will use that as the scalable field and distance to sum.

Once the two fields are selected it will then apply a formula to get a performance index. The formula in this first version is simply to multiply the two fields, similar to a very simple TRIMP index, but in the future we could change that, for example along the line of normalised power and apply a function scaling more realistically to how the scalable field impact the distance field. This page gives some interesting comparison of the different way to do that.

Fitness (Long Term) versus fatigue (Short Term)

Given the two fields above and the performance index, then we will try to compare the long term accumulated fitness versus the short term training. We pick two periods, the short term period and the long term period, and plot the average performance index of the long term period versus the short term period.

Currently the short term period is the last seven days and the long term period is the month prior to that.

So the idea is to show how much training accumulated over a month (long term fitness) versus how much you are currently training. If your short term training is significantly above the long term fitness, you maybe over doing it. And you maybe taking it too easy or resting if the short term fitness is quite below the long term fitness.

In a future version I could parametrise both the performance index function and the periods used, depending how much people feel the idea is useful or not. So don’t hesitate to give feedback either with a review, tweet, comment or bug report.

Example

Once you selected a field in the statistics view, tap the bottom plot to iterate between the different choices: Monthly value, performance index graph and histogram/distribution of values.

Here is my current running performance. You can see in this graph that recently I have been training a bit more which raised my long term fitness, while the toward the end november I did less running which lowered the long term fitness .

Screen Shot 2016-02-24 at 05.20.48

New Statistics Plots

Posted on by
Reply

In the version 1.20, I added to the main statistics page small preview graphs embedded in the table. I also rationalised somewhat the plots shown on individual fields.

Main Statistics Table

The statistics page start looking like this

EmbeddedPlots

For selected fields, you now see a small preview of a relevant graphs.

Here in distance it shows you the cumulative distance of the previous years, one of my favorite graph to track how you are doing on a given year compare to the previous ones.

Note that you can disable the embedded graphs with an option in settings in case you don’t like it.

For the Average Heart Rate and other non summable fields, it will show you the monthly average over the last 6 months.

Pressing the All button on the right will continue to rotate between the weekly, monthly and annual summary. The Sigma icon means it displays the total or average across all activity. If you press it, it will display the stats restricted to either the last week or last month. This enables you to see all details of the last month or week.

WeeklyStats

Here you can see that the Max Heart Rate over last week was 194, average moving pace 5:21 min/km. This enables you to see any statistics over that period easily. The weekly summary of the previous versions was limited to only a few key measures. Note that in this view the embedded plot becomes a weekly plot to compare this week’s statistics to the previous.

Field Statistics Details

If you press any field of the main statistics table, it will take you a more detail information on that fields, as for example here

StatsMonthly

This shows you two graphs and some basics stats. The first graph is a scatter plot against another variable. If you tap on that plot it will let you configure it and choose the second variable.

The bottom plot will rotate when you tap on it between a monthly summary, the performance analysis graph and an histogram of the different values as here. This post describes the performance analysis in more details.

StatsHistogram

Pressing the all button on the top right as before shows you weekly or monthly statistics.

MonthlyStatsDetails

Improved Statistics Page

Posted on by
2

Version 1.19 includes an improved and better organised statistics page.

The statistics page is quite simple to find on the iPhone. A few users on iPad sometimes miss it as it’s a bit less obvious, you need to press the stats icon pointed by the blue arrow below.

IpadStatsIcon

The New Fields Stats Page

The new stats page now have the fields pre-organised rather than trying to dynamically work out the ordering based on all the fields found on garmin connect data. This is both so that it works better with other services than garmin and due to the fact that the data include now a lot of new fields that confuses the old logic resulting in quite a messy page. Note that any fields not known by the app will still appear systematically at the very end of the list under the category Other.

NewStatsPage

One new feature is that you can now have a quick filter to see the summary stats for either the week or the month of the most recent activity. For that press the Sum Icon, which will then turn into a week or month icon as pointed by the arrow below.

Screen Shot 2014 04 12 at 11 38 58

Weekly or Monthly Summary

As before if you press the All button on the right it rotates through weekly, monthly or yearly summary. There is now also a quick filter for these pages. If you press the All button pointed by the arrow below you’ll enter to filtered page for the last 3m, 6m or year.

WeeklySummaryAll2TimeFilter

Note that if you want to get more detail on an activity you can now tap on the line for the period you are interested in and it will bring up the page with that full stats on the activities. Below you will see all the statistics for the week starting on the 23rd of march.

WeeklySummaryFor3m

It achieves that with the search feature, the activity list will also contains only the relevant activities. You can from the activity clear the search to see the full list of activities again. When the stats are about the current search it then replace the activity type icon. Note that you can this way get stats on any subset of activity you can define with a search in the activity list screen.

WeeklySummaryTapToDetail

Another subtle change in the Garmin API

Posted on by
3

UPDATE: thanks to a tip from @GViewerPro I have now a fix which will be a next release. Thanks a lot.

Since the login change, another less obvious change to the data service of garmin has resulted in a few issues with ConnectStats.

The track data contains now significantly less points. It used to provide points as often as recorded, but now the track data service returns a lot less points.

Here is an example of an activity downloaded before and after the service change. Notice the scatter point contains a lot less points.

BeforeMorePoints

AfterLessPoints

This has a few consequences in the app:

Smoothing

The app smoothing logic is not working as well anymore. We now have a point every 10 to 20 seconds, so it’s harder to figure out how to smooth. Intervals for example may not be plotted properly if they are less than a minute. If that happens turn off the smoothing, by selecting the graph full screen and pressing the Slider Icon to show the smoothing options.

Smoothingoptions

Auto Lap

The logic in auto lap was trying to account for times where the device was on pause. This logic gets mixed up with the new sparse points. A fix and a better logic is on the way for next update.

What I will do about it

I have somewhat tweaked the logic in the smoothing and auto lap feature and this will be incorporated in the new release 1.18. I am also investigating to use the tcx file available. This file seems to still contains all the data point. ConnectStats used to get its data from that file, but it was changed to a different api that contains more information, like the running dynamics of the new ForeRunner 620. I will probably see if I can do a merge of the files, use the tcx file for main data and the sparse data for running dynamics and other fields.

Multiple Profiles in ConnectStats

Posted on by
Reply

ConnectStats allows you to maintain multiple profiles. This can be useful to either track several users or now several services independently.

You access the profile setup from the settings view, the blue arrow 2 below

ServiceAndProfiles

You will then see the list of profiles existing in the app. It will start with only one named Default. Each profile will maintain its own list of activity and service setup.

You create a new profile by using the New Profile line. It will the prompt you for the name of the profile. Please avoid using special characters in the name. You can then simply tap on the profile you want to activate at any time.

Here is the list of profile in my current setup. You can see I created profile for different service setup and other accounts in garmin connect I use, as well as for the activity of friends.

Screenshot 29 03 2014 10 14

If you use the Delete Activities or Profile, this will let you either completely delete the current profile or some activity in the profile. Note that deleting activities here, is only locally in the app, not on the remote service, at the next refresh the activity will be downloaded again. It’s mostly useful if you want to make sure you refresh many activities from the remote services at once.

View or manage cache is simply a way to see how much disk space the app uses across the different files it keeps.

Garmin Connect New Policy Impact on ConnectStats and 3rd party apps

Posted on by
21

First I want to be clear that I have been a huge fan of Garmin’s for years. I own 6 devices and think they are extremely well done. I have also encouraged a lot of friends, colleagues and family to purchase Garmin devices over the years.

Monday March 2nd, Garmin announced they would stop making their API freely accessible and that 3rd party would have to pay $5,000 to access the data in Garmin connect.

I understand their infrastructure is costly to maintain but I feel such a steep fee risk killing the independent 3rd party applications which I believe adds value to Garmin customers and therefore Garmin as well.

I would propose a multi-tier system where 3rd party apps register and get a first limit of daily access for free, say 50,000 access per day. Heavy user breaching this limit would have to pay the 5,000$ fee.

I believe this system would be to both Garmin’s and its users benefit.

The Policy Change

On wednesday February 19th, Garmin updated its website which resulted in the approach used by most apps and website to access garmin connect data via an API to fail. The api used to have a license saying the following


This is the license file for GarminDeveloper.com. You are free
to access our API as long as you agree to create great things.

The link to that license has now been disabled http://connect.garmin.com/proxy/activity-search-service-1.2/downloads.html#License

On monday March 2nd, Garmin has notified developers that they are changing their policy and will charge $5,000 dollar fee for access to the API. Here is an extract of the message about the new policy.

Garmin has instituted a new policy regarding the accessibility of our developer programs. The previous strategy of freely available APIs quickly became unsustainable due to increasingly high demand. In response, we have established a new pathway for our Connect API. Firstly, there is a vetting process. We are strategically limiting the scope of this program to specific developers that will enhance the user experience. Secondly, those who are approved for inclusion will be charged one-time $5,000 administrative fee to cover the extensive engineering and server support required for the Connect program.

Consequence of the decision

This decision will likely force the eco system of independently developed 3rd party app to partially or completely disappear. This is a consequence which will affect customers. A lot of innovation can come from independent developers. Niche functionality can make its way to customers. It is likely not worth Garmin’s engineers’ time to cater to all customers software needs or custom data analysis needs. Enabling third party to provide for them helps make the Garmin devices more attractive. The size of the fee will dissuade a lot of experimentation for small players. The cost of entry is too high.

Most disturbingly it means it prevents people to use their own data in more creative ways than just using the data from the website. ConnectStats started as purely a tool for myself to look and slice my own data. I decided to share it as it could be useful to others, but right now it means I would not be able to use my own data in my code without paying $5,000, even if I pull the app from the store. This sounds wrong to me. Savvy users wanting to use a script to import their data efficiently for processing won’t be able to do that either.

In a world where the fitness market is exploding, I feel it would be to Garmin’s and its customers’ interest not to exclude this source of innovation for functionality.

My choices

I have the following choices for ConnectStats:

  1. kill the app. It would really pain me as part of the initial motivation was to provide a service for myself I couldn’t otherwise get: a more advanced viewer for my data on iPhone and iPad that what Garmin offers.
  2. swallow the cost. It’s a lot of money. I intentionally kept the purchase price at the minimum as to cover development cost and devices (money going back to Garmin by the way…) but my key motivation beyond my own use was sharing the app and not really generating large profits.
  3. increase the sell price of the app and hope people will continue to buy it. It somehow feels wrong to charge the users more just for the sake of them accessing the data they should own already. Right now if I increase the price from $.99 to $2.99 for ConnectStats, I am hoping to recover the Garmin fee over time. People pay several hundred dollars for their device. I hope they will accept the few extra dollars to pay Garmin for the right to access their data.
  4. switch ConnectStats to rely on another service. I am already working on that to use strava as primary download service and hope strava will pay the fee to get the data from garmin. This has some downside as some data won’t be available and Strava does not support swimming and skiing which I personally rely on.

Comparison to others services

I also wanted to comment on the fee structure proposed. A fee of $5000 is a lot of money by any standard. Niche independent Apps do not generate much revenue, mostly cover the costs. Here are some other services I use as comparison.

  • Apple charges $99 a year plus 30% of every sale to access their development tools and app distribution infrastructure. And this infrastructure and marketing power is of extremely high quality and adds a lot of value to independent developers like myself and to its customer. It wouldn’t occur to me to complain about this cost.
  • Strava provide an access to its data for free up to a certain rate limit once you are registered. Their API is extremely well documented, modern and their support and help to my questions have been great. I applaud Strava would provide a sophisticated website and API for free when their business model does not include revenue from any hardware.
  • Garmin support and documentation was so far inexistent. There are no or little documentation. I have never received an answer to any of my inquiries. Everything had to be reverse engineered. It doesn’t bother me at all given it was free but for $5,000 it’s a different issue.

I have been inquiring with Garmin about what exactly is provided when I pay the fee. What exactly do I get for that money: same API? Do I need to adapt my code to a new API? Do I get more documentation? Better support? Some marketing/promotion of the app as Apple provides?

Proposal

I believe the independent and small 3rd party partners add a lot of value to Garmin and the ability for users to access freely their own data is important. I understand the potential infrastructure cost and need to control usage for Garmin.

I suggest they offer a tiered system. Any user of the API would have to register and get some basic number of API access per day, say 50,000. Users above that limit would be required to pay the $5,000 fee to use the service. This would allow them to recoup some infrastructure cost from heavy user but still allow their customers to benefit from 3rd party functionality and access to their data.

What I plan to do for ConnectStats

I am doing 2 things:

  1. I have written to Garmin to explain why I feel a tier system would be better for all parties involved. I hope they will agree.
  2. ConnectStats is somewhat successful, receives good reviews and regular download. I hope by raising the price by an additional $2 this should help sponsor and recover the fee from Garmin. I have applied to be part of the program. Assuming Garmin approves my application, I plan to pay the fee myself and hope people will continue buying the app at the higher price to recover the money over time

Thanks for reading.

Comparing activities in ConnectStats

Posted on by
1

My activities are often on the same paths, so I wanted a way to compare activities. Of course segment in Strava are really useful in finding out how fast you can run or bike certain route, but I wanted a bit more control in what I was comparing. I also wanted a way that is not over complicated within the UI flow of connectstats. Here is what I came up with.

To compare an activity you need to first mark the activity that will compared to any other activity you will then look at. For that, you slide an activity in the list to the left, which reveals the Mark button.

Markactivity 6

After you press the button and the activity is then marked, note the mark icon on top of the activity icon.

Markedactivity 4

You can then look at any other activity and the graphs of the marked activity will then appear in the background of the current activity. Here I am looking at activities where I skied up the same slope with skins. It’s clear the first one my heart rate was much higher than the compared one. It was pretty much the same path up, so really just show how I adapted to the altitude over the course of the week, I think… Look at the HR histogram

SkiupHRHistogram 3

Or the rolling best plot

Skiupbestrolling 3

Now of course I claim I took the same path up, or did I? Hit the map and the two paths will appear. The compared path is slightly less opaque

Comparepaths 2

Here you see though large part of the ski up was the same, I didn’t start from the same exact same place, but arrived at the same destination.

Data relationships in a single activity

Posted on by
Reply

Here we’ll explore different ways to look at two different attribute together in connectstats.

Let’s say we want to look at how the speed and heart rate relate to each other. The first approach is simply to plot the two series together on the same graph as below.

2fieldsplots 12

The arrows 1 and 2 shows that on the two short interval when the running speed increased, the heart rate increased as well.

One approach to relate the information a bit better is to use a color overlay to show the one information on the a single plot

2fieldsasline

The arrow 1, shows where the line becomes darker and red as the heart rate increased with the speed, the arrow 2 shows the heart rate slowing down as the color goes back to the green

The last and, in my opinion, most interesting way to relate the two series of data is the scatter plot.

Scatterplot 17

Here you can see the points (arrow 1) where the speed/heart rate increased together. Array 2 is the beginning of the run with lower heart rate until it stabilised in the region of arrow 3 for most of the run. The regression line shows how the main relationship between the 2 data goes, here slightly to the downward slopping as expected.

It is also interesting in the lap view to see where in the larger scatter plot a given lap appear as below

Scatterlap 9

The green points show for this mile lap that the beginning was quite fast/high heart rate (arrow 1) and then slow down of both speed and heart rate (arrow 2), and clearly at the lower end of the overall speed of the run.

Data and smoothing used in connectstats

Posted on by
2

ConnectStats let us apply basics statistics technics to analyse our fitness data. We use these techniques to answer questions about the activities. How well did I do on this run? Is my stamina improving? Did I push myself more today that last week?

Raw Data

Before doing any analyses, let’s review the raw data we have access to. ConnectStats relies on the collection of the data is done with a garmin device and a collection of sensors.

The raw data varies by activity type and device.
For outdoor activities, the raw data will be gps coordinates, a timestamp, speed and values from different sensors: Heart Rate beat per minutes, cadence (legs or pedal), Power delivered on the bike, and sometimes more advance running dynamics like Vertical Oscillation, ground contact time. For swimming the data collected will be the time for each length and the number of stroke.

Here is an example of what the raw data looks like Blog txt

Depending on the activity or the device the raw data will be collected at fixed intervals or at different intervals trying to reduce the amount of data saved. If you move in a straight line for 5s at constant speed, it isn’t necessary to collect the points every seconds as they can easily be interpolated.

From this raw data, there are two main directions for the analysis we can do. The first is analysing a single activities and the second is analysing summary of each activities over time.

Focus on a single activity

The summary statistics are quite simple. Mostly we will want to look at the average of the different measure, maximum and minimum. Where it gets more interesting is to look at the series of data. As for most statistics analysis a first phase will be cleaning the data.

Data Cleaning

Let’s look at the graph of the data collected by the gps. You can see it is quite noisy, there is also large part of the data that is not really interesting to look at to. Some is gps noise, area with bad reception, pause due to stop lights,

Rawdatabad

To clean the data, we can first use assumption on expected constraints on the data. For example, define a minimum speed considered valid running speed, or a minimum valid heart rate to remove unreasonable spikes. Another technique used by connectstats is to filter points that have unrealistics acceleration.

FilterBadValues

The last smoothing technic we can apply is a moving average of a few points. Usually the app will use 5 points moving average for data, but it can be controlled in the sliding left settings window.

Filterbadnormalsmoothing

In the case below the graph start to be more readable now, especially with the 2 minutes moving average overlay, which shows the two short faster run intervals around 30min and 40min.

Historical Scatter Plots I like to use

Posted on by
Reply

One of my initial motivation to write the app was to look at my activities using scatter plots. I was especially interested in looking at the relationship of heart rate and speed.

Here you can find information how to access them.

scatter-hr-pace

The first thing I look is where is my last activity in my overall history. Here you can see that I was in the middle of the pack, a bit on the high heart rate side.

Screen_Shot_2014-01-18_at_17_44_54-6

 

It’s also interesting to check the pattern overtime. You can see here that the more recent colors are on the higher HR, slower pace area. Not good, I need to improve.

Screenshot_19_01_2014_22_25

Sometimes it can also be useful to check only the recent history, using this button to rotate between all, 1m, 3m, 1y

Screenshot_19_01_2014_22_31-4

Other times, I also want to have a more sophisticated filter for the graph, in that case I can use the search feature. If I define a search in the activity list I can then get the scatter plot only for those activities. The statistic tab will have an extra button beside running, cycling swimming and all: Search.

Screenshot_19_01_2014_22_34-2 Screenshot_19_01_2014_22_35-2 Screenshot_19_01_2014_22_36

Some other interesting historical relationship to look at for bikers: Power and Cadence or Power and Heart Rate

Screen Shot 2014-01-18 at 22.57.28 Screen Shot 2014-01-18 at 22.57.46