What the Alternative API mean for ConnectStats

Posted on March 29, 2018 by Brice

Most users experienced new activities from Garmin not syncing up with ConnectStats since last week (March 23rd or so). As described in the last post, it can be fixed by using an alternative API.

It looks like it’s going to be a permanent change. It also has some minor consequences for users.

What is the alternative API?

It’s probably obvious to most of you, but the API is the mechanism that third party (ConnectStats) can use to access the data stored by a provider (Garmin or Strava).

Garmin a long time ago had a public API to access the activities recorded in Garmin Connect. Back in 2014, Garmin decided to change it’s approach to third party. While the old API allowed a third party to access directly the running or cycling data from Garmin servers, they decided to switch to a push approach. The key consequence is that a third party now has to build and maintain an infrastructure to store the data in the cloud after it was pushed by Garmin. This is great for a platform like Strava, it makes for a great user experience. But it’s impractical for small independent hobbyist developers like myself with ConnectStats like app.

While Garmin announced they would deprecate the old API in 2014, in practice until last week it continued to work more or less. It just had regular tiny changes, which beside the fact that the change came as a surprise outage to developer via lots of sudden bug reports, they were usually easy to keep up with.

The old API continued to work and was used by their own Garmin Connect website. since they upgraded their website to the new look shortly after the API announcement, they also slowly started to upgrade the site to a very different API. This is what the settings “Alternative API” refers to.

Practical Difference between the two API

ConnectStats downloads data in two stages. The first is to get a summary of all recent activities, which is intended to be faster. Then when you go and look at an activity in details, it will downloads all the individually recorded data point to create the graphs and the track on the map. The details can be a lot of data, so it’s an optimisation to only download the summary first.

The main difference between the two APIs, is that the new one (alternative) contains a lot less information in the summary. Using the new API, a lot of the extra information (for example: running dynamics, min and max, etc) will only be downloaded while getting the full detail of the activity. It’s not a major issue, as in practice when you select an activity, you need to wait a bit for the graphs and map data anyway.

Where it can impact the user experience is in some of the historical statistics pages. Some historical statistics on the extra information will be missing until each activity detail is downloaded. To mitigate the issue, ConnectStats tries to automatically download the last dozen or so activities that are missing detail.

Will the old API come back?

Only Garmin knows. But at this point I would anticipate it won’t. ConnectStats lived on borrowed time since 2014 when they announced they would deprecate the API.

Also the behaviour of the old API is not that it errors, but it contains a lot less data (almost none). In the past changes, it always was either some error, or some subtle change that broke ConnectStats use of the API. This time feels different.

What’s next?

I am releasing a new version of ConnectStats (4.3) that will by default use the new API. I keep the setting to switch back, in case things change in the short term and to keep optionality.

As in all the previous discussion on API change, I also continue to prepare for a day when Garmin may completely shutdown the API ConnectStats uses.

At such time the only option to continue using ConnectStats will be to switch to another service provider like Strava.

Using Strava data though has the drawback that it contains a lot less details on each activities (no running dynamics, etc).

Alternatively, now the the app is open source, if it comes to that we may link it to other service and hopefully the app will continue to live.

2016 year review and double counted activities

Posted on December 19, 2016 by Brice

It’s time again to look at the past year summary. How did I do?

2016 was a fine year, I ran 1387 km, it is 100km behind 2015. But one aspect bothered me, looking at the distance, there is a clear increase on the graph in December 2015, which put me behind in 2016. It worried me a bit. I couldn’t remember an especially good December last year. Did I have another bug in ConnectStats?

I went to check on Garmin connect and it reported the same distance as connectstats: 1478km for 2015…

Hum. Next step was to go and look at the activities in December last year. The search feature in ConnectStats made it easy, just enter december 2015 in the search box of the activity list. Sure enough, I found some duplicate activities. This started to be very worrying. ConnectStats imported the same activity twice? New bug? After some more investigation, it turned out the problem was in Garmin itself… Quite a few activities in 2015 appear twice on the website as well…

They have a different activity identification number, but they are clearly the same activity except for the altitude gain as you can see on the snapshot above.

Next steps was to add a new feature to ConnectStats to double check for such duplicate activities and ignore them… Well, my stats are now lower only 1310km but correct. And 2016 is ahead of 2015 by 50km! Yeah!

I finally decided to check what Strava reported. My account is linked to import activities automatically. I was curious: would strava have the total including the duplicate as well? It actually didn’t… The total is 5km off from my new corrected total, but I suspect it is simply due to the slightly different way Strava computes the total distance from the gps file.

The fix to search and eliminate duplicate will be included in a new release early next year.

The Apps and Services I use

Posted on May 3, 2015 by Brice

ConnectStats was really built out of a need I had and over the years, while the landscape of services and apps has changed quite a bit, I continue to use it and develop it. So here is my view on the services and apps I use beside ConnectStats and why I feel ConnectStats is still useful.

Of course a big disclaimer is that my views are biased given I wrote ConnectStats. But at the same time, my goal is not to replace or take out any competing app or service, but more to complement with what I feel is missing, so hopefully, it remains an interesting perspective.

There are mainly 3 apps or services I use: Garmin Connect Mobile on the phone, Strava on the phone and on the web and ConnectStats. I focus here on the iOS app, not the web sites.

All my activity data collection is done with Garmin Devices. I use a Garmin Edge 510 for my cycling, and currently a Fenix 3 for running, and other occasional activities like hiking, skiing, etc. I had a Garmin Forerunner 620 previously and before that a Garmin Forerunner 610.

I will go through the following:

Syncing Activities
Casual Review of Activities
In-depth analysis of an activity
Statistics over time
Social Features
A few other apps and services to note
Pricing

Syncing Activities

From the device

Once an activity is recorded, I transfer it to the cloud either using the Garmin iOS app, the device wifi ability (fenix or forerunner 620) or the computer and Garmin Express.

I really wish the Garmin iOS App and bluetooth connection was more robust, it would be my preferred transfer method, but unfortunately it’s quite flaky. Great when it work, but regularly it just can’t connect, turning on/off bluetooth is usually not helping. On the day it works it’s great, on the day it doesn’t quite frustrating, and if I am at home I have to switch to garmin express on the mac and connect the device via cable.

There are no alternative to the Garmin tool for syncing, as they don’t publish any public API for that, so we have to stick to their tools.

Between Services

The situation has much improved now. If you set up connection between Garmin, Strava, TrainingPeak, SportTracks, all the activity get sync seamlessly. I built in the past a feature to upload from ConnectStats to Strava, but this is now really not necessary and I stop using the feature or developing it further, I recommend just using the direct link. I had a few users argue that ConnectStats feature was still useful because it allowed you to upload the activity after you did some edit like activity type or activity name, but I feel it’s a niche benefit.

ConnectStats caches locally all the data so once downloaded you can review at will all the activities and statistics without an internet connection. From my point of view it also makes the app quicker to use and navigate as everything is calculated locally. Of course a side effect is that it’s harder to do large data analysis like Strava does with segments or comparing runs automatically.

Casual Review of your activities

Garmin Connect Mobile

Most of the apps will let you see the basic data of an activity. For my taste, the Garmin iOS app probably fares the worst both in terms of look and data. You really only have basic data here as well as simple laps statistics. Also the look is extremely basic and not like a modern iOS app. The graphs are quite slow to come up as it seems to do a internet call. This is the case for all data, so if you are without internet connection, you won’t be able to review any activities, even one that you looked at in the past in the app.

Strava

Strava is the app that has the best look by far. I prefer it to how ConnectStats looks, Strava is much slicker. I only wish I had the time and the artistic skills to make ConnectStats look as good. It has most of the basic information handy and I particularly like the comparison to previous similar run. I like the Heart Rate Zone Analysis and pace analysis, though you need to subscribe to their premium service.

ConnectStats

ConnectStats also has a fairly basic look, I have little time and skills to make it look pretty, but I focused on having the most possible useful information accessible on one screen. One of the benefit of the Garmin Connect data versus Strava is that it contains a lot more information, like all the running dynamics, weather, maximum, calories, etc, etc. I wanted to have all that information displayed and an overall map and graph visible at once in one screen that you can scroll. One side effect is that to focus on any information makes it harder to design the screen to look good. The strava screens look much better but their are more specialised for pace, heart rate etc, therefore less generic information.

Activity Plots

This is something that always bothered me with most apps and services. The plots of pace especially, but sometimes heart rate as well seem to be scaled and smoothed so poorly that they didn’t allow you to see really what happened. This was one of the key initial motivation for writing ConnectStats (along with more information on the map). I spend a lot of time working on the scaling, filtering and smoothing of the graphs in connectstats so that it gives you a good picture of what happened at a glance. Recently I also added overlay of the laps average which I feel is quite helpful.

Here are the pace plots for the same activity in Strava, Garmin Connect and ConnectStats. I doubt anyone would be able to draw the conclusion I sped up a bit in the middle of this run, only to clearly slowdown toward the end, from either Strava or Garmin Connect’s graph… Note that the lap histogram on strava shows you that story as well, so one point for strava, but the plot is quite useless.

In-depth analysis of an activity

This was really one of the driver behind building ConnectStats so this remains the app I favour by far for this. Of course, likely biased by the fact I put there all the analysis I wanted, that others didn’t provide.

Plots

Garmin Connect Mobile will simply show you the basics plots and they are not very pretty and slow as they get pulled from the internet. Strava, if you are a premium customer, will show you in addition time in zone and pace zone analysis in pretty looking fashion.

ConnectStats shows a lot of extra plots. Time in zone is not as nice looking as the equivalent in strava. But there all the relevant fields can be seen on a plot, including run dynamics. And two types of plots just don’t exist in the two other apps and quite useful to me.

Scatter plot to see how two variables relate together, how steep is the regression between heart rate and pace, or cadence and power. You can also compare the graph of two activities.

Best rolling plots are also unique to ConnectStats and somehow addictive for me to look at.

Maps

The maps in Garmin Connect Mobile are quite basic. I do not like the very thick lines to show the route personally. Strava’s map look better, but also simple in feature.

ConnectStats shows a lot more flexibility in what it shows on the map. It use a color coding to show values of different variables like heart rate, pace, etc at each point on the map.

ConnectStats can also display laps as recorded or calculated ones like fastest kilometre (or mile), breakdown in kilometres, etc.

Finally ConnectStats lets you use either apple maps or google maps included terrain’s view, which can be handy when off road.

Auto Laps and Segments

Auto laps is a feature that also make me use ConnectStats over the other two. This features lets you calculate arbitrary laps after the fact. You can calculate the split time this way or the split distance. But also the lap corresponding to each heart rate zone, laps breaking the activity in fixed distances or best rolling laps (where did you run your faster 50m, 100m, 200m, etc, etc). For each of these calculated laps, you can then see it on the maps or see the relevant statistics as numbers or plots (like a recorded lap)

For segments Strava is the best app. ConnectStats doesn’t support them at all and Garmin Connect Mobile has them, but they are nowhere as functional as Strava’s.

Garmin Connect has a separate page showing you were you rank and very basic stats like time and speed for your run.

Strava’s segment view has both ability to compare yourself to other, but also analyse your performance in details, overtime and for a given effort. It shows a plot overtime of the speed you reached on a given segment along with the list of days you ran it. You can also analyse the effort on that segment and compare it to your PR using nice looking plots.

Statistics over time

Garmin Connect Mobile here has no such feature. Or if it has, they are well hidden and I could not find as much as a current week or month accumulated distance.

Strava and ConnectStats both go much further but in a different direction. ConnectStats focuses on statistics and more advanced analysis of the data, while Strava focuses on more basic data but presenting it better, and matching equivalent data over time.

Strava will present you the week and daily distance in great looking graphs and elegantly shows your progress toward a weekly goal. You can also see a few basics statistics over the year, like Avg Runs/Week, total year to date distance, etc. For a given activity, I really like the matched run feature which at a glance lets you see the progress when you run an activity over and over.

ConnectStats will show you many more graph, but the data is presented to you as is. It’s not put into context like strava and compared to a goal or against matched run or segments. That said the graphs and data you can extra from connectstats is quite wide. Because it’s less about context, ConnectStats lets you explore any data in the same fashion. All recorded data can be analysed. How did your Max HR evolve over time? your Avg Cadence? Any of the run dynamics? How does speed get affected by avg temperature? Or your weight?

ConnectStats also let you plot many type of graphs. You can see scatter plots of any measure vs another, weekly, monthly summary of basics stats but also quartiles, medians, etc.

There are a few unique plots which I constantly refer to as well. I feel the yearly comparison of the total distance ran or biked quite motivating to go beyond the previous year. The long term vs short term analysis is also a good indicator of your current form vs effort.

ConnectStats also has a calendar view that lets you have a good overview with color coding or basic statistics of how you did in a given month.

Social Features

ConnectStats has no social features, and this was the key reason I initially used Strava that has in my opinion the best social features.

The feed is neatly shows your and your friend’s activities. It’s extremely easy to add a quick thumb up or comment. And it also let you look in detail at your friends activity and of course compare yourself to them on given segment. Also automatically detects when you ran with a friend.

Garmin Connect Mobile has a news feed also, but it does not look as nice and unlike Strava where things are intuitive and simple (one feed and a filter), Garmin Connect Mobile has a news feed, and an activities within a fitness menu, a social menu. All very convoluted for a less simple and pleasant experience as Strava, not to mention the overall look of the app.

Conclusion

Strava and ConnectStats are my two favourite apps to analyse my data. Strava is much better looking and has great social and segment matching features, but ConnectStats continues to enable me to look with much more depths into the numbers. So for now I’ll continue to use and develop ConnectStats…

Another subtle change in the Garmin API

Posted on March 30, 2014 by Brice

UPDATE: thanks to a tip from @GViewerPro I have now a fix which will be a next release. Thanks a lot.

Since the login change, another less obvious change to the data service of garmin has resulted in a few issues with ConnectStats.

The track data contains now significantly less points. It used to provide points as often as recorded, but now the track data service returns a lot less points.

Here is an example of an activity downloaded before and after the service change. Notice the scatter point contains a lot less points.

BeforeMorePoints

AfterLessPoints

This has a few consequences in the app:

Smoothing

The app smoothing logic is not working as well anymore. We now have a point every 10 to 20 seconds, so it’s harder to figure out how to smooth. Intervals for example may not be plotted properly if they are less than a minute. If that happens turn off the smoothing, by selecting the graph full screen and pressing the Slider Icon to show the smoothing options.

Smoothingoptions

Auto Lap

The logic in auto lap was trying to account for times where the device was on pause. This logic gets mixed up with the new sparse points. A fix and a better logic is on the way for next update.

What I will do about it

I have somewhat tweaked the logic in the smoothing and auto lap feature and this will be incorporated in the new release 1.18. I am also investigating to use the tcx file available. This file seems to still contains all the data point. ConnectStats used to get its data from that file, but it was changed to a different api that contains more information, like the running dynamics of the new ForeRunner 620. I will probably see if I can do a merge of the files, use the tcx file for main data and the sparse data for running dynamics and other fields.

Garmin Connect New Policy Impact on ConnectStats and 3rd party apps

Posted on March 5, 2014 by Brice

First I want to be clear that I have been a huge fan of Garmin’s for years. I own 6 devices and think they are extremely well done. I have also encouraged a lot of friends, colleagues and family to purchase Garmin devices over the years.

Monday March 2nd, Garmin announced they would stop making their API freely accessible and that 3rd party would have to pay $5,000 to access the data in Garmin connect.

I understand their infrastructure is costly to maintain but I feel such a steep fee risk killing the independent 3rd party applications which I believe adds value to Garmin customers and therefore Garmin as well.

I would propose a multi-tier system where 3rd party apps register and get a first limit of daily access for free, say 50,000 access per day. Heavy user breaching this limit would have to pay the 5,000$ fee.

I believe this system would be to both Garmin’s and its users benefit.

The Policy Change

On wednesday February 19th, Garmin updated its website which resulted in the approach used by most apps and website to access garmin connect data via an API to fail. The api used to have a license saying the following

This is the license file for GarminDeveloper.com. You are free to access our API as long as you agree to create great things.

The link to that license has now been disabled http://connect.garmin.com/proxy/activity-search-service-1.2/downloads.html#License

On monday March 2nd, Garmin has notified developers that they are changing their policy and will charge $5,000 dollar fee for access to the API. Here is an extract of the message about the new policy.

Garmin has instituted a new policy regarding the accessibility of our developer programs. The previous strategy of freely available APIs quickly became unsustainable due to increasingly high demand. In response, we have established a new pathway for our Connect API. Firstly, there is a vetting process. We are strategically limiting the scope of this program to specific developers that will enhance the user experience. Secondly, those who are approved for inclusion will be charged one-time $5,000 administrative fee to cover the extensive engineering and server support required for the Connect program.

Consequence of the decision

This decision will likely force the eco system of independently developed 3rd party app to partially or completely disappear. This is a consequence which will affect customers. A lot of innovation can come from independent developers. Niche functionality can make its way to customers. It is likely not worth Garmin’s engineers’ time to cater to all customers software needs or custom data analysis needs. Enabling third party to provide for them helps make the Garmin devices more attractive. The size of the fee will dissuade a lot of experimentation for small players. The cost of entry is too high.

Most disturbingly it means it prevents people to use their own data in more creative ways than just using the data from the website. ConnectStats started as purely a tool for myself to look and slice my own data. I decided to share it as it could be useful to others, but right now it means I would not be able to use my own data in my code without paying $5,000, even if I pull the app from the store. This sounds wrong to me. Savvy users wanting to use a script to import their data efficiently for processing won’t be able to do that either.

In a world where the fitness market is exploding, I feel it would be to Garmin’s and its customers’ interest not to exclude this source of innovation for functionality.

My choices

I have the following choices for ConnectStats:

kill the app. It would really pain me as part of the initial motivation was to provide a service for myself I couldn’t otherwise get: a more advanced viewer for my data on iPhone and iPad that what Garmin offers.
swallow the cost. It’s a lot of money. I intentionally kept the purchase price at the minimum as to cover development cost and devices (money going back to Garmin by the way…) but my key motivation beyond my own use was sharing the app and not really generating large profits.
increase the sell price of the app and hope people will continue to buy it. It somehow feels wrong to charge the users more just for the sake of them accessing the data they should own already. Right now if I increase the price from $.99 to $2.99 for ConnectStats, I am hoping to recover the Garmin fee over time. People pay several hundred dollars for their device. I hope they will accept the few extra dollars to pay Garmin for the right to access their data.
switch ConnectStats to rely on another service. I am already working on that to use strava as primary download service and hope strava will pay the fee to get the data from garmin. This has some downside as some data won’t be available and Strava does not support swimming and skiing which I personally rely on.

Comparison to others services

I also wanted to comment on the fee structure proposed. A fee of $5000 is a lot of money by any standard. Niche independent Apps do not generate much revenue, mostly cover the costs. Here are some other services I use as comparison.

Apple charges $99 a year plus 30% of every sale to access their development tools and app distribution infrastructure. And this infrastructure and marketing power is of extremely high quality and adds a lot of value to independent developers like myself and to its customer. It wouldn’t occur to me to complain about this cost.
Strava provide an access to its data for free up to a certain rate limit once you are registered. Their API is extremely well documented, modern and their support and help to my questions have been great. I applaud Strava would provide a sophisticated website and API for free when their business model does not include revenue from any hardware.
Garmin support and documentation was so far inexistent. There are no or little documentation. I have never received an answer to any of my inquiries. Everything had to be reverse engineered. It doesn’t bother me at all given it was free but for $5,000 it’s a different issue.

I have been inquiring with Garmin about what exactly is provided when I pay the fee. What exactly do I get for that money: same API? Do I need to adapt my code to a new API? Do I get more documentation? Better support? Some marketing/promotion of the app as Apple provides?

Proposal

I believe the independent and small 3rd party partners add a lot of value to Garmin and the ability for users to access freely their own data is important. I understand the potential infrastructure cost and need to control usage for Garmin.

I suggest they offer a tiered system. Any user of the API would have to register and get some basic number of API access per day, say 50,000. Users above that limit would be required to pay the $5,000 fee to use the service. This would allow them to recoup some infrastructure cost from heavy user but still allow their customers to benefit from 3rd party functionality and access to their data.

What I plan to do for ConnectStats

I am doing 2 things:

I have written to Garmin to explain why I feel a tier system would be better for all parties involved. I hope they will agree.
ConnectStats is somewhat successful, receives good reviews and regular download. I hope by raising the price by an additional $2 this should help sponsor and recover the fee from Garmin. I have applied to be part of the program. Assuming Garmin approves my application, I plan to pay the fee myself and hope people will continue buying the app at the higher price to recover the money over time

Thanks for reading.

ConnectStats Blog

Connectstats development and use in my activities

Category Archives: Garmin