Learning from Services Woes

So in the last few weeks, ConnectStats had a lot of issues related to online services. That was quite a learning experience. I hope most of it is behind, but could be a few left overs.


Strava made a change in its API related to how it authenticate users for access. This was a documented and announced changed, unlike a lot of the Garmin API changes in the past,. The issue is mostly that I hadn’t understood the change, so I didn’t get the right fix out in time. So I have only myself to blame…

Because I don’t use Strava in ConnectStats myself too much except for testing, I had not noticed that my initial fix for the API change only worked after you did a new login process. In my tests, I was always login from scratch and it was working. But many users that had already logged in before the change and my fix ended up stuck in a failure state. A workaround was to go to the config page, in the strava settings and force a log out, after the next login things should work again.

I also now have pushed a new version that will force a logout/login cycle when it is in the bad state and now have better and better understanding of the OAuth 2.0 authorisation protocol…

ConnectStats Service

So after the previous set of issues, the new service had managed to be stable for new users for a bit over a week. Everything was running smoothly, so I decided to extend the rollout to existing users as well.

This worked well for a week, I added close to 2000 users in one week, which is not bad. But that’s when I suddenly got a notification from my website service provider that my account was suspended because it was creating an overload of database resources…

So I called my service provider and I upgraded my service to the next tier.

This unfortunately didn’t work very well, I noticed that somehow I had received a huge surge of activities at once. While I still don’t understand why that happened, I reported it to Garmin for investigation, but the result was even the new server was overwhelmed.

So I did some scary “on the fly” surgery of the database, deleted what appeared like the spurious activities until everything came back to normal.

In the process I noticed that the new server, while it had less constraints on resources, was actually processing information much slower that the old one, which resulted in further issue because I had implemented a timeout for processing, which was constantly hit and therefore making the situation worse (after timeout it would kill a process and restart it from the beginning).

So I reached out to my website hosting provider who advised me to purchase yet the next tier of server, which I did.

Things got better, the spurious activities stopped coming and the server was happy processing request.

That said, I noticed from the app it was clearly still noticeably much slower to download activities files to ConnectStats. But at that point I clearly had a better server and there was no way I needed more power. So after some research on optimisation of MySql, there was some simple solution which I implemented and now everything seems to be happy and rolling.

I also now have learned a lot about building a scalable online service online from scratch…

Virtual Ride

Another aspect that is hard for me to test is the use of ConnectStats with Virtual Rides or Runs from Zwift. They clearly are becoming more and more popular as I had quite a few users reaching out to highlight issues with those.

Virtual activities broke when using the new service or strava as a service in the app. So I have pushed a new version of ConnectStats that attempts to fix this. As any feature I don’t use myself, it’s a bit tricky to fix, as I can’t test it. I have to blindly iterate through, push the fix and see from other users if it worked… Hopefully the latest release will fix all remaining issues, but if it doesn’t, I appreciate Zwift users patience and feedback until I get it right…

Getting the weather for an activity in ConnectStats

As of version 5.1 you can obtain weather information for Connectstats activities if you use the new service I implemented using Garmin Health API and DarkSky. But in order to access it there are a few things to know and do.

The weather will only be added to new activities, once you have started using the new service, which is not currently the default in the app.

To enable the new service, you should go to your config page in Service and accounts and change the Source for the Garmin information to be All as below.

Note that if you do that it should preserve the history you have already downloaded from the Garmin Connect website and only start downloading new activities from the new ConnectStats service. Note that in case of issue you can always revert to only using the Garmin Website as a source.

New activities after you have completed the first login will start having the weather information added.

The reason I didn’t back populate the weather on older activities is both technical, as trying to back populate the older activities was a bit tricky to handle and cost, as I was worried it would make too many request to the weather API and start costing too much.

New services and Withings working again

Finally I got withings and the new service for ConnectStats to work again with version 5.1! At the time of this writing version 5.1 is pending apple approval for release.


The Withings authentication process has been broken for a while…

It took me trying a whole new approach to do the authentification process for the withings API, a few exchanges with withings support and learning way more than I ever wanted to know about the OAuth 2.0 protocol to get it all in the right place again.

The key change I believe is that withings started to have the OAuth 2.0 authentification token to expire and the way I was using the authentification library was slightly incorrect. But before I tried to use that library correctly, I tried to upgrade to a more modern approach than the one I was using that had been declared obsolete. These though required Withings to make adjustment on their side (non http scheme), which they kindly did. I never got it to work. Ultimately, after studying the deprecated library, using the ever more useful proxy app Charles I finally figured out how to get things working again! In the process I also streamlined the way the setup screen works, fix the ability to logout and back in, and synchronised the interface into Strava to use the same new code.

New ConnectStats Service

To use the new Garmin Health API, I had to implement a server that would receive, process and save all the activities of ConnectStats users when their device uploaded them to Garmin.

I knew that processing these requests as they came in realtime was a bad shortcut in the first implementation of the service for the new Garmin API. I had not expected how bad the deadlock and timeout would be when used in real life as often many files would be sent to my server at once. I had adopted the “make it work, optimise later” approach, but here the optimise step was immediately needed…

While I had postponed implementing it, I knew what the right approach was: I had to add all the incoming requests from the service to a queue and process sequentially a few at a time.

To host my server, I am using an entry level service from GoDaddy, which is a great relatively cheap hosting but with fairly limited optionality in configuration. Which meant it wasn’t an option to bring some pre-built queuing system. I had to build it myself with the tools available…

It’s of course not advisable to implement such a common system yourself, but, well, I do all this to learn anyway… It was definitely interesting to figure out how to make sure it behaved in a robust manner in all the edge cases I could think of. As with Connectstats itself, I open sourced the code of the server. Hopefully it can be useful to someone interested in having a lightweight server to work quickly with the Garmin Health API or can help people struggling to implement their own.

I am releasing this now in version 5.1. In that version only new users will use the new service by default, but existing user will have to switch manually in the configuration page. Crossing finger that this time it will work better and scale…

Bonus with the new service

Beside resiliency to the next API change from Garmin on their web site, the new service allowed me to bring in weather information from DarkSky, so that’s an added bonus for using the new service. Because I now fully control the service that serves activities to Connectstats, it maybe possible in the future to add more such new features.

Thought on servers, hosting, services etc

This experience made me appreciate better the differences in services. I use a service from GoDaddy that is very simple, turn-key and fairly cheap. While it forces you to be a bit more nibble in the implementation, it was a bit interesting to compare it to using more scalable and more configurable services like DigitalOcean or AWS. I briefly considered, but it’s clear that it would require a non-significant learning process and higher costs. So I am sticking with GoDaddy for now, and hope the system will handle an increase of users without requiring me to upgrade to a fancier hosting service.

The weather service from DarkSKy is very nicely done and reasonably priced and free for the first 1000 calls per day. I anticipate based on the current number of users that the cost will remain small.