Category Archives: iOS development

AppStore Submission Rejection Syndrome

Posted on by
5

Users reported an occasional Access Denied error with the Strava service and last week ConnectStats release. Thanks to the bug report, it was fairly easy to get to the specific workflow in the strava service that was confusing ConnectStats. The fix was the easy part. Then I went on to the dreaded “submit and get apple approval” part of the release process.

The review process is a bit of a black box random process. You submit the same application over and over and sometimes it’s approved in a day, sometime faster, sometime slower, and sometime you get “Rejected”…

Continue reading →

How to easily move iOS Simulator data files after Xcode upgrade

Posted on by
Reply

If you maintain an app like ConnectStats that has a lot of local files on the device and does not rely on the cloud constantly for input, each time Xcode upgrade all the iOS Simulator starts empty again and loose all the files from the previous version.

The iOS simulator keep it’s file in some complicated folder structure and the folder change each time you start it, so it can be quite tricky to locate the files. And so quite painful to locate all the folder for the previous and new version and move all the files over.

I’ll go over the solution I use in this post, using a small cli tool I wrote

Continue reading →

Swift Package for Fit File Parsing

Posted on by
2

Garmin sends ConnectStats all the data as Fit Files, so I had built a library to parse it for the app. I used the code from the official Fit SDK. Originally I used the c++ implementation provided in the SDK as it was providing a way to get all the data pretty generically. The SDK while quite generic was actually quite slow in parsing.

About 2 years ago, as I wanted to get ConnectStats to get the extra information from the fit file and parse them systematically for each activity, the speed started to become an issue. So I wrote a new library in swift based on the c implementation of the SDK.

That ended up much faster than the previous version, and ConnectStats has been using it for a while, as well as FitFileExplorer which is a tool I wrote to see the raw fit files.

While I had provided that library on GitHub and it was open source, it wasn’t really easy to integrated into another app. Following a request last week, I investigated how to make it a Swift Package which I wasn’t really aware of. Always a good challenge to learn something new!

Continue reading →

How to AirDrop to Linux

Posted on by
3
A Father and Son File Sharing App Project

I mostly work and play within the Apple ecosystem and AirDrop, Continuity and other integration between devices is extremely useful and convenient. My son recently converted himself to Linux and started to miss the ability to easily move photos, text, or url between his phone and his Linux computer. Being a hacker himself, he thought it would be a great idea to devise an app that let’s you move information from your phone to your computer, and more generally a “cross platform AirDrop substitute for geeks”…

Of course, one solution would be to google and figure out if such an app exists, but where is the fun in that? We decided to take on a Father-Son new project and build our own solution.

Continue reading →

Debugging with Xcode and Python time series calculations

Posted on by
Reply

Trying to understand what was wrong in my Best Rolling Curve Calculation pushed my ability to use Xcode debugger to its limit. The quirks I was trying to understand were happening on large time series after multiple level of sampling, rolling averages, etc on noisy data.

I ended up in a mixed environment so I could easily exchange time series data between Xcode and pandas in a Jupyter notebook, which enabled me to explore and get to the bottom it!

Continue reading →

Important release to correct data import problems

Posted on by
3

A new version 4.4 is now available.

This fixes the issue around missing data on import, as well as incorrect units for some fields like min or max elevation.

Bad Release Problem Solved!

As a bonus, I added support for Garmin Device Power while running, as least for the devices for which users sent me sample fit files.

Missing Data

Because of my own silly bug, ConnectStats was not trying hard enough to look for all activities on the initial download, and was stopping too early, resulting in missing older activities. This is now fixed in 4.4. But it’s hard for the app to know if it missed old activities or not. So if you think you are in that situation, the best is to force the app to reload all activities, by going to “settings”, selecting “profile” and “Force Reload Old Activities” as below. This will start a download that will try to reload everything, it will also help fix data previously imported with incorrect units.

 

Incorrect Units

Even though I have build a lot of tests over time, it is always difficult to make sure when a new API changes that the app is behaving properly. This is due to the fact that the API is undocumented and that I have limited scope for testing of new features, given that I do it on my own. I basically only test on my activities and the activities people have sent me over time in the past. Though we should not complain about the lack of documentation given as mentioned in the past, this API is not intended for external use, but just for garmin own website… So I am just grateful Garmin lets ConnectStats continue to use it, even if we just have to reverse engineer everything.

To illustrate the issue and give people a feel for the reason behind some of the bugs, the bad units were due to inconsistencies in the scale of the number depending on which data you were receiving: summary or detail for an activity.

In the summary the elapsed duration data comes as milliseconds:

While in the detail file it comes as seconds…Beside the elapsedDuration, maxElevation and minElevation everything else is consistent… Go figure… These are hard to anticipate and catch when you test on your own, so I want to thank all the users who have the patience to report details about such bugs and also the patience to wait for me to push fixes.

 

What the Alternative API mean for ConnectStats

Posted on by
5

Most users experienced new activities from Garmin not syncing up with ConnectStats since last week (March 23rd or so). As described in the last post, it can be fixed by using an alternative API.

It looks like it’s going to be a permanent change. It also has some minor consequences for users.

What is the alternative API?

It’s probably obvious to most of you, but the API is the mechanism that third party (ConnectStats) can use to access the data stored by a provider (Garmin or Strava).

Garmin a long time ago had a public API to access the activities recorded in Garmin Connect. Back in 2014, Garmin decided to change it’s approach to third party. While the old API allowed a third party to access directly the running or cycling data from Garmin servers, they decided to switch to a push approach. The key consequence is that a third party now has to build and maintain an infrastructure to store the data in the cloud after it was pushed by Garmin. This is great for a platform like Strava, it makes for a great user experience. But it’s impractical for small independent hobbyist developers like myself with ConnectStats like app.

While Garmin announced they would deprecate the old API in 2014, in practice until last week it continued to work more or less. It just had regular tiny changes, which beside the fact that the change came as a surprise outage to developer via lots of sudden bug reports, they were usually easy to keep up with.

The old API continued to work and was used by their own Garmin Connect website. since they upgraded their website to the new look shortly after the API announcement, they also slowly started to upgrade the site to a very different API. This is what the settings “Alternative API” refers to.

Practical Difference between the two API

ConnectStats downloads data in two stages. The first is to get a summary of all recent activities, which is intended to be faster. Then when you go and look at an activity in details, it will downloads all the individually recorded data point to create the graphs and the track on the map. The details can be a lot of data, so it’s an optimisation to only download the summary first.

The main difference between the two APIs, is that the new one (alternative) contains a lot less information in the summary. Using the new API, a lot of the extra information (for example: running dynamics, min and max, etc) will only be downloaded while getting the full detail of the activity. It’s not a major issue, as in practice when you select an activity, you need to wait a bit for the graphs and map data anyway.

Where it can impact the user experience is in some of the historical statistics pages. Some historical statistics on the extra information will be missing until each activity detail is downloaded. To mitigate the issue, ConnectStats tries to automatically download the last dozen or so activities that are missing detail.

Will the old API come back?

Only Garmin knows. But at this point I would anticipate it won’t. ConnectStats lived on borrowed time since 2014 when they announced they would deprecate the API.

Also the behaviour of the old API is not that it errors, but it contains a lot less data (almost none). In the past changes, it always was either some error, or some subtle change that broke ConnectStats use of the API. This time feels different.

What’s next?

I am releasing a new version of ConnectStats (4.3) that will by default use the new API. I keep the setting to switch back, in case things change in the short term and to keep optionality.

As in all the previous discussion on API change, I also continue to prepare for a day when Garmin may completely shutdown the API ConnectStats uses.

At such time the only option to continue using ConnectStats will be to switch to another service provider like Strava.

Using Strava data though has the drawback that it contains a lot less details on each activities (no running dynamics, etc).

Alternatively, now the the app is open source, if it comes to that we may link it to other service and hopefully the app will continue to live.

 

ConnectStats is now open source

Posted on by
1

I lately have not been very active in ConnectStats development. I have been quite busy, which didn’t help, but also I do not have a lot of new ideas, beside a few little features requests people sent me.

So I have decided to open source the code for ConnectStats (and my other apps). Maybe some people will want to help, or the code could help others who’d want to build similar apps. Hopefully more people will help think of new ideas or make the app better.

I definitely plan to continue maintaining the app. I released a new version (4.0) which will not have many new features but will be in sync with the refactoring and little cleanup I made before pushing the code to GitHub.

You can find the code here https://github.com/roznet/connectstats

The repository also contains the code for HealthStats, which I need to fix to work with the latest version of iOS, FitFileExplorer, an utility for macOS to open files, as well as TennisStats, an experiment to record and analyse tennis matches.

I also open sourced a few more of my apps:

My other iOS app MacroDial can also be found here https://github.com/roznet/macrodial.

And finally Simulator Data Finder, an utility to access iOS simulator files conveniently on macOS, is also available here https://github.com/roznet/iossimfinder.

 

2016 year review and double counted activities

Posted on by
2

It’s time again to look at the past year summary. How did I do?

2016 was a fine year, I ran 1387 km, it is 100km behind 2015. But one aspect bothered me, looking at the distance, there is a clear increase on the graph in December 2015, which put me behind in 2016. It worried me a bit. I couldn’t remember an especially good December last year. Did I have another bug in ConnectStats?

I went to check on Garmin connect and it reported the same distance as connectstats: 1478km for 2015…

Hum. Next step was to go and look at the activities in December last year. The search feature in ConnectStats made it easy, just enter december 2015  in the search box of the activity list. Sure enough, I found some duplicate activities. This started to be very worrying. ConnectStats imported the same activity twice? New bug? After some more investigation, it turned out the problem was in Garmin itself… Quite a few activities in 2015 appear twice on the website as well…

They have a different activity identification number, but they are clearly the same activity except for the altitude gain as you can see on the snapshot above.

Next steps was to add a new feature to ConnectStats to double check for such duplicate activities and ignore them… Well, my stats are now lower only 1310km but correct. And 2016 is ahead of 2015 by 50km! Yeah!

I finally decided to check what Strava reported. My account is linked to import activities automatically. I was curious: would strava have the total including the duplicate as well? It actually didn’t… The total is 5km off from my new corrected total, but I suspect it is simply due to the slightly different way Strava computes the total distance from the gps file.

The fix to search and eliminate duplicate will be included in a new release early next year.

Update on development and new API issues

Posted on by
21

There is shutterstock_120037903currently quite a few issues with ConnectStats following some changes in the Garmin API. I am working on it but it is quite more involved than I initially thought.

As of last release, the laps are not downloaded anymore, swimming activities do not download properly, nor do the multi sport activities. This is all due to an API access that disappeared from Garmin last week. I released an emergency fix that let people going for the main types of activities (running and biking) but I am aware there are still lot more issues. To fix the rest, I need to upgrade to a newer access point that is significantly different from the old one. I am also trying to future proof the app by removing use of all older API similar to the one that disappeared that ConnectStats is still using. I want in addition to improve the ability to seamlessly switch to Strava should Garmin decide to shut down everything for ConnectStats.

Also because the changes required currently are quite fundamental (completely new APIs), it requires quite a bit of testing and even then the testing is limited by the fact I only have limited set of activities to test (mine plus a few people sent me over the years). So it is possible that after the next release some issues will appear when others try it, but I will do my best to fix as soon as possible.

As I have explained in the past, I work on ConnectStats on the side of a demanding day job and a family, so development can be quite slow, given I can only work on it a few hours a week at best.

I am sorry about this. The most frustrating aspect for me is that I spent the precious few hours I have for development of ConnectStats on pure maintenance and not on new features. It has limited benefit for the users beside keeping the app running…

Currently I estimate it will take another week or two to complete the upgrade.