ConnectStats is now open source

I lately have not been very active in ConnectStats development. I have been quite busy, which didn’t help, but also I do not have a lot of new ideas, beside a few little features requests people sent me.

So I have decided to open source the code for ConnectStats (and my other apps). Maybe some people will want to help, or the code could help others who’d want to build similar apps. Hopefully more people will help think of new ideas or make the app better.

I definitely plan to continue maintaining the app. I released a new version (4.0) which will not have many new features but will be in sync with the refactoring and little cleanup I made before pushing the code to GitHub.

You can find the code here

The repository also contains the code for HealthStats, which I need to fix to work with the latest version of iOS, FitFileExplorer, an utility for macOS to open files, as well as TennisStats, an experiment to record and analyse tennis matches.

I also open sourced a few more of my apps:

My other iOS app MacroDial can also be found here

And finally Simulator Data Finder, an utility to access iOS simulator files conveniently on macOS, is also available here


2016 year review and double counted activities

It’s time again to look at the past year summary. How did I do?

2016 was a fine year, I ran 1387 km, it is 100km behind 2015. But one aspect bothered me, looking at the distance, there is a clear increase on the graph in December 2015, which put me behind in 2016. It worried me a bit. I couldn’t remember an especially good December last year. Did I have another bug in ConnectStats?

I went to check on Garmin connect and it reported the same distance as connectstats: 1478km for 2015…

Hum. Next step was to go and look at the activities in December last year. The search feature in ConnectStats made it easy, just enter december 2015  in the search box of the activity list. Sure enough, I found some duplicate activities. This started to be very worrying. ConnectStats imported the same activity twice? New bug? After some more investigation, it turned out the problem was in Garmin itself… Quite a few activities in 2015 appear twice on the website as well…

They have a different activity identification number, but they are clearly the same activity except for the altitude gain as you can see on the snapshot above.

Next steps was to add a new feature to ConnectStats to double check for such duplicate activities and ignore them… Well, my stats are now lower only 1310km but correct. And 2016 is ahead of 2015 by 50km! Yeah!

I finally decided to check what Strava reported. My account is linked to import activities automatically. I was curious: would strava have the total including the duplicate as well? It actually didn’t… The total is 5km off from my new corrected total, but I suspect it is simply due to the slightly different way Strava computes the total distance from the gps file.

The fix to search and eliminate duplicate will be included in a new release early next year.

Update on development and new API issues

There is shutterstock_120037903currently quite a few issues with ConnectStats following some changes in the Garmin API. I am working on it but it is quite more involved than I initially thought.

As of last release, the laps are not downloaded anymore, swimming activities do not download properly, nor do the multi sport activities. This is all due to an API access that disappeared from Garmin last week. I released an emergency fix that let people going for the main types of activities (running and biking) but I am aware there are still lot more issues. To fix the rest, I need to upgrade to a newer access point that is significantly different from the old one. I am also trying to future proof the app by removing use of all older API similar to the one that disappeared that ConnectStats is still using. I want in addition to improve the ability to seamlessly switch to Strava should Garmin decide to shut down everything for ConnectStats.

Also because the changes required currently are quite fundamental (completely new APIs), it requires quite a bit of testing and even then the testing is limited by the fact I only have limited set of activities to test (mine plus a few people sent me over the years). So it is possible that after the next release some issues will appear when others try it, but I will do my best to fix as soon as possible.

As I have explained in the past, I work on ConnectStats on the side of a demanding day job and a family, so development can be quite slow, given I can only work on it a few hours a week at best.

I am sorry about this. The most frustrating aspect for me is that I spent the precious few hours I have for development of ConnectStats on pure maintenance and not on new features. It has limited benefit for the users beside keeping the app running…

Currently I estimate it will take another week or two to complete the upgrade.

ConnectStats 2.5 rejected…

shutterstock_310884725So as per last week post, I had submitted to the apple review process a new version of connectstats with the 3D flyover feature…

Unfortunately, Apple rejected this version. The reason is that the code is using some of the HealthKit API. Actually it is appears to be using it because I am currently working on a version that will let you look at the steps and other information recorded via the Apple Health App or your Apple Watch. It is not yet enabled as I am still working on it and I am waiting until it’s ready for prime time. But the Apple review team rejected it on the basis that my code should not appear to potentially use health data if it’s not clear to the user why.

So I will wait until the next version is ready to release the 3D flyover, at this point would be too painful to patch an old version or  remove that code that I know I will need soon.

How to track the non reproducible bug in version 2.1

So, version 2.1 was crashing for quite a few users on startup. Obviously the app wasn’t crashing for me in any of my tests.

First line of defence when crashes happens is to ask the user what they were doing and try to replicate the same action to see if I can recreate the crash. But here they weren’t doing anything it wasn’t starting.

Second is to look for patterns in the crash. The first few users reporting the issue were all using an iPad. There are a few differences between the iPhone and the iPad, mainly that the iPad displays an activity detail at the same time as the activity list. This can result in some difference in logic and makes the startup on iPad marginally more prone to issues. This can lead to hard to reproduce bugs too as it is a function of the type of activity the user would load on startup (the last one downloaded)

Another help is the crash report from apple. In this case, they were very few and reporting crash in location that are not triggered at start up or inside apple libraries. So not helpful to pinpoint the problem.

Some users affected by the issue kindly sent me their activity data so I could try to see if the problem was linked to their specific activities data. It wasn’t, I could start the app with their data fine.

This leaves one other possibility, a thread concurrency issue. This can be really random, and a function of both the speed of the hardware and the exact activities you have. ConnectStats is quite multi-threaded to try to keep the app responsive as many calculations are performed in parallel. So it is possible that some of this parallel activity result in a collision or some instructions run too early (when required data is not ready).

I try to be very careful on collision conditions and to make the code robust to data not being ready. In version 2.1, in order to optimise the startup, I had done a clean up of all the notifications between events in the app to limit them to the minimum needed. So likely the problem was linked to that: issue affecting random users, not a logic issue (users data didn’t let me reproduce it), but the issue was systematic for some users: so this gives another hint. The one issue that can exhibit these symptoms is if you trigger a User Interface event not from the thread dedicated to the UI (main thread).

I did a review of all the events that happen on start up and found one that, in the clean up of 2.1, wasn’t not forcefully directed to that main thread…

So while I couldn’t reproduce, I feel strongly that this must be the issue:

  • linked to some change I made in version 2.1,
  • random: depends of when precisely event get triggered
  • systematic for a given user: for a set of hardware speed and size of data, always happens or not

I submitted a fix for that event issue to apple, let’s cross fingers until version 2.1.1 is approved by apple and hope the issue is indeed resolved. I requested an expedited review, but it’s not always granted.

Frustrations of a botched release

I try to release ConnectStats regularly. Once in a while, as today, I push an update with an issue. I have been receiving a lot of bug reports today from users who can’t start the app after last night release.

I know it’s quite frustrating for users, and it is for me as well. In this case so far it has been impossible for me to reproduce the issue.

So as I try to sort it out, I decided to write a bit about the testing and debugging of ConnectStats

Before the problem happens

Automatted Testing

First I wrote a bunch of tests that verify the very basic functionality of the app at the lowest level. For example it checks that all the unit conversions continue to be correct, that the statistics computation continue as expected, etc, etc

I have also built a testing sister app to ConnectStats that share most of the code, but run a bunch of test replicating the download process, or statistics on saved samples of activities.

Device Testing

While the automated testing is critical, most of the issues, like the one today I suspect are due to subtle interaction between different aspect of the app, when run with different set of activities or different devices.

Given I develop ConnectStats alone in my free time, this kind of testing is fairly limited for me. I mostly test with my phone and my activities, as well as a secondary device. For example I do not use an iPad, so issues on iPad are more likely to appear as I don’t have as much opportunity to try these.

Once a problem happens

Bug Reports

Typically, the first thing that happens is I’ll start receiving some bug report. The bug report contains workflow information and diagnostic collected as the app executes. It’s only as good as what I anticipated to record. But quite a few bug can be understood this way, especially when people email me and answer a few questions about what they were doing or how to reproduce the problem.

Apple Crash Reports

When as today, a large amount of people email me that the app crash on startup it’s much harder, as the log does not contain any useful information. I can also see some crash report collected by apple, but they take a while to appear. So right now I just need to wait and hope that the information in apple’s report will help me.

Bad Reviews

Of course the next thing that happens, is that ConnectStats will start receiving bad reviews. It’s basically a race, fix it before too many bad reviews accumulate…


Working with iOS Simulator files

Simulator Data Finder greatly improves my workflow in working and debugging apps in Xcode 6. You can download it here. My apps relies on many downloaded json and xml files, as well as keeping all its data as sqlite files. I use Simulator Data Finder to easily access these files, copy them from one simulator to another or bring into the simulator the files from a device.

Accessing the files

The first use is to access the files, either by the finder or terminal. The terminal is actually quite handy to work with sqlite files and sqlite3 directly, use grep or any kind of other shell tools for old school developers… Simulator Data finder has a clipboard button so when terminal’s tool are not enough, it’s easy to paste the directory and access it from emacs for example.


Copying file between iOS Simulator versions

Another typical use is when you have a lot of files and you want to keep the same setup into a new iOS Simulator version. For example you have everything setup on the iPhone 6 simulator, but want to see how it looks on iPhone 5. Simulator Data Finder makes that a breeze. You can easily choose the two simulators from the table, open the two Documents folder with the finder and copy the files over.

This makes it very easy when a new version of iOS or Xcode comes out to move all the files you need. The older version will be easily accessible and clearly labeled.

If you work across two device simulators, the modified date also lets you see at a glance which of the two simulator has the latest modified files.


Copying file from device to the simulator

The other task that Simulator Data Finder makes easy is to move around of files between the device container and the simulators.

When a developer device is connected, download the application container to your download folder using the standard name. Simulator Data Finder will then present all the container it finds in the Download folder that match the app you have currently selected. If you pop up the finder for the container and for the simulator it becomes very easy to copy files. The containers will also be conveniently sorted with the most recent on the top of the table.


Adapting and sharing code for iOS and OS X

So now that I have started writing OS X application for the mac and there are obviously a lot of similarity with iOS I decided to write some companion app on OS X for my new project TennisStats.

Most of the code is quite abstracted between the UI and the data logic behind, which helped a lot, but I embarked upon sharing my main plotting library inherited from ConnectStats. What will it take to share that library between iOS and OS X?

Compiler Macros

The first thing was to figure out the right macros to predicate the code between the two OS. TARGET_OS_IPHONE and TARGET_OS_MAC are define and the one to use here. Though one thing I quickly learned was the really I should use TARGET_OS_IPHONE as TARGET_OS_MAC is defined both for OS X and iOS.

The key library can then be imported with #if. Note that CoreGraphics seems to be the same across both.

UI and NS prefix

A lot of the classes prefixed by UI provided by apple on iOS have equivalent named with NS. Was it going to be as easy as using one for the other?

Brief look at the class reference showed a lot of similarity and iOS was derived from OS X, so why not… So I started to add the following typedefs

And the replace each instance of UIColor, UIView, etc with the equivalent version prefixed with RZ. Most of it compiled! NSBezierPath has the most differences, and I had to write an extension to provide the following functions

So now it compiles… Will it just work and is that easy?


Oops… A new issue to worry about, the coordinates in OS X and iOS have their Y axis inverted… Luckily I had quite nicely abstracted the CGPoint calculation in a Geometry class… So a key target predicate

And a few more around the title and axis location, et voila!


In Summary

It was much easier than I expected. And definitely makes the thought of building an OS X companion app very realistic…

Find the iOS Simulator Document Directory

I really enjoy working with Xcode 6, but it has been quite annoying that the iOS Simulator changes the name of the directory it uses as data container each time you start it. There are a few manual method to find and get to the document container I had been using. I will describe them here. Because my apps tends to use a lot of files in data that I need to check while debugging, I also created a little tool that wraps the manual methods into a very useful (at least to me) and easy workflow. you can download the app here.

Manual logging

The easiest way is to add logging of the directory location on startup. I typically make it conditional to being in the simulator as to no clutter the log in a device. Here is what I typically add to the applicationDidFinishLaunching function.

I then copy from the output of the console the directory and paste it into finder, terminal, emacs or your tool of choice

Simple. Efficient. Tedious

Needle in a haystack

A different approach would be to save a small file in the document directory on start up which then allows you to find that simulator without having to start the app and refer to the console. It would look something like this:

If as above you save a file with the app identifier in the name as above, you can then later generically find that directory using a find command and then wrap that into your favourite scripting language


You can also find a file in the Library of the device directory of a simulator that contains the path of the last container. That file is under the path Library/MobileInstallation/LastLaunchServicesMap.plist. You can then print it using plutil -p LastLaunchServicesMap.plist, the key User contains information about each application in the simulator. Somehow it still seems to be missing some apps, but contains most of them.

Simulator Data Finder

To make my life easier, I built this app that leverages the last two methods to make it easy for you to find your simulators and files. The app can present you with a list of simulators and app as below. It then has a easy access button to finder, copy to clipboard or terminal for the path. It will try to find the directory using the LastLaunchServicesMap.plist file and if it still doesn’t find it, you can download this header file and add the following macro call in your applicationDidFinishLaunching,

You can download the app here. You can read here why I was unable to publish this app in the apple App Store which would have been more convenient.

One added bonus to the app is that it organises for easy access container you’d have downloaded from a device for a given app. It matches them by bundle identifier and currently looks for them in the download directory.

The path to rejection in the apple App Store

I have been developing apps on iOS for quite a while now as a side hobby. The latest Xcode has an ever changing directory for the Xcode simulator, which has been a bit painful to work around, despite a few tricks. So I decided it was a good opportunity for me to get into OS X application development and build a little tool that would let me organise and access files on the simulator for each app work on. I had been always writing my user interfaces fully programmatically so I also decided I would take this opportunity to explore interface builder and storyboard.

The first impression of OS X was that it’s actually not that different, some classes have different names and APIs but it’s remarkably similar. I felt right at home using NSTable. I got the basic of the app working very quickly and it instantaneously proved quite useful to my app development workflow. So much so that decided to share it on the AppStore as a free utility for other developers.

So on I go, paid my $99 to apple to become a Mac registered developer, as until know I only was an ios developer.


First hurdle was that I had to sandbox the application. This meant that access to the terminal via AppleScript stopped working, but most significantly it wasn’t possible to access the simulator directory as it was outside the sandbox. I used the terminal as I wanted the app to let you pop up a finder window or a terminal window on the directory of a selected app.

Not to worry, according to the sandbox guideline you can access any directory as long as the user grant access by showing intent: all I had to do was to bring up a open dialog window on the directory where the simulator reside and once the user pressed ok, the app had access.

For the terminal, once you figure the syntax you could explicitly ask for access to terminal in the entitlement file. Et voila, all the functionality was working again in a sandboxed app.

There was though one major issue. Every time you pop up the open dialog, it would pop up a new duplicate window. It was really annoying and drove me nuts for a while. Without sandboxing it was all working fine, as soon as you turned on sandboxing that duplicate window kept coming up. I searched in the development forums, documentation, etc. I didn’t find any solution or mention of that problem. So I filed a bug report (easy to reproduce the problem with a very simple project) and used one of my support tickets (you get two as a registered developer) to ask an apple engineer for help.

Almost two weeks later I got the answer from apple. It is a bug on apple side, to solve it I should not use storyboard… Great. So I refactored all my code to use an xib file instead of storyboard and now it worked!

First Submission

Feeling all set to go, I purchased an icon on shutterstock, quickly added some help and documentation to the app and submitted it to the Apple Store for review. One week later, I got my first rejection. I so far never received a rejection for my iOS app, so it was quite disappointing. The feedback was that I used entitlement that I shouldnt.

I figured this must have been the access to terminal. Even though it seemed to use the standard method to request entitlement, I had always been a bit worried in the back of my mind that accessing terminal could be a security issue because it would let you execute potentially damaging commands.

It made sense and I removed that functionality and resubmitted the app. One week later another rejection with the same reason. You can’t be serious! Actually it was: silly me I had resubmitted the app without selecting the new binary, so they reviewed the same app I had submitted the first time. Another week wasted.

Final Rejection

I carefully selected the updated binary and resubmitted the app.

Almost two weeks later the app goes into review. There are typically two stages in a review: waiting for review, that in my previous experience takes a few days to a week, then it becomes in review, which takes a few hours to a day or two. Nice, I felt hopefully it will go through soon…

Not so fast. A few days pass. Then a week. Still in review. I almost forgot about it when after over three weeks, I start to wonder. Did something happen? I did some research on the Internet, app review times gives some average days to review of 8 or 9 days. I am way beyond… After a month of being in review I write to apple asking if there is any information they need from me to review the app. The answer comes quickly: no information needed, they are still reviewing the app and just had to be patient.

After more than once month I just received the final rejection. The app is rejected because “I am modifying user file in a way that is not publicly documented”. I am not modifying any file in the app! But it then hits me: apple just does not want me to write this app.

I write a last appeal to the reviewer: I do not modifying any file in my app, only bring up a finder window on a directory after the user has granted access and showed intent by clicking Authorize on an open dialog box. The final answer from the reviewer: we just don’t want app to access simulator files.

Here we go. Hard to not feel a bit of frustration and disappointment, almost three months after starting the app. I still believe this app is really useful and became an important part of my debugging workflow. So I put a bit of wrap around and made it available to download via this website and hope others will find it useful as well.