Category Archives: iOS development

ConnectStats 2.5 rejected…

Posted on by
1

shutterstock_310884725So as per last week post, I had submitted to the apple review process a new version of connectstats with the 3D flyover feature…

Unfortunately, Apple rejected this version. The reason is that the code is using some of the HealthKit API. Actually it is appears to be using it because I am currently working on a version that will let you look at the steps and other information recorded via the Apple Health App or your Apple Watch. It is not yet enabled as I am still working on it and I am waiting until it’s ready for prime time. But the Apple review team rejected it on the basis that my code should not appear to potentially use health data if it’s not clear to the user why.

So I will wait until the next version is ready to release the 3D flyover, at this point would be too painful to patch an old version or  remove that code that I know I will need soon.

How to track the non reproducible bug in version 2.1

Posted on by
Reply

So, version 2.1 was crashing for quite a few users on startup. Obviously the app wasn’t crashing for me in any of my tests.

First line of defence when crashes happens is to ask the user what they were doing and try to replicate the same action to see if I can recreate the crash. But here they weren’t doing anything it wasn’t starting.

Second is to look for patterns in the crash. The first few users reporting the issue were all using an iPad. There are a few differences between the iPhone and the iPad, mainly that the iPad displays an activity detail at the same time as the activity list. This can result in some difference in logic and makes the startup on iPad marginally more prone to issues. This can lead to hard to reproduce bugs too as it is a function of the type of activity the user would load on startup (the last one downloaded)

Another help is the crash report from apple. In this case, they were very few and reporting crash in location that are not triggered at start up or inside apple libraries. So not helpful to pinpoint the problem.

Some users affected by the issue kindly sent me their activity data so I could try to see if the problem was linked to their specific activities data. It wasn’t, I could start the app with their data fine.

This leaves one other possibility, a thread concurrency issue. This can be really random, and a function of both the speed of the hardware and the exact activities you have. ConnectStats is quite multi-threaded to try to keep the app responsive as many calculations are performed in parallel. So it is possible that some of this parallel activity result in a collision or some instructions run too early (when required data is not ready).

I try to be very careful on collision conditions and to make the code robust to data not being ready. In version 2.1, in order to optimise the startup, I had done a clean up of all the notifications between events in the app to limit them to the minimum needed. So likely the problem was linked to that: issue affecting random users, not a logic issue (users data didn’t let me reproduce it), but the issue was systematic for some users: so this gives another hint. The one issue that can exhibit these symptoms is if you trigger a User Interface event not from the thread dedicated to the UI (main thread).

I did a review of all the events that happen on start up and found one that, in the clean up of 2.1, wasn’t not forcefully directed to that main thread…

So while I couldn’t reproduce, I feel strongly that this must be the issue:

  • linked to some change I made in version 2.1,
  • random: depends of when precisely event get triggered
  • systematic for a given user: for a set of hardware speed and size of data, always happens or not

I submitted a fix for that event issue to apple, let’s cross fingers until version 2.1.1 is approved by apple and hope the issue is indeed resolved. I requested an expedited review, but it’s not always granted.

Frustrations of a botched release

Posted on by
Reply

I try to release ConnectStats regularly. Once in a while, as today, I push an update with an issue. I have been receiving a lot of bug reports today from users who can’t start the app after last night release.

I know it’s quite frustrating for users, and it is for me as well. In this case so far it has been impossible for me to reproduce the issue.

So as I try to sort it out, I decided to write a bit about the testing and debugging of ConnectStats

Before the problem happens

Automatted Testing

First I wrote a bunch of tests that verify the very basic functionality of the app at the lowest level. For example it checks that all the unit conversions continue to be correct, that the statistics computation continue as expected, etc, etc

I have also built a testing sister app to ConnectStats that share most of the code, but run a bunch of test replicating the download process, or statistics on saved samples of activities.

Device Testing

While the automated testing is critical, most of the issues, like the one today I suspect are due to subtle interaction between different aspect of the app, when run with different set of activities or different devices.

Given I develop ConnectStats alone in my free time, this kind of testing is fairly limited for me. I mostly test with my phone and my activities, as well as a secondary device. For example I do not use an iPad, so issues on iPad are more likely to appear as I don’t have as much opportunity to try these.

Once a problem happens

Bug Reports

Typically, the first thing that happens is I’ll start receiving some bug report. The bug report contains workflow information and diagnostic collected as the app executes. It’s only as good as what I anticipated to record. But quite a few bug can be understood this way, especially when people email me and answer a few questions about what they were doing or how to reproduce the problem.

Apple Crash Reports

When as today, a large amount of people email me that the app crash on startup it’s much harder, as the log does not contain any useful information. I can also see some crash report collected by apple, but they take a while to appear. So right now I just need to wait and hope that the information in apple’s report will help me.

Bad Reviews

Of course the next thing that happens, is that ConnectStats will start receiving bad reviews. It’s basically a race, fix it before too many bad reviews accumulate…

 

Working with iOS Simulator files

Posted on by
Reply

Simulator Data Finder greatly improves my workflow in working and debugging apps in Xcode 6. You can download it here. My apps relies on many downloaded json and xml files, as well as keeping all its data as sqlite files. I use Simulator Data Finder to easily access these files, copy them from one simulator to another or bring into the simulator the files from a device.

Accessing the files

The first use is to access the files, either by the finder or terminal. The terminal is actually quite handy to work with sqlite files and sqlite3 directly, use grep or any kind of other shell tools for old school developers… Simulator Data finder has a clipboard button so when terminal’s tool are not enough, it’s easy to paste the directory and access it from emacs for example.

OneSimulator

Copying file between iOS Simulator versions

Another typical use is when you have a lot of files and you want to keep the same setup into a new iOS Simulator version. For example you have everything setup on the iPhone 6 simulator, but want to see how it looks on iPhone 5. Simulator Data Finder makes that a breeze. You can easily choose the two simulators from the table, open the two Documents folder with the finder and copy the files over.

This makes it very easy when a new version of iOS or Xcode comes out to move all the files you need. The older version will be easily accessible and clearly labeled.

If you work across two device simulators, the modified date also lets you see at a glance which of the two simulator has the latest modified files.

BetweenSimulator

Copying file from device to the simulator

The other task that Simulator Data Finder makes easy is to move around of files between the device container and the simulators.

When a developer device is connected, download the application container to your download folder using the standard name. Simulator Data Finder will then present all the container it finds in the Download folder that match the app you have currently selected. If you pop up the finder for the container and for the simulator it becomes very easy to copy files. The containers will also be conveniently sorted with the most recent on the top of the table.

DeviceAppData

Adapting and sharing code for iOS and OS X

Posted on by
Reply

So now that I have started writing OS X application for the mac and there are obviously a lot of similarity with iOS I decided to write some companion app on OS X for my new project TennisStats.

Most of the code is quite abstracted between the UI and the data logic behind, which helped a lot, but I embarked upon sharing my main plotting library inherited from ConnectStats. What will it take to share that library between iOS and OS X?

Compiler Macros

The first thing was to figure out the right macros to predicate the code between the two OS. TARGET_OS_IPHONE and TARGET_OS_MAC are define and the one to use here. Though one thing I quickly learned was the really I should use TARGET_OS_IPHONE as TARGET_OS_MAC is defined both for OS X and iOS.

The key library can then be imported with #if. Note that CoreGraphics seems to be the same across both.

#if TARGET_OS_IPHONE
#import 
#import 
#else
#import 
#import 
#endif
#import

UI and NS prefix

A lot of the classes prefixed by UI provided by apple on iOS have equivalent named with NS. Was it going to be as easy as using one for the other?

Brief look at the class reference showed a lot of similarity and iOS was derived from OS X, so why not… So I started to add the following typedefs

#if TARGET_OS_IPHONE
#define RZColor UIColor
#define RZFont UIFont
#define RZImage UIImage
#define RZView UIView
#define RZBezierPath UIBezierPath
#else
#define RZColor NSColor
#define RZFont NSFont
#define RZImage NSImage
#define RZView NSView
#define RZBezierPath NSBezierPath
#endif

And the replace each instance of UIColor, UIView, etc with the equivalent version prefixed with RZ. Most of it compiled! NSBezierPath has the most differences, and I had to write an extension to provide the following functions

extern CGContextRef UIGraphicsGetCurrentContext();
extern NSString * NSStringFromCGPoint(CGPoint point);

@interface NSBezierPath (QuartzHelper)

+(NSBezierPath*)bezierPathWithBezierPath:(NSBezierPath*)other;
+ (NSBezierPath *)bezierPathWithCGPath:(CGPathRef)CGPath;
-(CGPathRef)CGPath;
-(void)addLineToPoint:(CGPoint)point;

@end

So now it compiles… Will it just work and is that easy?

Ooops

Oops… A new issue to worry about, the coordinates in OS X and iOS have their Y axis inverted… Luckily I had quite nicely abstracted the CGPoint calculation in a Geometry class… So a key target predicate 

-(CGPoint)pointForX:(CGFloat)x andY:(CGFloat)y{
    
#if TARGET_OS_IPHONE
    return CGPointMake( (   _graphDataRect.origin.x + (x-_dataXYRect.origin.x)* _horizontalScaling),
                       _graphDataRect.origin.y+_graphDataRect.size.height - (y - _dataXYRect.origin.y) * _verticalScaling);
#else
    return CGPointMake( (   _graphDataRect.origin.x + (x-_dataXYRect.origin.x)* _horizontalScaling),
                       _graphDataRect.origin.y+ (y-_dataXYRect.origin.y) * _verticalScaling);
#endif
}

And a few more around the title and axis location, et voila!

Better

In Summary

It was much easier than I expected. And definitely makes the thought of building an OS X companion app very realistic…

Find the iOS Simulator Document Directory

Posted on by
Reply

I really enjoy working with Xcode 6, but it has been quite annoying that the iOS Simulator changes the name of the directory it uses as data container each time you start it. There are a few manual method to find and get to the document container I had been using. I will describe them here. Because my apps tends to use a lot of files in data that I need to check while debugging, I also created a little tool that wraps the manual methods into a very useful (at least to me) and easy workflow. you can download the app here.

Manual logging

The easiest way is to add logging of the directory location on startup. I typically make it conditional to being in the simulator as to no clutter the log in a device. Here is what I typically add to the applicationDidFinishLaunching function.

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
#if TARGET_IPHONE_SIMULATOR
	NSLog(@"Simulator : %@", NSSearchPathForDirectoriesInDomains( NSDocumentDirectory, NSUserDomainMask, YES)[0]);
#endif
	//...
}

I then copy from the output of the console the directory and paste it into finder, terminal, emacs or your tool of choice

Simple. Efficient. Tedious

Needle in a haystack

A different approach would be to save a small file in the document directory on start up which then allows you to find that simulator without having to start the app and refer to the console. It would look something like this:

#if TARGET_IPHONE_SIMULATOR
    NSString * __rzSimNeedlePath = NSSearchPathForDirectoriesInDomains( NSDocumentDirectory, NSUserDomainMask, YES)[0];
    [[NSData data] writeToFile:[NSString stringWithFormat:@"%@/.needle", __rzSimNeedlePath] atomically:YES];
#endif

If as above you save a file with the app identifier in the name as above, you can then later generically find that directory using a find command and then wrap that into your favourite scripting language

find ~/ -name .needle -print

LastLaunchServicesMap.plist

You can also find a file in the Library of the device directory of a simulator that contains the path of the last container. That file is under the path Library/MobileInstallation/LastLaunchServicesMap.plist. You can then print it using plutil -p LastLaunchServicesMap.plist, the key User contains information about each application in the simulator. Somehow it still seems to be missing some apps, but contains most of them.

Simulator Data Finder

To make my life easier, I built this app that leverages the last two methods to make it easy for you to find your simulators and files. The app can present you with a list of simulators and app as below. It then has a easy access button to finder, copy to clipboard or terminal for the path. It will try to find the directory using the LastLaunchServicesMap.plist file and if it still doesn’t find it, you can download this header file and add the following macro call in your applicationDidFinishLaunching,

You can download the app here. You can read here why I was unable to publish this app in the apple App Store which would have been more convenient.

One added bonus to the app is that it organises for easy access container you’d have downloaded from a device for a given app. It matches them by bundle identifier and currently looks for them in the download directory.

The path to rejection in the apple App Store

Posted on by
2

I have been developing apps on iOS for quite a while now as a side hobby. The latest Xcode has an ever changing directory for the Xcode simulator, which has been a bit painful to work around, despite a few tricks. So I decided it was a good opportunity for me to get into OS X application development and build a little tool that would let me organise and access files on the simulator for each app work on. I had been always writing my user interfaces fully programmatically so I also decided I would take this opportunity to explore interface builder and storyboard.

The first impression of OS X was that it’s actually not that different, some classes have different names and APIs but it’s remarkably similar. I felt right at home using NSTable. I got the basic of the app working very quickly and it instantaneously proved quite useful to my app development workflow. So much so that decided to share it on the AppStore as a free utility for other developers.

So on I go, paid my $99 to apple to become a Mac registered developer, as until know I only was an ios developer.

Sandboxing

First hurdle was that I had to sandbox the application. This meant that access to the terminal via AppleScript stopped working, but most significantly it wasn’t possible to access the simulator directory as it was outside the sandbox. I used the terminal as I wanted the app to let you pop up a finder window or a terminal window on the directory of a selected app.

Not to worry, according to the sandbox guideline you can access any directory as long as the user grant access by showing intent: all I had to do was to bring up a open dialog window on the directory where the simulator reside and once the user pressed ok, the app had access.

For the terminal, once you figure the syntax you could explicitly ask for access to terminal in the entitlement file. Et voila, all the functionality was working again in a sandboxed app.

There was though one major issue. Every time you pop up the open dialog, it would pop up a new duplicate window. It was really annoying and drove me nuts for a while. Without sandboxing it was all working fine, as soon as you turned on sandboxing that duplicate window kept coming up. I searched in the development forums, documentation, etc. I didn’t find any solution or mention of that problem. So I filed a bug report (easy to reproduce the problem with a very simple project) and used one of my support tickets (you get two as a registered developer) to ask an apple engineer for help.

Almost two weeks later I got the answer from apple. It is a bug on apple side, to solve it I should not use storyboard… Great. So I refactored all my code to use an xib file instead of storyboard and now it worked!

First Submission

Feeling all set to go, I purchased an icon on shutterstock, quickly added some help and documentation to the app and submitted it to the Apple Store for review. One week later, I got my first rejection. I so far never received a rejection for my iOS app, so it was quite disappointing. The feedback was that I used entitlement that I shouldnt.

I figured this must have been the access to terminal. Even though it seemed to use the standard method to request entitlement, I had always been a bit worried in the back of my mind that accessing terminal could be a security issue because it would let you execute potentially damaging commands.

It made sense and I removed that functionality and resubmitted the app. One week later another rejection with the same reason. You can’t be serious! Actually it was: silly me I had resubmitted the app without selecting the new binary, so they reviewed the same app I had submitted the first time. Another week wasted.

Final Rejection

I carefully selected the updated binary and resubmitted the app.

Almost two weeks later the app goes into review. There are typically two stages in a review: waiting for review, that in my previous experience takes a few days to a week, then it becomes in review, which takes a few hours to a day or two. Nice, I felt hopefully it will go through soon…

Not so fast. A few days pass. Then a week. Still in review. I almost forgot about it when after over three weeks, I start to wonder. Did something happen? I did some research on the Internet, app review times gives some average days to review of 8 or 9 days. I am way beyond… After a month of being in review I write to apple asking if there is any information they need from me to review the app. The answer comes quickly: no information needed, they are still reviewing the app and just had to be patient.

After more than once month I just received the final rejection. The app is rejected because “I am modifying user file in a way that is not publicly documented”. I am not modifying any file in the app! But it then hits me: apple just does not want me to write this app.

I write a last appeal to the reviewer: I do not modifying any file in my app, only bring up a finder window on a directory after the user has granted access and showed intent by clicking Authorize on an open dialog box. The final answer from the reviewer: we just don’t want app to access simulator files.

Here we go. Hard to not feel a bit of frustration and disappointment, almost three months after starting the app. I still believe this app is really useful and became an important part of my debugging workflow. So I put a bit of wrap around and made it available to download via this website and hope others will find it useful as well.

Garmin Connect New Policy Impact on ConnectStats and 3rd party apps

Posted on by
21

First I want to be clear that I have been a huge fan of Garmin’s for years. I own 6 devices and think they are extremely well done. I have also encouraged a lot of friends, colleagues and family to purchase Garmin devices over the years.

Monday March 2nd, Garmin announced they would stop making their API freely accessible and that 3rd party would have to pay $5,000 to access the data in Garmin connect.

I understand their infrastructure is costly to maintain but I feel such a steep fee risk killing the independent 3rd party applications which I believe adds value to Garmin customers and therefore Garmin as well.

I would propose a multi-tier system where 3rd party apps register and get a first limit of daily access for free, say 50,000 access per day. Heavy user breaching this limit would have to pay the 5,000$ fee.

I believe this system would be to both Garmin’s and its users benefit.

The Policy Change

On wednesday February 19th, Garmin updated its website which resulted in the approach used by most apps and website to access garmin connect data via an API to fail. The api used to have a license saying the following


This is the license file for GarminDeveloper.com. You are free
to access our API as long as you agree to create great things.

The link to that license has now been disabled http://connect.garmin.com/proxy/activity-search-service-1.2/downloads.html#License

On monday March 2nd, Garmin has notified developers that they are changing their policy and will charge $5,000 dollar fee for access to the API. Here is an extract of the message about the new policy.

Garmin has instituted a new policy regarding the accessibility of our developer programs. The previous strategy of freely available APIs quickly became unsustainable due to increasingly high demand. In response, we have established a new pathway for our Connect API. Firstly, there is a vetting process. We are strategically limiting the scope of this program to specific developers that will enhance the user experience. Secondly, those who are approved for inclusion will be charged one-time $5,000 administrative fee to cover the extensive engineering and server support required for the Connect program.

Consequence of the decision

This decision will likely force the eco system of independently developed 3rd party app to partially or completely disappear. This is a consequence which will affect customers. A lot of innovation can come from independent developers. Niche functionality can make its way to customers. It is likely not worth Garmin’s engineers’ time to cater to all customers software needs or custom data analysis needs. Enabling third party to provide for them helps make the Garmin devices more attractive. The size of the fee will dissuade a lot of experimentation for small players. The cost of entry is too high.

Most disturbingly it means it prevents people to use their own data in more creative ways than just using the data from the website. ConnectStats started as purely a tool for myself to look and slice my own data. I decided to share it as it could be useful to others, but right now it means I would not be able to use my own data in my code without paying $5,000, even if I pull the app from the store. This sounds wrong to me. Savvy users wanting to use a script to import their data efficiently for processing won’t be able to do that either.

In a world where the fitness market is exploding, I feel it would be to Garmin’s and its customers’ interest not to exclude this source of innovation for functionality.

My choices

I have the following choices for ConnectStats:

  1. kill the app. It would really pain me as part of the initial motivation was to provide a service for myself I couldn’t otherwise get: a more advanced viewer for my data on iPhone and iPad that what Garmin offers.
  2. swallow the cost. It’s a lot of money. I intentionally kept the purchase price at the minimum as to cover development cost and devices (money going back to Garmin by the way…) but my key motivation beyond my own use was sharing the app and not really generating large profits.
  3. increase the sell price of the app and hope people will continue to buy it. It somehow feels wrong to charge the users more just for the sake of them accessing the data they should own already. Right now if I increase the price from $.99 to $2.99 for ConnectStats, I am hoping to recover the Garmin fee over time. People pay several hundred dollars for their device. I hope they will accept the few extra dollars to pay Garmin for the right to access their data.
  4. switch ConnectStats to rely on another service. I am already working on that to use strava as primary download service and hope strava will pay the fee to get the data from garmin. This has some downside as some data won’t be available and Strava does not support swimming and skiing which I personally rely on.

Comparison to others services

I also wanted to comment on the fee structure proposed. A fee of $5000 is a lot of money by any standard. Niche independent Apps do not generate much revenue, mostly cover the costs. Here are some other services I use as comparison.

  • Apple charges $99 a year plus 30% of every sale to access their development tools and app distribution infrastructure. And this infrastructure and marketing power is of extremely high quality and adds a lot of value to independent developers like myself and to its customer. It wouldn’t occur to me to complain about this cost.
  • Strava provide an access to its data for free up to a certain rate limit once you are registered. Their API is extremely well documented, modern and their support and help to my questions have been great. I applaud Strava would provide a sophisticated website and API for free when their business model does not include revenue from any hardware.
  • Garmin support and documentation was so far inexistent. There are no or little documentation. I have never received an answer to any of my inquiries. Everything had to be reverse engineered. It doesn’t bother me at all given it was free but for $5,000 it’s a different issue.

I have been inquiring with Garmin about what exactly is provided when I pay the fee. What exactly do I get for that money: same API? Do I need to adapt my code to a new API? Do I get more documentation? Better support? Some marketing/promotion of the app as Apple provides?

Proposal

I believe the independent and small 3rd party partners add a lot of value to Garmin and the ability for users to access freely their own data is important. I understand the potential infrastructure cost and need to control usage for Garmin.

I suggest they offer a tiered system. Any user of the API would have to register and get some basic number of API access per day, say 50,000. Users above that limit would be required to pay the $5,000 fee to use the service. This would allow them to recoup some infrastructure cost from heavy user but still allow their customers to benefit from 3rd party functionality and access to their data.

What I plan to do for ConnectStats

I am doing 2 things:

  1. I have written to Garmin to explain why I feel a tier system would be better for all parties involved. I hope they will agree.
  2. ConnectStats is somewhat successful, receives good reviews and regular download. I hope by raising the price by an additional $2 this should help sponsor and recover the fee from Garmin. I have applied to be part of the program. Assuming Garmin approves my application, I plan to pay the fee myself and hope people will continue buying the app at the higher price to recover the money over time

Thanks for reading.