Trying to understand what was wrong in my Best Rolling Curve Calculation pushed my ability to use Xcode debugger to its limit. The quirks I was trying to understand were happening on large time series after multiple level of sampling, rolling averages, etc on noisy data.

I ended up in a mixed environment so I could easily exchange time series data between Xcode and pandas in a Jupyter notebook, which enabled me to explore and get to the bottom it!

You can read all about the problem I was trying to solved in the article, but to simplify it was to understand why after loading, processing and doing bunch of computation on GPS data loaded from a fit file I ended up with a curve like this

Traditional tools and approach

ConnectStats has most of the statistics and time series basic functionality I need implemented in Objective C. I started by adding a couple of function to display basic views of the data, and of course.

Of course the next useful tool is conditional breakpoint, if I want to understand out of the 10000 or so calculation what happens for index 29 where a specific calculation seem wrong is nice, but in my experience it was quite slow during the type of algorithm I was running (n^2) and of course the quirks where not happening on smaller “constructed” sample data… And above all it was very tricky to figure out which index had an issue and where it was really going wrong.

Adding logic with many debug printout, changing the graphs displayed in the app was cumbersome and slow. All tasks that are really nice and easy to do in python/jupyter with pandas.

Bridging Jupyter and iOS simulator

The idea of the bridge is very simple and consist in saving csv files in the data container of the Simulator. First I needed an easy way to save any series from the simulator into csv. Easy enough, nothing that implementing a -(NSString*)asCSVString couldn’t solve.

I wrapped the call to the calculation that had an issue into it’s own little XCUnitTest, so that I could easily rerun after any change and get all the updated csv files with a single press on Ctrl-Option-Command-G…

The only trick remaining was to get from the Jupyter notebook to the directory of the simulator. For that the handy xcrun utility saved the day, that I could use in the following function, which loads all the csv files in the simulator directory as a pandas with keys based on the filename

def load_csv_for_activities():
    stream = os.popen( 'xcrun simctl get_app_container booted net.ro-z.connectstats data')
    simdir = os.path.join( stream.read().rstrip(), 'Documents')
    files = os.listdir(simdir)
    csv = {}
    for file in files:
        if file.endswith('.csv') :
            short = file.replace('.csv','')
            elem = short.split( '_', 2 )
            if len(elem)==3:
                (fmt,name,activityId) = elem
                if activityId not in csv:
                    csv[activityId] = {}
                if fmt == 'tp':
                    csv[activityId][name] = read_trackpoints(simdir,file)
                elif fmt == 's':
                    csv[activityId][name] = read_timeserie(simdir,file)
    return csv

Now it was easy in python to explore quickly the raw data, reproduce calculation, for instance here trying to compute differences and look graphically at a range of the data

Or here use some of the pandas functions to identify quickly the index where a drop in value happened, which would have been quite a pain to implement in objective C just for the purpose of debugging…

Communicating this way between Xcode debugger and Jupyter made for a very efficient and quick workflow that enabled me to get to the bottom of the issue!

I hope this can give some ideas to other about how to debug iOS apps using python…