Moving weather data from DarkSky to Visual Crossing

ConnectStats history with the weather had a few chapter. First, I started to get the data from the Garmin Website so I could display the condition, the temperature and mostly the wind direction and speed in the map, which is always a good visual clue of how the wind helped or hurt in an activity.

When Garmin stopped publishing its weather data in the API, ConnectStats ended up without the weather for a while. Until Garmin opened its new API which required me to implement my own server, and indirectly made it easier to query the weather to display in an activity.

In March, Apple announced it was purchasing DarkSky and as a consequence it was going to shut down the API access to third party developer by next summer. So it was time to look for a new API. I will move the app to use Visual Crossing.

Looking for an alternative to DarkSky

Service providing weather data via an API are surprisingly expensive. At the time I used DarkSky, one of the reason was that the price was manageable for my use. I have been documenting the stats around ConnectStats usage, and for the weather, Connectstats makes about 70,000 calls per months at the last count. DarkSky charges by the unit of one dollar per 10,000 calls, so the cost is about $7/months or a few coffees, very manageable.

Most of the other services charge for a maximum number of queries, but the total cost is much higher. Another provider I had looked at when looking for DarkSky, OpenWeatherMap is charging $40/month for up to 10,000,000 calls per month, which would be plenty but the catch is that it’s only for current weather condition. First it’s quite expensive, second it does not work for ConnectStats as I needed the ability to query weather for a specific time at specific location, typically a time in the last day or so (the time the activity started). Pricing for historical queries was even more expensive than the standard API, mostly I would guess because these would likely be used by folks doing historical analysis and downloading lots of data. The minimum cost for my usage was $150/month for 5000 calls per day. Way more than I am willing to pay for my hobby, especially when the weather data is not core to ConnectStats.

Wind direction and speed next to the map contributed to Air Power when downwind…

Along came a blog post

As I was debating whether to simply give up on the weather, I came across a post about replacing darkSky api from Visual Crossing. I had never heard of them. A quick look at their website, indicated that you could for $25/months get unlimited access per day, subject to a concurrency limit, which was fine by me, I am not a heavy user, and the price, while a bit higher than DarkSky remained quite manageable. They also suggested in the blog post to reach out to get help and potential discount, so I did and explained my use case.

I have to say I have been very impressed with the quality of the interaction that followed. Very personalised interaction, and they did offer me a discount! I feel important to disclose it, I don’t believe there is a conflict of interest in writing positively about them and the discount, given based on my research and the pricing compared to my need, they are the vendor I would have gone with anyway. But I feel it’s important to be fully transparent.

The technical conversion

The API conceptually works very similarly to DarkSky and writing the code to integrate was very fast and easy.

Visual Crossing scored another point with me during my integration work: their API had some Time Zone requirements that were not straight forward to implement for me. The app gets GPS timestamps and I suspect a lot of users of their API use address queries and local time. When I mentioned it, they kindly replied within a few days that they had enhanced their API to accommodate my requirement and that I could now use GPS time stamps in the query. So they were not joking when they said in their blog post to reach out and they would help if needed…

While the code to integrate the API was easy, the tricky part is to make the switch in production given the co-dependence of the App release and the server release. Some coordination is required. There will be a few steps:

  1. the next release of the App (6.4) will contains code that works with both data from DarkSky and VisualCrossing.
  2. I will then upgrade the server to start quering both DarkSky and Visual Crossing
  3. Once I can confirm that the server processes the load, the server connection to visual crossing has no issue as it scales, and the app processes it fine, I will switch to only download from Visual Crossing.

I expect the transition will take in practice a few weeks assuming no issues.

As usual all the code is open source, and available on GitHub, both for the App and the server

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.